Rufus Frank Milwee (March 16, 1872 – January 29, 1955) was an American politician. He was a member of the Arkansas House of Representatives, serving from 1919 to 1925. He was a member of the Democratic party.