Orr was born at Craytonville, South Carolina located in Anderson County, South Carolina.He graduated at the University of Virginia in 1841 and became an attorney. He served as a DemocraticCongressman from South Carolina from 1849 to 1859, serving as the Speaker of the House from 1857 to 1859. Congressman Orr was an advocate of states' rights who used his position of power to assist those persons who promoted the continuation of slavery. He foresaw the dreadful consequences of the decision by South Carolina to attempt to secede from the Union, but he remained loyal to that State. He was one of the three commissioners sent to Washington, D.C. to effect the transfer of property to the State of South Carolina.
Afterward, he organized and commanded Orr's Regiment of South Carolina Rifles, which saw little action before he resigned in 1862 and entered the Confederate Senate. Ironically, the regiment continued to bear his name throughout the war and fought in some of the most prominent battles of the Army of Northern Virginia. In the Senate, he remained a strong proponent of states' rights.
At the end of the war, Orr was elected governor in 1865 of the state of South Carolina and served until 1868 with the passage of a new state constitution. He died in St. Petersburg, Russia shortly after being named Minister to Russia by President Ulysses S. Grant. He is interred in the First Presbyterian Church Cemetery in Anderson, SC.