Duff was born at Fetteresso Castle, Stonehaven, Kincardineshire, son of Arthur Duff (grandson of Robert Duff) and was educated at Blackheath School, London. He entered the Royal Navy in 1848, and was made a commander in 1865. In 1871, he married Louisa Scott and they had seven children: Helen, Robert W., Arthur, Heather, Isabel, Dorothy and Patrick.[1][2]
Duff was appointed Governor of New South Wales in March 1893 and was subsequently awarded the GCMG. He reached Sydney to take up his duties on May 29, 1893. In February 1895, he became ill while visiting Hobart and returned to Government House in Sydney, where he died of multiple hepatic abscesses and septicaemia on 15 March. He was interred in Waverley Cemetery. He is the only NSW Governor to have died in Office.[1]