Lord Revelstoke married Louisa Emily Charlotte, daughter of John Crocker Bulteel, in 1861. They had seven sons and three daughters. Their fifth was the man of letters Maurice Baring. Lady Revelstoke died in 1892. Lord Revelstoke survived her by five years and died in July 1897, aged 69. He was succeeded in the barony by his second but eldest surviving son John.