He married Katherine Tyler of Boston, Massachusetts, widow of Captain David Ochterlony and mother of Sir David Ochterlony Bt, who rose to the rank of Major General in the East India Company.
He is mentioned in the poem "Epitaph on a tuft-hunter" by Thomas Moore 1779-1852:
Lament, lament, Sir Isaac Heard,
Put mourning round thy page, Debrett,
For here lies one who ne'er preferred
A Viscount to a Marquis yet.
Beside him place the God of Wit,
Before him Beauty's rosiest girls,
Apollo for a _star_ he'd quit,
And Love's own sister for an Earl's.
Did niggard fate no peers afford,
He took of course to peers' relations;
And rather than not sport a Lord
Put up with even the last creations;
Even Irish names could he but tag 'em
With "Lord" and "Duke," were sweet to call;
And at a pinch Lord Ballyraggum
Was better than no Lord at all.
Heaven grant him now some noble nook,
For rest his soul! he'd rather be
Genteelly damned beside a Duke,
Than saved in vulgar company.