Bannister was appointed the first attorney-general of New South Wales in March 1823, and he arrived in Sydney early in 1824. On 17 May1824 he was sworn in at the first sitting of the Supreme Court of New South Wales. He had been given a salary of £1200 a year with the right to practise as a barrister, but he became discontented with his position, and in October 1825 was in conflict with Governor Thomas Brisbane on the question whether he was bound to draft a bill which seemed to him to be repugnant to the laws of England. He appears to have taken his office and his responsibilities far too seriously, and though Governor Ralph Darling spoke of Bannister as "often misled by an injudicious zeal, but indefatigable, conscientious and honourable in the highest degree", he found it extremely difficult to work with him. In September 1826, in a dispatch to under-secretary Hay, Darling described one of Bannister's letters to the governor as "very offensive and insolent".
In April 1826 Bannister wrote to Darling to say that he could no longer hold his office at its present remuneration, and on 13 October1826 he was informed that his resignation had been accepted. This furnished Bannister with a grievance for the rest of his long life. He left for England on 22 October 1826 and afterwards did a large amount of writing; the British Museum Catalogue lists about 30 of his publications. Many are pamphlets but among the longer works are: Statements and Documents relating to Proceedings in New South Wales in 1824, 1825 and 1826 (1827), Humane Policy; or Justice to the Aborigines (1830), British Colonization and Coloured Tribes (1838), and William Paterson, the Merchant Statesman (1858).