Alencar was born into a family of small entrepreneurs from Muriaé, in the inland state of Minas Gerais on October 17, 1931, he was the eleventh son of Antônio Gomes da Silva and Dolores Peres Gomes da Silva. He started working while still a child, helping out his father in the family business, and then worked with his brothers until he became a successful businessman on his own. In 1967, Alencar founded Coteminas, one of the largest Brazilian textile companies.
Before being elected for the Senate in 1998, Alencar ran for governor of Minas Gerais in 1994. While in Senate, José Alencar worked on several committees, including Economic Matters and Social Matters. In 2002 he was tapped to be Lula's running mate, in an effort to assuage worries about the then-candidate's supposed anti-business bias. He was the Honorary President of center-right Liberal Party, but left his party at the end of 2005, to associate himself with the Brazilian Republican Party.
In office Alencar has often criticized his own administration for failing to lower interest rates. In November 2004 he was sworn in as Defense Minister, following the resignation of José Viegas Filho. He tried to resign on several occasions, claiming that a businessman would hardly be the best choice for running a nation's military forces. Nevertheless, President Lula convinced him to stay until March 2006, when Alencar did resign his ministerial post.
Despite his disagreements with some Lula administration policies, Alencar was officially invited by Lula to once again be his running mate in the 2006 general elections. With Lula's re-election victory on October 29, Alencar secured his position as Vice President for another term.