He was chosen as the Leader of the Opposition in the Rajya Sabha on 3 June 2009 by L.K. Advani. On 16 June 2009, he resigned from the post of General Secretary of BJP as per his party's One Man One Post principle. He was also a member of the Central Election Committee of the party. As the leader of opposition in Rajya Sabha, he participated in the debates on the Women's Reservation Bill in the Rajya Sabha and also supported Anna Hazare in his 2011 Indian anti-corruption movement for the Jan Lokpal Bill. He successfully introduced the eighty-fourth amendment to the Constitution of India in 2002 freezing parliamentary seats until 2026 and the ninety-first amendment to the Constitution of India in 2004 penalising defections. However, being in the party since 1980 he never contested any direct election until 2014. He was the BJP candidate for the Amritsar seat in the Lok Sabha (replacing Navjot Singh Sidhu) for the 2014 general election, but lost to the Indian National Congress candidate Amarinder Singh. He was elected as a Rajya Sabha member from Gujarat. He was re-elected to the Rajya Sabha from Uttar Pradesh in March 2018.