In 1926 before entering in Allahabad University Ghosh met with Bhagat Singh and Batukeshwar Dutt. He was a member of Hindustan Socialist Republican Association. He was arrested and latter imprisoned after Lahore Conspiracy Case trial in 1929 but released due to lack of evidence. He was again arrested in 1931 and came into contact with Srinivas Sardeshai in jail. After release he joined in the Communist Party of India. In 1934, he was elected to the Central Committee of the CPI and in 1936 he was elected to its Polit Bureau. In 1938, Ghosh became the member of the editorial board of the Party's mouthpiece, the "National Front". He was the General Secretary of the Communist Party of India from 1951 till his death in 1962. He was leading the Communist Party of India during the China-India war in 1962 and supported India's position instead of that of the People's Republic of China. He was the prominent person in the centrist faction before the split of the Communist Party of India (Marxist) from the Communist Party of India.