He was of English, Scots-Irish, and Irish ancestry. He had a brother William, four years his senior, and an older sister Elizabeth, who died in childhood. Johnson's birth in a two-room shack was a political asset in the mid-19th century, and he would frequently remind voters of his humble origins. Johnson, due to national interest in new railroad construction and in response to the need for better transportation in his own district, also supported government assistance for the East Tennessee and Virginia Railroad. If Johnson considered retiring from politics upon deciding not to seek reelection, he soon changed his mind. In his first biennial speech, Johnson urged simplification of the state judicial system, abolition of the Bank of Tennessee, and establishment of an agency to provide uniformity in weights and measures; the last was passed. Johnson was critical of the Tennessee common school system and suggested funding be increased via taxes, either statewide or county by county—a mixture of the two was passed.