Lyster was born in Dublin, Ireland, the third son of Chaworth Lyster, a captain in the army, and his wife Anne, "née" Keightly. His uncle was William Saurin, Attorney-General for Ireland. The Saurin family was partly of French Hugenot extraction. Another of his uncles was James Saurin, Bishop of Dromore. At the age of 13, Lyster, after an illness, was sent on a voyage round the world and visited Sydney and Melbourne in 1842. After his return to England, he went to India intending to become a planter, but the climate not suiting him, he again returned to England. In 1847, he was in South Africa and fought in the 7th Cape Frontier War, and a year later, was in the United States where he tried his fortunes as an actor with little success.