Ellis and her husband had three sons: John William (1853–1918), Sidney Alexander (1856–1857) and Sidney Thomas (1858–1864). On 31 March 1859, Ellis, her husband and her two surviving sons, together with her 19-year-old brother, Tom Colebrook, and her 18-year-old cousin, John Drew Colebrook, sailed to New Zealand. They had been told that their youngest son's health might improve in a warmer climate. Her husband preferred India, but she was concerned at reports of uprisings. They arrived in Auckland on 16 July 1859. They were poor during their first few years in Auckland. She went to Governor Thomas Gore Browne's peace conference at Kohimarama in 1860 and wrote in support of Māori interests. She encouraged her sons to learn Māori and play with Māori children, so that John became an interpreter, then a teacher. Her husband disapproved of her opinions on Māori and the role of women, while she disapproved of his drinking and control over the family finances.