Kinder was born in London, United Kingdom, on 17 September 1819, the oldest surviving child of a wealthy merchant and his second wife. In 1838 Kinder attended Cambridge University originally studying Mathematics but Classics and Theology were where his true interests lay. He graduated in 1845 with a MA. While at Cambridge he joined the Cambridge Camden Society which was a group interested in church architecture. Kinder received his Anglican priesthood in 1845 and became an ordained deacon in 1846. In 1848, Kinder was the new Master at Alleyne's Grammar School where he remained for eight years. However, after he was petitioned to be removed from his post due to his strict religious practices, he was interviewed by Bishop G. A Selwyn to relocate to Auckland, New Zealand to be the headmaster of the Church of England Grammar School. Kinder began his journey to New Zealand in July 1855. He moved into the headmasters house across the road from the school which is now a heritage site known as Kinder House. During the New Zealand Land Wars of the 1860s, Kinder was a Chaplain to the British Forces. In 1872, Kinder became the Master of St John's College in Tamaki. Kinder received a Doctor of Divinity in 1873 by the Archbishop of Canterbury.