Cypripedium calceolus
Whittaker lived in the small village of Morley near Derby at Ferriby Brook (the name of his house) with his wife Mary in the late 1850s. He continued to teach here, taking up to twelve scholars into his classes, usually after they had left the local school, and were between eight and eighteen years of age. He eventually established a large collection of living plants. Local horticultural groups reported he was growing over 1,300 different species. In 1864 he was publishing works noting the local extinction of the lady's slipper orchid, "Cypripedium calceolus", with the Rev. Henry Harpur Crewe. By 1871 he was no longer a schoolmaster, and is then listed as a "seedsman and florist". By 1881 these returns described him as a "nurseryman and farmer", with two servants living in his house, plus twenty-year-old William Whitehead. Whitehead was to eventually become a partner in their market gardening venture. Whittaker was elected an 'Associate' of the Botanical Society of Edinburgh, and remained on its membership listings from at least 1881 until 1891.