Grey was the fourth son of Edward Grey, bishop of Hereford while his mother was the daughter of James Croft of Greenham Lodge, near Newbury, Berkshire. Grey studied at Christ Church, Oxford, but did not complete studies and joined the war office as a clerk. Nominated to work in the Bengal civil service he entered Haileybury College in January 1839 and passed out in July 1840. During his first term, he was rusticated for late night parties in his room but made up for it in later terms. He went to Bengal in 1840 and worked for a while in various offices before becoming a private secretary to Sir Herbert Maddock in 1845. In 1851 he became secretary of the Bank of Bengal. He became a secretary to the government of Bengal in 1854. He went on furlough to England in January 1857 but was forced back by the rebellion in that year. He was appointed director general of the post office in 1859 after holding various temporary offices. In 1861 he became a member of the council of the governor-general of Bengal. In 1867 he became lieutenant-governor of Bengal.