Ord was born in Cumberland, Maryland, the son of James and Rebecca Ord. Family tradition made James Ord the illegitimate son of George IV of the United Kingdom and Maria Fitzherbert but he seems likely to have been the son of Ralph Ord, who was baptised at Wapping, Middlesex, in 1757, the son of John Ord, a factor (agent) from Berwick-upon-Tweed. Edward Ord was considered a mathematical genius and was appointed to the United States Military Academy by President Andrew Jackson. His roommate at West Point was future general William Tecumseh Sherman. He graduated in 1839 and was commissioned a second lieutenant in the 3rd U.S. Artillery. He fought in the Second Seminole War in Florida and was promoted to first lieutenant.