 Allen was the son of John and Sarah Allen. His father was a native of Connecticut and of English origin, and his mother was originally from Vermont and of Scottish descent. He left home when he was eight years old "to carve out a fortune for himself", as Sylvester put it in his History of Rensselaer County, New York. He began work in an auger factory in Hamden, Connecticut, at an early age and continued until he was seventeen years old. At that time, he purchased a clock store, which he traded for horses in Vermont and Canada. He shipped his horses to the West Indies, but the ship was wrecked and his horses were lost, thus placing him badly in debt. However, he soon obtained credit, and again shipped a number of horses to the West Indies, this time successfully, enabling him to pay all his debts and even earning him a profit.
