Swift was born at Fort Laramie, Wyoming, the son of Major General Eben Swift and Susan Palmer. He graduated from West Point in 1904 and was commissioned in the cavalry. He served as aide-de-camp to General John J. Pershing in the Philippines and then served in Mexico. While a First Lieutenant commanding C Troop, 13th Cavalry, he accompanied First Lieutenant George S. Patton on the hunt for Julio Cardenas, commander of Pancho Villa's personal bodyguard. During World War I he served as Assistant Chief of Staff for the 86th Division.