Born to a former "samurai" family from Kokura (now part of Kitakyushu City), Fukuoka Prefecture, Sugiyama graduated from the 12th class of the Imperial Japanese Army Academy in 1901. He served as a junior officer with 3rd Battalion of the 14th Regiment of the IJA 12th Division in the Russo-Japanese War, and was wounded in the face during the Battle of Shaho. Due to scars from that injury, he was unable to fully open his right eye.