Tosa Domain
Yamaji was born in Tosa Domain (present day Kōchi Prefecture) in what is now part of the city of Kōchi, where his father was an upper-ranked samurai in the service of the Yamauchi clan. At the age of 13, he lost sight in one of his eyes, but notwithstanding his disability, he was appointed a company commander of the "Jinshotai", a Tosa-Domain shock force, during the Boshin War of the Meiji Restoration, participating in the Battle of Toba–Fushimi, and in subsequent campaigns in northern Japan against the pro-Tokugawa Ōuetsu Reppan Dōmei. During the course of the conflict, he was promoted to company commander, and awarded a stipend of 150 "koku".