Yang Xuangan
Li Mi came from a line that was part of the nobility during successive dynasties Western Wei, Northern Zhou, and Sui Dynasty. By the time of his father Li Kuan (), who carried the Sui-created title of Duke of Pushan, the clan, although not originally from there, lived at the Sui capital Chang'an. Due to his father's position, Li Mi became a guard of Emperor Yang of Sui, and he was said to view money lightly, using it to instead gather friends around him. One day, however, when Emperor Yang saw him, Emperor Yang was apprehensive of his appearance, and told his associate Yuwen Shu the Duke of Xu to have Li Mi removed. Thereafter, Yuwen persuaded Li Mi to resign from the imperial guard corps, and Li instead often traveled around the capital, riding a bull and reading while doing so—particularly the "Book of Han". When the prime minister Yang Su the Duke of Yue saw him, he was surprised by Li Mi's studiousness, and as Li knew it was the prime minister, he respectfully bowed and stated his name. Yang Su took Li to his mansion and talked with him. Impressed, he told his son Yang Xuangan, "Li Mi's intelligence and capability is far beyond yours." Thereafter, Yang Xuangan and Li Mi became friends. Sometimes, Yang Xuangan would bully Li, and Li responded: