Emperor Gaozong of Tang
It is not known when Xue Song was born, but he was said to be born in the modern Beijing region. His grandfather Xue Rengui was a well-known general during the reign of Emperor Gaozong of Tang, and his father Xue Chuyu () served as a general as well. Despite Xue Song's coming from a distinguished lineage, it was said that he paid no attention to property management and fell into poverty. It was also said that he was strong and capable in riding and archery, but paid no attention to studies.