From a Westphalian noble family, Romberg was born in his family castle in 1729. He started his career in the Prussian army by becoming a Gefreitenkorporal in Infanterieregiment Graf Wied (No. 41) in 1746. He was badly wounded at the Battle of Kolín in 1757. In 1773 he was put in command of a grenadier battalion and the following year he was made a member of the Pour le Mérite Order. In 1780 he was put in command of Infanterieregiment von Wolffersdorff (No. 9) under Karl Friedrich von Wolffersdorff - in the same year he was made an oberst.