Originally from Pitten in present-day Lower Austria, the family's ancestors took their residence at Stubenberg in the March of Styria about 1160. One Ulrich of Stubenberg is documented as a participant of the Fifth Crusade, killed at Damietta in 1218. The Stubenbergs established contacts to the rising House of Habsburg at an early stage and revolted against the rule of King Ottokar II of Bohemia, who finally was defeated by his Habsburg rival King Rudolf I of Germany in the 1278 Battle on the Marchfeld. Under Habsburg rule, the family members were able to restore their devastated estates; in the mid 14th century they had Neuhaus Castle ("Burg Neuhaus") erected near Stubenberg and built Stubenberg Castle as their new residence.