The Ortenburg inherited large estates in the Drava valley upon the extinction of the Lurn counts in 1135. In 1192 Count Otto II of Ortenburg and his brother Hermann founded the settlement of Spittal below the castle. Otto II is also mentioned as a participant of the Crusade of Henry VI in 1195. In 1276 the Ortenburgs sided with Count Meinhard II of Gorizia-Tyrol and the newly elected Habsburg king Rudolf I of Germany against the occupying forces of King Ottokar II of Bohemia. Damaged by the 1348 Friuli earthquake, however, the significance of the comital castle diminished in favour of the nearby Spittal residence.