Arnoldstein's name derives from the alleged founder of its castle, one knight Arnold, probably a "ministerialis" serving the Bishops of Bamberg. Upon his coronation in 1014, Emperor Henry II had granted large estates in the Duchy of Carinthia to the Bamberg diocese, that originally had been a possession of the Patriarchs of Aquileia. The castle itself was first mentioned about 1085, here Bishop Otto of Bamberg established a Benedictine abbey in 1106 to secure his Carinthian fiefs. Nevertheless, in 1176, the monks had to accept the secular "Vogt" overlordship of the Carinthian Sponheim dukes. In the mid-14th century their estates were devastated both by the 1348 Friuli earthquake and the Black Death pandemic.