Internationally, Halldórsson had four consecutive wins at the European Cup qualifiers from 1975 to 1981, including a meet record of . His first major competition was the 1974 European Athletics Championships, but the 25-year-old Halldórsson failed to make it out of the qualifying round. After a ninth-place finish at the 1976 European Athletics Indoor Championships, he entered the 1976 Summer Olympics, where he came 15th in the qualifying round. He topped the podium at the 1977 European Athletics Indoor Championships, taking the first and only continental gold medal of his career. This made him the first Icelandic medallist and champion of the competition and he remains he only male to have topped the podium (Vala Flosadóttir won the women's pole vault in 1996). Later that year he threw a personal best of , which ranked him fourth in the world and was an Icelandic national record (he remains the country's second best shot putter after Pétur Guðmundsson).