Born in Linz, Ulmer joined the Frank Stronach Akademie to make his professional debut for Austria Wien in the 2004–2005 season. In summer 2008, he joined SV Ried. On 28 January 2009, it was announced that Ulmer was transferred to Red Bull Salzburg with immediate effect. During the 2017–18 season, Salzburg had their best ever European campaign. They finished top of their Europa League group, for a record fourth time, before beating Real Sociedad and Borussia Dortmund thus making their first ever appearance in the UEFA Europa League semi-final. On 3 May 2018, he played in the Europa League semi-finals as Olympique de Marseille played out a 1–2 away loss but a 3–2 aggregate win to secure a place in the 2018 Europa League Final.