Turns out exit code 2 also means the puppet agent run went fine, who knew?