Fix some double-frees/use-after-free

r? @frank-trampe
Yes but there is already a goto for the failing case, so I figured it's better to use that.