Ensure REPL will start with compilation errors.

Add some gymnastics to make sure we can get the REPL up even with errors
in our code.