Don't swallow keypresses following : in high load

We exit immediately, so we never get a chance to use them.