Can clean up ->caller more quickly in many cases.

This greatly alleviates the missing closure handling optimizations in
the GC so far; CORE.setting compiles in 73% of the time it previously
did with this.