Remove some failure

Avoid reborrowing a RefCell over and over since it'll bloat at least the
IR and maybe the resulting binary with failure cruft.