Remove some workarounds for rustc borrow check bugs. Makes the CPU simpler.