Move the world around instead of the camera again.
Simplify the camera matrix operations.