Remove even more allocations that can happen every frame from camera updates.