Prevent interaction and FOV updates when the camera is active