Repository: godotengine/godot (language: cpp)

Issue:
Occlusion Culling on OpenXR is unstable (meshes flickering) in Godot4.4-stable
### Tested versions

- Reproducible in Godot v4.4-stable, Godot v4.4.1-rc1
   - Meta Quest 3
   - X11+SteamVR/OpenXR Vulkan (mobile and forward+)
       - OpenXR: Running on OpenXR runtime:  SteamVR/OpenXR   2.9.6
       - OpenXR: XrGraphicsRequirementsVulkan2KHR:
         - minApiVersionSupported:  1.0.0
         - maxApiVersionSupported:  1.2.0

- Not Reproducible in Godot v4.3-stable

### System information

Godot v4.4.stable - Ubuntu 22.04.5 LTS 22.04 on X11 - X11 display driver, Multi-window, 1 monitor - Vulkan (Mobile) - dedicated NVIDIA GeForce RTX 3050 Laptop GPU - 12th Gen Intel(R) Core(TM) i7-12700H (14 threads)

### Issue description

https://github.com/user-attachments/assets/b1f55edf-e9d9-4691-80e5-fc0363f7f0fc

When Occlusion Culling is enabled in Godot 4.4-stable some meshes are mysteriously culled. Only Reproducible when using OpenXR and `root.use_xr == true`

### Steps to reproduce

* Extract minimal reproduction in OpenXR
* install [Godot Openxr vendors addon](https://github.com/GodotVR/godot_openxr_vendors/releases/) in 4_4/addons and 4_3/addons
* Run 4_4/project.godot or 4_3/project.godot

### Minimal reproduction project (MRP)

[xr-ocull-broken.zip](https://github.com/user-attachments/files/19265466/xr-ocull-broken.zip)


Discussion / hints:
Also noticed that occlusion culling only runs on one of the two eyes when using stereoscopic rendering.. probably fine?

bisect:
v4.4.dev3.official [f4af8201b] GOOD
v4.4.dev4.official [36e6207bb] GOOD
suspects:

* Fix Occlusion Culling Buffer occasionally getting corrupted ([GH-98758](https://github.com/godotengine/godot/pull/98758)).
0f0e24016ceb11b2503447bb70bce723322dcb7c BAD
0f0e24016ceb11b2503447bb70bce723322dcb7c^ GOOD

v4.4.dev5.official [9e6098432] BAD
v4.4.dev7.official [46c8f8c5c] BAD


I'm struggling to set up the project (first time using VR). Is it required to have hardware for the stereoscopic view? I'd appreciate any help.

Additionally does disabling jitter (Rendering->Occlusion Culling->Jitter Projection) have any affect?
Hm. I wonder if this relates to how the projection from OpenXR can have an "asymmetric fov"? Basically, the center of the projection may not be the center of the viewport.

The comment in `Projection::get_viewport_half_extents()` seems to indicate that this could be a problem:

```
Vector2 Projection::get_viewport_half_extents() const {
	// NOTE: This assumes a symmetrical frustum, i.e. that :
	// - the matrix is a projection across z-axis (i.e. is invertible and columns[0][1], [0][3], [1][0] and [1][3] == 0)
	// - the projection plane is rectangular (i.e. columns[0][2] and [1][2] == 0 if columns[2][3] != 0)
	// - there is no offset / skew (i.e. columns[2][0] == columns[2][1] == 0)
	real_t w = -get_z_near() * columns[2][3] + columns[3][3];
	return Vector2(w / columns[0][0], w / columns[1][1]);
}
```
> Hm. I wonder if this relates to how the projection from OpenXR can have an "asymmetric fov"? Basically, the center of the projection may not be the center of the viewport.

Pretty sure this is it, but I don't understand the projection matrix for OpenXR well enough yet to figure out how to fix the current implementation

oh, the code is using the **camera's transform** and _guessing_ the coordinates of the near plane rectangle based on that..

does the perspective transform add some translation for OpenXR for left  vs right eye on top?

Normal 

```
CAM_PROJ [X: (0.733064, 0.0, 0.0, 0.0), Y: (0.0, 1.303225, 0.0, 0.0), Z: (0.0, 0.0, -1.000025, -1.0), W: (0.0, 0.0, -0.100001, 0.0)]
cam center tfm (-5.982937, 2.307015, 0.085401)
top corner world (-6.051144, 2.345381, 0.035401)
right corner world (-5.91473, 2.268649, 0.035401)

projection.get_endpoints(p_cam_transform, endpoints);
proj: ep 4 (top corner world): (-6.051144, 2.345381, 0.035401)
proj: ep 7 (right corner world): (-5.91473, 2.268649, 0.035401)
proj: tcw - ep4 (0.0, 0.0, 0.0)
proj: rcw - ep7 (0.0, 0.0, 0.0)
```

OpenXR

```
CAM_PROJ [X: (0.726552, 0.0, 0.0, 0.0), Y: (0.0, 0.835475, 0.0, 0.0), Z: (0.000007, -0.193193, -1.000012, -1.0), W: (0.0, 0.0, -0.046693, 0.0)]
cam center tfm (-5.982937, 2.307015, 0.085401)
top corner world (-6.015212, 2.335723, 0.063205)
right corner world (-5.950646, 2.280228, 0.060944)

projection.get_endpoints(p_cam_transform, endpoints);
proj: ep 4 (top corner world): (-6.003316, 2.337547, 0.106775)
proj: ep 7 (right corner world): (-5.939691, 2.282415, 0.105469)
proj: tcw - ep4 (-0.000015, 0.00533, 0.000275)
proj: rcw - ep7 (-0.000015, 0.00533, 0.000275)
```

Normal
![Image](https://github.com/user-attachments/assets/a15cad8e-85b4-48dd-931e-340cad48c926)

OpenXR
![Image](https://github.com/user-attachments/assets/6d145c17-301d-47d5-9b6b-e7b9f628b5f9)

Fix patch:
diff --git a/modules/raycast/raycast_occlusion_cull.cpp b/modules/raycast/raycast_occlusion_cull.cpp
index 9f95307cf1c5..88691be8a382 100644
--- a/modules/raycast/raycast_occlusion_cull.cpp
+++ b/modules/raycast/raycast_occlusion_cull.cpp
@@ -525,14 +525,14 @@ void RaycastOcclusionCull::buffer_set_size(RID p_buffer, const Vector2i &p_size)
 	buffers[p_buffer].resize(p_size);
 }
 
-Vector2 RaycastOcclusionCull::_jitter_half_extents(const Vector2 &p_half_extents, const Size2i &p_viewport_size) {
+Vector2 RaycastOcclusionCull::_get_jitter(const Rect2 &p_viewport_rect, const Size2i &p_buffer_size) {
 	if (!_jitter_enabled) {
-		return p_half_extents;
+		return Vector2();
 	}
 
 	// Prevent divide by zero when using NULL viewport.
-	if ((p_viewport_size.x <= 0) || (p_viewport_size.y <= 0)) {
-		return p_half_extents;
+	if ((p_buffer_size.x <= 0) || (p_buffer_size.y <= 0)) {
+		return Vector2();
 	}
 
 	int32_t frame = Engine::get_singleton()->get_frames_drawn();
@@ -568,8 +568,8 @@ Vector2 RaycastOcclusionCull::_jitter_half_extents(const Vector2 &p_half_extents
 			jitter = Vector2(0.5f, 0.5f);
 		} break;
 	}
-
-	jitter *= Vector2(p_half_extents.x / (float)p_viewport_size.x, p_half_extents.y / (float)p_viewport_size.y);
+	Vector2 half_extents = p_viewport_rect.get_size() * 0.5;
+	jitter *= Vector2(half_extents.x / (float)p_buffer_size.x, half_extents.y / (float)p_buffer_size.y);
 
 	// The multiplier here determines the jitter magnitude in pixels.
 	// It seems like a value of 0.66 matches well the above jittering pattern as it generates subpixel samples at 0, 1/3 and 2/3
@@ -578,7 +578,16 @@ Vector2 RaycastOcclusionCull::_jitter_half_extents(const Vector2 &p_half_extents
 	// False shown can lower percentage that are occluded, and therefore performance.
 	jitter *= 0.66f;
 
-	return p_half_extents + jitter;
+	return jitter;
+}
+
+Rect2 _get_viewport_rect(const Projection &p_cam_projection) {
+	// NOTE: This assumes a rectangular projection plane, i.e. that:
+	// - the matrix is a projection across z-axis (i.e. is invertible and columns[0][1], [0][3], [1][0] and [1][3] == 0)
+	// - the projection plane is rectangular (i.e. columns[0][2] and [1][2] == 0 if columns[2][3] != 0)
+	Size2 half_extents = p_cam_projection.get_viewport_half_extents();
+	Point2 bottom_left = -half_extents * Vector2(p_cam_projection.columns[3][0] * p_cam_projection.columns[3][3] + p_cam_projection.columns[2][0] * p_cam_projection.columns[2][3] + 1, p_cam_projection.columns[3][1] * p_cam_projection.columns[3][3] + p_cam_projection.columns[2][1] * p_cam_projection.columns[2][3] + 1);
+	return Rect2(bottom_left, 2 * half_extents);
 }
 
 void RaycastOcclusionCull::buffer_update(RID p_buffer, const Transform3D &p_cam_transform, const Projection &p_cam_projection, bool p_cam_orthogonal) {
@@ -595,11 +604,12 @@ void RaycastOcclusionCull::buffer_update(RID p_buffer, const Transform3D &p_cam_
 	Scenario &scenario = scenarios[buffer.scenario_rid];
 	scenario.update();
 
-	Vector2 viewport_half = p_cam_projection.get_viewport_half_extents();
-	Vector2 jitter_viewport_half = _jitter_half_extents(viewport_half, buffer.get_occlusion_buffer_size());
-	Vector3 near_bottom_left = Vector3(-jitter_viewport_half.x, -jitter_viewport_half.y, -p_cam_projection.get_z_near());
+	Rect2 vp_rect = _get_viewport_rect(p_cam_projection);
+	Vector2 bottom_left = vp_rect.position;
+	bottom_left += _get_jitter(vp_rect, buffer.get_occlusion_buffer_size());
+	Vector3 near_bottom_left = Vector3(bottom_left.x, bottom_left.y, -p_cam_projection.get_z_near());
 
-	buffer.update_camera_rays(p_cam_transform, near_bottom_left, 2 * viewport_half, p_cam_projection.get_z_far(), p_cam_orthogonal);
+	buffer.update_camera_rays(p_cam_transform, near_bottom_left, vp_rect.get_size(), p_cam_projection.get_z_far(), p_cam_orthogonal);
 
 	scenario.raycast(buffer.camera_rays, buffer.camera_ray_masks.ptr(), buffer.camera_rays_tile_count);
 	buffer.sort_rays(-p_cam_transform.basis.get_column(2), p_cam_orthogonal);
diff --git a/modules/raycast/raycast_occlusion_cull.h b/modules/raycast/raycast_occlusion_cull.h
index a831336df53a..51c1a196eaf0 100644
--- a/modules/raycast/raycast_occlusion_cull.h
+++ b/modules/raycast/raycast_occlusion_cull.h
@@ -161,7 +161,7 @@ class RaycastOcclusionCull : public RendererSceneOcclusionCull {
 	bool _jitter_enabled = false;
 
 	void _init_embree();
-	Vector2 _jitter_half_extents(const Vector2 &p_half_extents, const Size2i &p_viewport_size);
+	Vector2 _get_jitter(const Rect2 &p_viewport_rect, const Size2i &p_buffer_size);
 
 public:
 	virtual bool is_occluder(RID p_rid) override;


Test patch: