Repository: godotengine/godot (language: cpp)

Issue:
Physics interpolation is broken on XR nodes and XRServer
### Tested versions

- Reproducible in v4.4 (0e3dbba6a)

### System information

Godot v4.4.rc.mono (0e3dbba6a) - Windows 11 (build 26100) - Multi-window, 1 monitor - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4090 (NVIDIA; 32.0.15.7247) - AMD Ryzen 9 7950X3D 16-Core Processor (32 threads)

### Issue description

Physics interpolation is broken in XR in general, causing the following issues:
- `XROrigin3D` updates `XRServer::world_origin` only when the transform changes, which means `XRServer::world_origin` never has the interpolated value for the transform on each render frame. `XRServer::world_origin` is used to know where the camera is in the world and to render properly, if it isn't also interpolated then there's is a mismatch between the active `XROrigin3D` transform and what `XRServer` has saved. This results in the following behavior:

https://github.com/user-attachments/assets/5e9d7764-7c03-4a29-bc98-de86d0451f9f


- All XR nodes that have a `XRPositionalTracker` will update their position every time the `"pose_changed"` signal is triggered. This happens every time `XRServer::_process` gets called because it asks the active XR interface to poll events and it happens every render frame. Tracked nodes being updated every frame make sense because if we instead poll events on each physics frame and interpolate them the motion sickness on something like the HMD would be unreal, and you don't want your hand trackers to lag behind your real hands. However, when interpolation is enabled the engine really doesn't want you to be updating your transforms on render frame and this causes the childs of those tracked nodes to jitter a bit.

Here is an example in the attached MRP, when the "hand" is interpolated, it follows the parent just fine but the childs aren't smooth at all (also spams in the console the warning that an interpolated transform was set outside physics process)

https://github.com/user-attachments/assets/4f32e267-3b35-41c2-8659-c5fbe2b96330

Disabling interpolation in the hand results in really smooth child movement but then when it comes to following the parent everything falls apart:

https://github.com/user-attachments/assets/51271cb6-32ee-4773-b4d9-4141c55fd1a9

I created a WIP pull request (https://github.com/godotengine/godot/pull/103233) with a fix for the `XRServer::world_origin` interpolation where it works as expected:

https://github.com/user-attachments/assets/107ca829-afc1-4875-827d-a34c7602ee83

However for the second issue I'm out of ideas about how to proceed. Disabling interpolation on the tracked node makes it jitter because it will now follow the non interpolated transform of the parent even if the parent is set as interpolated.

### Steps to reproduce

Enable interpolation in a project and make XROrigin3D a child of any physics object

### Minimal reproduction project (MRP)

It can be reproduced on the official `openxr_character_centric_movement` demo, modifying `start_vr.gd` to remove the line `Engine.physics_ticks_per_second = current_refresh_rate` so the physics FPS can be different than the target framerate
https://github.com/godotengine/godot-demo-projects/tree/master/xr/openxr_character_centric_movement
However here I include a modified version that just adds a moving platform where you can easily see the problem

[MRP.zip](https://github.com/user-attachments/files/18934793/MRP.zip)

Here is an MRP to test the tracker issue with a fake hand for anyone who wants to test without XR hardware:

[tracking_mrp.zip](https://github.com/user-attachments/files/18947247/tracking_mrp.zip)


Discussion / hints:
I don't have any XR devices so can't seem to run the MRP, but a function that can probably help you is:
https://docs.godotengine.org/en/3.6/tutorials/physics/interpolation/advanced_physics_interpolation.html#get-global-transform-interpolated

Regarding the PR, you can do physics interpolation either client (scene) side or on the server. Sometimes we do it on the client side and sometimes server side.

Client side is often easier to implement (and indeed the mechanism is already built in to help do this via `get_global_transform_interpolated()`, although care has to be taken over priming). This function is primarily for users, but can also be used by core. Although having said that, in `Cameras` we do this manually as it gets around priming issues - it depends whether the node will always be active or will be switched on and off.

`Cameras` is a good example to see how they do it if you want to try client side, probably the 3D camera as it is slightly simpler. You may want to use `NOTIFICATION_INTERNAL_PROCESS` and `NOTIFICATION_INTERNAL_PHYSICS_PROCESS`.

The reason for doing it server side is, in most cases, for efficiency. So for e.g. large number of canvas items, or 3d nodes, we do the interpolation on the server, because it potentially is more efficient by e.g. avoiding lots of calls to `set_transform()` across the command queue on every frame, and is especially more efficient when the frame rate is higher than the tick rate.

On the other hand for single special items that there are only one or two of in a scene at once (e.g. `Cameras`) we do sometimes do the interpolation client side, as there is no significant loss to efficiency, and it keeps the server side code as simple as possible (indeed the overall code complexity is less).

I suspect for your XR origin it may be less complex to do the interpolation client side, and keep the server updated each frame. But doing it server side as in the PR isn't necessarily "wrong", it's just worth considering that you can do it the other way around. Ultimately it's up to the XR maintainers which way they prefer (and in some cases it only works sensibly one way around).

For targets on the client side you will also often want to use `get_global_transform_interpolated()` so you get a predicted interpolated position, instead of the physics tick position. This is the technique used for IK, where IK targets are evaluated each frame. 
> I don't have any XR devices so can't seem to run the MRP, but a function that can probably help you is: https://docs.godotengine.org/en/3.6/tutorials/physics/interpolation/advanced_physics_interpolation.html#get-global-transform-interpolated
> 
> Regarding the PR, you can do physics interpolation either client (scene) side or on the server. Sometimes we do it on the client side and sometimes server side.
> 
> Client side is often easier to implement (and indeed the mechanism is already built in to help do this via `get_global_transform_interpolated()`, although care has to be taken over priming). This function is primarily for users, but can also be used by core. Although having said that, in `Cameras` we do this manually as it gets around priming issues - it depends whether the node will always be active or will be switched on and off.
> 
> `Cameras` is a good example to see how they do it if you want to try client side, probably the 3D camera as it is slightly simpler. You may want to use `NOTIFICATION_INTERNAL_PROCESS` and `NOTIFICATION_INTERNAL_PHYSICS_PROCESS`.
> 
> The reason for doing it server side is, in most cases, for efficiency. So for e.g. large number of canvas items, or 3d nodes, we do the interpolation on the server, because it potentially is more efficient by e.g. avoiding lots of calls to `set_transform()` across the command queue on every frame, and is especially more efficient when the frame rate is higher than the tick rate.
> 
> On the other hand for single special items that there are only one or two of in a scene at once (e.g. `Cameras`) we do sometimes do the interpolation client side, as there is no significant loss to efficiency, and it keeps the server side code as simple as possible (indeed the overall code complexity is less).
> 
> I suspect for your XR origin it may be less complex to do the interpolation client side, and keep the server updated each frame. But doing it server side as in the PR isn't necessarily "wrong", it's just worth considering that you can do it the other way around. Ultimately it's up to the XR maintainers which way they prefer (and in some cases it only works sensibly one way around).
> 
> For targets on the client side you will also often want to use `get_global_transform_interpolated()` so you get a predicted interpolated position, instead of the physics tick position. This is the technique used for IK, where IK targets are evaluated each frame.

Thank you for your feedback! 
My initial approach was actually calling `set_process_internal` and updating the transform of the `XRServer` on each render frame with the interpolated value, basically what you just said of doing it in the client side. However, after taking a look at other implementations I thought It would be more appreciated to keep it similar to those as they seemed more performant than *forcing* an update every frame. I completely missed  `Camera3D::_update_camera` 😅
I agree that doing it similar to what the camera does is better here as it will be way easier to maintain and should work perfectly fine. We can only have 1 `XROrigin3D` node active at a time so only 1 would be processing internally, meaning the performance gain of doing it on the server side would be unnoticeable.

EDIT: I modified the PR to use this approach instead. The second issue still persists and maybe someone with physics experience can give us some advice.
> However for the second issue I'm out of ideas about how to proceed. Disabling interpolation on the tracked node makes it jitter because it will now follow the non interpolated transform of the parent even if the parent is set as interpolated.

I'm not sure I exactly understand the problem, but it could be it's the same as https://github.com/godotengine/godot/issues/103025#issuecomment-2670873689 and the same solution (or kind of solution) may apply.

Generally if you are setting the world position of a node each frame based on the hand controllers, then you should be ignoring the parent transform, maybe by setting as top level. But maybe the solution above is what you need here if you simply want fast updates relative to the interpolated parent xform. It's difficult to say without seeing an example of this working.
> > However for the second issue I'm out of ideas about how to proceed. Disabling interpolation on the tracked node makes it jitter because it will now follow the non interpolated transform of the parent even if the parent is set as interpolated.
> 
> I'm not sure I exactly understand the problem, but it could be it's the same as [#103025 (comment)](https://github.com/godotengine/godot/issues/103025#issuecomment-2670873689) and the same solution (or kind of solution) may apply.
> 
> Generally if you are setting the world position of a node each frame based on the hand controllers, then you should be ignoring the parent transform, maybe by setting as top level. But maybe the solution above is what you need here if you simply want fast updates relative to the interpolated parent xform. It's difficult to say without seeing an example of this working.

I updated the issue with a visual explanation and also attached an MRP for anyone who doesn't have XR hardware.
I'm aware that the behavior is actually the intended behavior for any normal `Node3D` but XR tracked nodes **must** update their transform on every render frame otherwise it would be unusable.
In the attached MRP I also added a node with the script of the comment you mentioned but doesn't seem to have any effect, maybe I did it wrong.

EDIT: If you rearrange the scene tree to look like this:

![Image](https://github.com/user-attachments/assets/3912b8dc-ffbe-4c0f-818a-b3535fe9cd0e)
It actually works:

https://github.com/user-attachments/assets/441ad413-9bec-4484-84ed-62d58609efda

However with this approach all childs of `FollowHand` must have interpolation disabled which it's fine but it won't work for XR because those nodes must be childs of a `XROrigin3D`, if we make `XROrigin3D` be the one following the interpolated transform of its parent instead it will work, but will defeat the whole point of the first fix.
Needing to add a helper node in between the `CharacterController3D` or whatever the parent is and `XROrigin3D` feels like a hacky workaround.
Try this for `fake_hand.gd`:
```
extends Node3D

var offset = Vector3()

# Called when the node enters the scene tree for the first time.
func _ready() -> void:
	DisplayServer.mouse_set_mode(DisplayServer.MOUSE_MODE_CAPTURED)
	physics_interpolation_mode = Node.PHYSICS_INTERPOLATION_MODE_OFF
	pass # Replace with function body.


# Called every frame. 'delta' is the elapsed time since the previous frame.
func _process(_delta: float) -> void:
	var tr = get_parent().get_global_transform_interpolated()
	tr.origin += offset
	global_transform = tr

var mouse_sensitivity = 0.002
func _input(event):
	if event is InputEventMouseMotion and Input.mouse_mode == Input.MOUSE_MODE_CAPTURED:
		offset.x += event.relative.x * mouse_sensitivity
		offset.y += -event.relative.y * mouse_sensitivity
```

I'll try and write more explanation tomorrow but should be enough to get started.
> Try this for `fake_hand.gd`:
> 
> ```
> extends Node3D
> 
> var offset = Vector3()
> 
> # Called when the node enters the scene tree for the first time.
> func _ready() -> void:
> 	DisplayServer.mouse_set_mode(DisplayServer.MOUSE_MODE_CAPTURED)
> 	physics_interpolation_mode = Node.PHYSICS_INTERPOLATION_MODE_OFF
> 	pass # Replace with function body.
> 
> 
> # Called every frame. 'delta' is the elapsed time since the previous frame.
> func _process(_delta: float) -> void:
> 	var tr = get_parent().get_global_transform_interpolated()
> 	tr.origin += offset
> 	global_transform = tr
> 
> var mouse_sensitivity = 0.002
> func _input(event):
> 	if event is InputEventMouseMotion and Input.mouse_mode == Input.MOUSE_MODE_CAPTURED:
> 		offset.x += event.relative.x * mouse_sensitivity
> 		offset.y += -event.relative.y * mouse_sensitivity
> ```
> 
> I'll try and write more explanation tomorrow but should be enough to get started.

It works, thank you!
I was currently testing another approach for the same concept but looks like mine didn't really work, I will try to adapt this to work internally and report the result here

EDIT: The approach I was trying didn't quite work when moving the parent, I was setting the global transform directly in `XRNode3D::_pose_changed`. If instead the we save the pose transform and apply it as an offset to the parent interpolated transform on every internal process it works fine:

```cpp
Node3D *parent = Object::cast_to<Node3D>(get_parent());

if (parent && parent->is_physics_interpolated_and_enabled() && !is_set_as_top_level()) {
	set_global_transform(parent->get_global_transform_interpolated() * pose_offset);
}
```

Left is with the fix applied, right is how it currently works.

https://github.com/user-attachments/assets/7a8f530a-b50e-44ce-b8bf-b5608645c2f8

Sadly it's extremely hard to notice on video, it's something that is only apparent when you're using the VR headset but the left cube moves as smooth as you would expect while the right cube jitters quite a bit.
Tomorrow I will update the PR with this changes and I guess it will be ready for review.

I updated the PR with the fix for XRNode3D and XRCamera3D. This fix only works if interpolation is set as disabled on those nodes, maybe we should also add a warning to let people know that those aren't meant to work with interpolation enabled?

Another issue I found is if a `MeshInstance3D` is the child of one of those nodes, you will get the following warning from time to time: `Non-interpolated instance triggered from physics process` this is caused because `VisualInstance3D` seems to call `RenderingServer::instance_set_transform` on `NOTIFICATION_TRANSFORM_CHANGED` but that notification is received on physics frame by `SceneTree::flush_transform_notifications` in `SceneTree::physics_process`. The transform change is being done in `NOTIFICATION_INTERNAL_PROCESS` so I'm unsure why this is happening.
EDIT: I think it's triggered because moving the parent (`CharacterBody3D`) of `XROrigin3D` on physics frame by calling `move_and_slide` triggers a transform update to its childs and causes the warning, I was able to replicate the warning on the tracking_mrp by just moving the character around. The issue isn't specific to XR.
> I updated the PR with the fix for XRNode3D and XRCamera3D. This fix only works if interpolation is set as disabled on those nodes, maybe we should also add a warning to let people know that those aren't meant to work with interpolation enabled?

You can hard code those nodes to start with interpolation set to off (`set_physics_interpolation_mode(Node::PHYSICS_INTERPOLATION_MODE_OFF)`).
We do this for some nodes already, e.g. `ParallaxLayer`.

> Another issue I found is if a MeshInstance3D is the child of one of those nodes, you will get the following warning from time to time: Non-interpolated instance triggered from physics process this is caused because VisualInstance3D seems to call RenderingServer::instance_set_transform on NOTIFICATION_TRANSFORM_CHANGED but that notification is received on physics frame by SceneTree::flush_transform_notifications in SceneTree::physics_process.

I can't be sure, but usually the solution for this is simple, don't send the transform to the visual server in response to this notification if it is non-interpolated. You are either responding to transform changes on the physics tick (using interpolation) or you are managing transform yourself on the frame. Trying to do both at once is usually user error.

> You can hard code those nodes to start with interpolation set to off (`set_physics_interpolation_mode(Node::PHYSICS_INTERPOLATION_MODE_OFF)`). We do this for some nodes already, e.g. `ParallaxLayer`.

I see, I will do that then, thank you 😊.

> I can't be sure, but usually the solution for this is simple, don't send the transform to the visual server in response to this notification if it is non-interpolated. You are either responding to transform changes on the physics tick (using interpolation) or you are managing transform yourself on the frame. Trying to do both at once is usually user error.

I didn't add that behavior, it's a side effect of having a non interpolated node as a child of an interpolated one. This is defined at `VisualInstance3D::_notification`, changing its behavior I think it's outside of the scope of this issue and PR, maybe a different issue should be created regarding this?
> I didn't add that behavior, it's a side effect of having a non interpolated node as a child of an interpolated one. This is defined at VisualInstance3D::_notification,

Ah, I see. This might need a simple separate (non-XR) MRP to investigate best way of solving. 🤔 

Note there is a `set_notify_transform(true)` on `VisualInstance` which could potentially be used to switch this off.
> Ah, I see. This might need a simple separate (non-XR) MRP to investigate best way of solving. 🤔

The tracked_mrp with the `fake_hand.gd` change is enough to get this warning:

[tracking_mrp.zip](https://github.com/user-attachments/files/18971135/tracking_mrp.zip)

![Image](https://github.com/user-attachments/assets/f31f5b31-fe3d-4d0f-89d2-d90c909546ef)

> Note there is a `set_notify_transform(true)` on `VisualInstance` which could potentially be used to switch this off.

I will test if that fixes the issue but do you think it should be part of this PR? It fixes an issue completely different than what this aims to fix
> I will test if that fixes the issue but do you think it should be part of this PR? It fixes an issue completely different than what this aims to fix

I'm not sure yet.

The warnings are editor only, and in theory there should be no glitches providing there is an update of the transform on the frame again before rendering.

I'll try and have a look at this tomorrow.
> > I will test if that fixes the issue but do you think it should be part of this PR? It fixes an issue completely different than what this aims to fix
> 
> I'm not sure yet.
> 
> The warnings are editor only, and in theory there should be no glitches providing there is an update of the transform on the frame again before rendering.
> 
> I'll try and have a look at this tomorrow.

I just tried changing `set_notify_transform(true)` and doesn't quite work, `VisualInstance` also does more things than just calling `RenderingServer::instance_set_transform` on transform change. Instead I set it to not update the rendering server if the node isn't interpolated and we're inside a physics frame, it seems to work properly and no warning is shown 🤔
Nvm, looks like as a side effect it doesn't apply some initial transforms when instantiating

After thinking about it for a while, I get why users should receive a warning when changing the transform of an interpolated node outside of physics process, but why is there a warning when changing the transform of a non interpolated node on physics process?
If a node is not interpolated I don't see why is there a problem changing its transform from wherever I want. 
Maybe there are side effects that I don't know.
> After thinking about it for a while, I get why users should receive a warning when changing the transform of an interpolated node outside of physics process, but why is there a warning when changing the transform of a non interpolated node on physics process?
If a node is not interpolated I don't see why is there a problem changing its transform from wherever I want.
Maybe there are side effects that I don't know.

You are correct, in that it's not as problematic as changing interpoated nodes outside of physics process, but I still decided in the general case it was worth warning (for now):

1) Physics interpolation warnings only kick off with repeated use, so changing a few initial transforms is no problem. It identifies a regular pattern of use.
2) The regular changing of transforms on physics ticks is indicative of common user misunderstanding how to use physics interpolation among beginners.
3) Even with additional transforms sent per frame, sending transforms on physics ticks is superfluous and will reduce performance 

Regarding (2), the general paradigm for physics interpolation is:
* Interpolation ON for a node - move on physics tick
* Interpolation OFF for a node - move on frame

It is only now we are starting to deal with these special cases (IK, ragdoll, XR) that we may have to address this problem that you mention of propagating transforms on physics ticks. It's possible that there will be a simple fix in some way preventing the propagation (but as you say watching for special circumstances where it is desired).

The alternative would be to identify these circumstances and turn off the warning, but that still leaves (3) and is susceptible to the worst kind of problem, that the user *thinks* everything is working correctly, but they are moving non-interpolated nodes on physics tick only. Ideally we want it to be *obvious* when the user is doing something wrong.
> You are correct, in that it's not as problematic as changing interpoated nodes outside of physics process, but I still decided in the general case it was worth warning (for now):
> 
> 1. Physics interpolation warnings only kick off with repeated use, so changing a few initial transforms is no problem. It identifies a regular pattern of use.
> 2. The regular changing of transforms on physics ticks is indicative of common user misunderstanding how to use physics interpolation among beginners.
> 3. Even with additional transforms sent per frame, sending transforms on physics ticks is superfluous and will reduce performance
> 
> Regarding (2), the general paradigm for physics interpolation is:
> 
> * Interpolation ON for a node - move on physics tick
> * Interpolation OFF for a node - move on frame
> 
> It is only now we are starting to deal with these special cases (IK, ragdoll, XR) that we may have to address this problem that you mention of propagating transforms on physics ticks. It's possible that there will be a simple fix in some way preventing the propagation (but as you say watching for special circumstances where it is desired).
> 
> The alternative would be to identify these circumstances and turn off the warning, but that still leaves (3) and is susceptible to the worst kind of problem, that the user _thinks_ everything is working correctly, but they are moving non-interpolated nodes on physics tick only. Ideally we want it to be _obvious_ when the user is doing something wrong.

Alright that makes more sense then 🤔
Identifying these circumstances might be a bit hard though, would it be something like not giving a warning if any of the parents is one of those special nodes? The problem is it can become pretty convoluted as we keep adding edge cases, maybe a flag has to be added to the node itself?
I'm just looking at it now, it's likely I can sort this as a separate PR (i.e. the warnings and extra transforms).
> I'm just looking at it now, it's likely I can sort this as a separate PR (i.e. the warnings and extra transforms).

I'm not sure if I understand what you mean by extra transforms 😅
@Asaduji see #103331 .

I'm not absolutely sure yet whether it will cover all cases yet but it is pretty simple and does what *should* be required to remove the extra notifications.

It seems fine for the initial placement problem you were having above, but needs some thorough testing.

Hopefully that (or similar) will work in conjunction with your PR.
> [@Asaduji](https://github.com/Asaduji) see [#103331](https://github.com/godotengine/godot/pull/103331) .
> 
> I'm not absolutely sure yet whether it will cover all cases yet but it is pretty simple and does what _should_ be required to remove the extra notifications.
> 
> It seems fine for the initial placement problem you were having above, but needs some thorough testing.
> 
> Hopefully that (or similar) will work in conjunction with your PR.

Thank you for your work here 🤝
I will test your fix in XR and see if everything works fine, I guess I just have to do it in `Node3D::_propagate_transform_changed` instead for 4.x.
Do we have to wait for this to be fixed before this PR can be reviewed?

It's probably a good idea for us to finalize and merge fix for the notifications before merging your PR, but I'm sure yours can be reviewed / tested with the other fix. I'll try and do the 4.x PR tomorrow morning. I say this because if we did decide to use e.g. `toplevel` then that would have knock on implications for your PR.

It's quite common to have dependent PRs, we often do things like add the dependent PR as a first commit and mark that it is dependent on it, then rebase the 2nd PR once the first is merged.
> It's probably a good idea for us to finalize and merge fix for the notifications before merging your PR, but I'm sure yours can be reviewed / tested with the other fix. I'll try and do the 4.x PR tomorrow morning. I say this because if we did decide to use e.g. `toplevel` then that would have knock on implications for your PR.
> 
> It's quite common to have dependent PRs, we often do things like add the dependent PR as a first commit and mark that it is dependent on it, then rebase the 2nd PR once the first is merged.

Understood 🫡
I tested your fix and seems to be working fine so far, however I found another warning: the debug shape of a collision object can cause "Interpolated instance triggered from outside physics process" no matter if the collision object is set as interpolated or not.
To replicate, use this mrp, make sure `Visible collision shapes` is enabled and just move the character around:

[tracking_mrp.zip](https://github.com/user-attachments/files/18996552/tracking_mrp.zip)
> I tested your fix and seems to be working fine so far, however I found another warning: the debug shape of a collision object can cause "Interpolated instance triggered from outside physics process" no matter if the collision object is set as interpolated or not.

I can't seem to replicate this so far, there are some issues with your MRP on mine, it seems to move to the right continuously. Do you have variable refresh / high refresh rate monitor by any chance?

The MRP tick rate is set to 60 and my monitor is approx 60 so there may be no aliasing. To be clear does this warning appear before the fix, after the fix, or both?

UPDATE: Ah I'm getting it if I force a warning of every occurrence. It may be tricky to find the source as I can't seem to step through the code due to the collision shapes only showing when running from the editor, maybe there's some way of forcing this when running solo. 🤔 

Ah got it: `--debug-collisions` is needed as it doesn't preserve the editor setting to the running project.

This may warrant a separate issue. I don't think it's serious, the debug collision models probably are hacked in so bypass some of the normal protections.

Yeah looks to be because neither `CollisionShape3D` or `AnimatableBody3D` are `VisualInstances` so they don't have the normal machinery for things shown on the screen. It can be fixed up, but the warnings are fine to ignore for now. It's an unrelated minor bug so probably best for a separate issue.

UPDATE:
Ah no, it's something far more obvious: You've placed a physics node (`AnimatableBody3D`) on a non-interpolated branch of the scene tree. You've set `FakeHand` to OFF.

This unsurprisingly won't work well.

In fact going from non-interpolated branch to an interpolated sub branch is likely to go horribly wrong, it's really not supported (or envisaged), maybe I should add a warning for this. I'm not sure there is a use-case, but if you can think of one let me know.
The above PR #103355 (or 4.x version) should add warnings for the debug collision shapes, however that isn't really related to this issue so we should probably move discussion of that to the PR or create a separate issue.

I'll be away the weekend, I'd also suggest evaluating fixing the XR by setting the first non-interpolated child to `top_level`.
At present I'm not 100% sure whether to go for #103351 and explicity prevent notifications propagating, or to (maybe automatically) set the first non-interpolated child to `top_level`.
> In fact going from non-interpolated branch to an interpolated sub branch is likely to go horribly wrong, it's really not supported (or envisaged), maybe I should add a warning for this. I'm not sure there is a use-case, but if you can think of one let me know.

I'm aware that moving a physics object outside physics process isn't recommended, as it might appear to be clipping other objects etc, I was just testing if `AnimatableBody3D` would work well just being a child of the fake hand and realized it worked "fine" but the debug shape itself was triggering warnings.

> The above PR [#103355](https://github.com/godotengine/godot/pull/103355) (or 4.x version) should add warnings for the debug collision shapes, however that isn't really related to this issue so we should probably move discussion of that to the PR or create a separate issue.

Sounds like the right fix, as I said I was aware that you aren't meant to move the physics object on render frame, my point was that doing so revealed that debug shapes don't seem to respect the interpolation mode of the physics object they're following 🤔
Anyways I guess it doesn't matter since you're not supposed to do that anyways 😅

> I'll be away the weekend, I'd also suggest evaluating fixing the XR by setting the first non-interpolated child to `top_level`. At present I'm not 100% sure whether to go for [#103351](https://github.com/godotengine/godot/pull/103351) and explicity prevent notifications propagating, or to (maybe automatically) set the first non-interpolated child to `top_level`.

I will test setting them as `top_level` and see what happens, just in case you decide to go with that route.
To me however not silently setting them as `top_level` sounds better because then what happens if someone wants to set them as `top_level` on purpose so they don't "follow" the parent? Unless I misunderstood what you meant, your point is setting them as `top_level` so they don't receive the transform updates of the parent and then manually applying the parent transform to them on each frame?
 If so then this description wouldn't be true for these special nodes 🤔: https://docs.godotengine.org/en/stable/classes/class_node3d.html#class-node3d-property-top-level

Fix patch:
diff --git a/doc/classes/XRCamera3D.xml b/doc/classes/XRCamera3D.xml
index e49e884f33ea..ca1cd431f684 100644
--- a/doc/classes/XRCamera3D.xml
+++ b/doc/classes/XRCamera3D.xml
@@ -10,4 +10,7 @@
 	<tutorials>
 		<link title="XR documentation index">$DOCS_URL/tutorials/xr/index.html</link>
 	</tutorials>
+	<members>
+		<member name="physics_interpolation_mode" type="int" setter="set_physics_interpolation_mode" getter="get_physics_interpolation_mode" overrides="Node" enum="Node.PhysicsInterpolationMode" default="2" />
+	</members>
 </class>
diff --git a/doc/classes/XRNode3D.xml b/doc/classes/XRNode3D.xml
index fce3708d9ab0..cc2f4b874e7e 100644
--- a/doc/classes/XRNode3D.xml
+++ b/doc/classes/XRNode3D.xml
@@ -46,6 +46,7 @@
 		</method>
 	</methods>
 	<members>
+		<member name="physics_interpolation_mode" type="int" setter="set_physics_interpolation_mode" getter="get_physics_interpolation_mode" overrides="Node" enum="Node.PhysicsInterpolationMode" default="2" />
 		<member name="pose" type="StringName" setter="set_pose_name" getter="get_pose_name" default="&amp;&quot;default&quot;">
 			The name of the pose we're bound to. Which poses a tracker supports is not known during design time.
 			Godot defines number of standard pose names such as [code]aim[/code] and [code]grip[/code] but other may be configured within a given [XRInterface].
diff --git a/scene/3d/xr/xr_nodes.cpp b/scene/3d/xr/xr_nodes.cpp
index 4ab6e4cab925..12e01c785113 100644
--- a/scene/3d/xr/xr_nodes.cpp
+++ b/scene/3d/xr/xr_nodes.cpp
@@ -72,7 +72,35 @@ void XRCamera3D::_removed_tracker(const StringName &p_tracker_name, int p_tracke
 
 void XRCamera3D::_pose_changed(const Ref<XRPose> &p_pose) {
 	if (p_pose->get_name() == pose_name) {
-		set_transform(p_pose->get_adjusted_transform());
+		Node3D *parent = Object::cast_to<Node3D>(get_parent());
+
+		if (is_inside_tree() && parent && parent->is_physics_interpolated_and_enabled() && !is_set_as_top_level() && !is_physics_interpolated()) {
+			pose_offset = p_pose->get_adjusted_transform();
+		} else {
+			set_transform(p_pose->get_adjusted_transform());
+		}
+	}
+}
+
+void XRCamera3D::_notification(int p_what) {
+	switch (p_what) {
+		case NOTIFICATION_ENTER_TREE: {
+			if (!Engine::get_singleton()->is_editor_hint()) {
+				set_desired_process_modes(true, false);
+			}
+		} break;
+
+		case NOTIFICATION_INTERNAL_PROCESS: {
+			if (!is_inside_tree() || is_physics_interpolated() || Engine::get_singleton()->is_editor_hint()) {
+				break;
+			}
+
+			Node3D *parent = Object::cast_to<Node3D>(get_parent());
+
+			if (parent && parent->is_physics_interpolated_and_enabled() && !is_set_as_top_level()) {
+				set_global_transform(parent->get_global_transform_interpolated() * pose_offset);
+			}
+		} break;
 	}
 }
 
@@ -196,6 +224,9 @@ Vector<Plane> XRCamera3D::get_frustum() const {
 }
 
 XRCamera3D::XRCamera3D() {
+	// XRCamera3D gets its transform updated every render frame and shouldn't be interpolated.
+	set_physics_interpolation_mode(Node::PHYSICS_INTERPOLATION_MODE_OFF);
+
 	XRServer *xr_server = XRServer::get_singleton();
 	ERR_FAIL_NULL(xr_server);
 
@@ -400,7 +431,13 @@ void XRNode3D::_removed_tracker(const StringName &p_tracker_name, int p_tracker_
 
 void XRNode3D::_pose_changed(const Ref<XRPose> &p_pose) {
 	if (p_pose.is_valid() && p_pose->get_name() == pose_name) {
-		set_transform(p_pose->get_adjusted_transform());
+		Node3D *parent = Object::cast_to<Node3D>(get_parent());
+
+		if (is_inside_tree() && parent && parent->is_physics_interpolated_and_enabled() && !is_set_as_top_level() && !is_physics_interpolated()) {
+			pose_offset = p_pose->get_adjusted_transform();
+		} else {
+			set_transform(p_pose->get_adjusted_transform());
+		}
 		_set_has_tracking_data(p_pose->get_has_tracking_data());
 	}
 }
@@ -440,7 +477,32 @@ void XRNode3D::_update_visibility() {
 	}
 }
 
+void XRNode3D::_notification(int p_what) {
+	switch (p_what) {
+		case NOTIFICATION_ENTER_TREE: {
+			if (!Engine::get_singleton()->is_editor_hint()) {
+				set_process_internal(true);
+			}
+		} break;
+
+		case NOTIFICATION_INTERNAL_PROCESS: {
+			if (!is_inside_tree() || is_physics_interpolated() || Engine::get_singleton()->is_editor_hint()) {
+				break;
+			}
+
+			Node3D *parent = Object::cast_to<Node3D>(get_parent());
+
+			if (parent && parent->is_physics_interpolated_and_enabled() && !is_set_as_top_level()) {
+				set_global_transform(parent->get_global_transform_interpolated() * pose_offset);
+			}
+		} break;
+	}
+}
+
 XRNode3D::XRNode3D() {
+	// XRNode3D gets its transform updated every render frame and shouldn't be interpolated.
+	set_physics_interpolation_mode(Node::PHYSICS_INTERPOLATION_MODE_OFF);
+
 	XRServer *xr_server = XRServer::get_singleton();
 	ERR_FAIL_NULL(xr_server);
 
@@ -714,6 +776,12 @@ void XROrigin3D::_set_current(bool p_enabled, bool p_update_others) {
 		ERR_FAIL_NULL(xr_server);
 
 		xr_server->set_world_origin(get_global_transform());
+
+		if (is_physics_interpolated()) {
+			set_process_internal(true);
+		}
+	} else if (is_physics_interpolated()) {
+		set_process_internal(false);
 	}
 
 	// Check if we need to update our other origin nodes accordingly
@@ -784,10 +852,16 @@ void XROrigin3D::_notification(int p_what) {
 
 		case NOTIFICATION_LOCAL_TRANSFORM_CHANGED:
 		case NOTIFICATION_TRANSFORM_CHANGED: {
-			if (current && !Engine::get_singleton()->is_editor_hint()) {
+			if (current && !Engine::get_singleton()->is_editor_hint() && !is_physics_interpolated_and_enabled()) {
 				xr_server->set_world_origin(get_global_transform());
 			}
 		} break;
+
+		case NOTIFICATION_INTERNAL_PROCESS: {
+			if (current && !Engine::get_singleton()->is_editor_hint() && is_physics_interpolated_and_enabled()) {
+				xr_server->set_world_origin(get_global_transform_interpolated());
+			}
+		} break;
 	}
 
 	if (current) {
@@ -800,3 +874,9 @@ void XROrigin3D::_notification(int p_what) {
 		}
 	}
 }
+
+void XROrigin3D::_physics_interpolated_changed() {
+	if (current && !Engine::get_singleton()->is_editor_hint()) {
+		set_process_internal(is_physics_interpolated());
+	}
+}
diff --git a/scene/3d/xr/xr_nodes.h b/scene/3d/xr/xr_nodes.h
index 141cef3ec48a..65ba86abd20a 100644
--- a/scene/3d/xr/xr_nodes.h
+++ b/scene/3d/xr/xr_nodes.h
@@ -46,12 +46,14 @@ class XRCamera3D : public Camera3D {
 	StringName tracker_name = "head";
 	StringName pose_name = SceneStringName(default_);
 	Ref<XRPositionalTracker> tracker;
+	Transform3D pose_offset;
 
 	void _bind_tracker();
 	void _unbind_tracker();
 	void _changed_tracker(const StringName &p_tracker_name, int p_tracker_type);
 	void _removed_tracker(const StringName &p_tracker_name, int p_tracker_type);
 	void _pose_changed(const Ref<XRPose> &p_pose);
+	void _notification(int p_what);
 
 public:
 	PackedStringArray get_configuration_warnings() const override;
@@ -79,6 +81,7 @@ class XRNode3D : public Node3D {
 	StringName pose_name = SceneStringName(default_);
 	bool has_tracking_data = false;
 	bool show_when_tracked = false;
+	Transform3D pose_offset;
 
 protected:
 	Ref<XRPositionalTracker> tracker;
@@ -95,6 +98,7 @@ class XRNode3D : public Node3D {
 	void _set_has_tracking_data(bool p_has_tracking_data);
 
 	void _update_visibility();
+	void _notification(int p_what);
 
 public:
 	void _validate_property(PropertyInfo &p_property) const;
@@ -197,6 +201,7 @@ class XROrigin3D : public Node3D {
 protected:
 	void _notification(int p_what);
 	static void _bind_methods();
+	virtual void _physics_interpolated_changed() override;
 
 public:
 	PackedStringArray get_configuration_warnings() const override;


Test patch: