Repository: godotengine/godot (language: cpp)

Issue:
`RenderingDevice` cannot release texture resource when window is minimized
### Tested versions

4.3.stable.mono

### System information

Godot v4.3.stable.mono - Windows 10.0.22631 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4070 (NVIDIA; 31.0.15.3161) - AMD Ryzen 9 7900X 12-Core Processor (24 Threads)

### Issue description

When the game window is minimized, the texture created by `RenderingDevice.create_texture()` won't be freed by calling `RenderingDevice.free_rid()`. 

![image](https://github.com/user-attachments/assets/6659e9ec-eba2-4df5-be26-d9591dac8972)
when the game window is not minimized

![image](https://github.com/user-attachments/assets/6ac45e18-4752-4561-b4e2-66e0d7e9c245)
when the game window is minimized after a while



BTW, the Memory inspected by Editor Debugger grows much slower than inspected by Windows Task Manager, when this bug is triggered. 
![image](https://github.com/user-attachments/assets/af05bb99-ca8c-4cbc-8296-242de007c2d7)
not minimized
![image](https://github.com/user-attachments/assets/8c6c460b-9a6c-4954-a414-a3c341e2f964)
minimized after a while

### Steps to reproduce

```gdscript
extends Node2D

var rd : RenderingDevice;
var texture : RID;

func _ready() -> void:
	rd = RenderingServer.get_rendering_device()

func _process(delta: float) -> void:
	#RenderingServer.call_on_render_thread(create_texture);
	create_texture()

func create_texture() -> void:
	if texture.is_valid():
		rd.free_rid(texture)
	
	var format = RDTextureFormat.new()
	format.width = 1024
	format.height = 1024
	format.depth = 1
	format.format = RenderingDevice.DATA_FORMAT_R8G8B8A8_UNORM
	format.array_layers = 1
	format.mipmaps = 1
	format.usage_bits = RenderingDevice.TEXTURE_USAGE_SAMPLING_BIT | RenderingDevice.TEXTURE_USAGE_CAN_UPDATE_BIT | RenderingDevice.TEXTURE_USAGE_STORAGE_BIT
	
	texture = rd.texture_create(format, RDTextureView.new(), [])
```

1. This script will create a texture pre frame and release the texture created last frame. 
2. Attach it to a node and run the scene.
3. Minimize the game window (not editor). (just hide this window below other windows won't trigger this bug)


### Minimal reproduction project (MRP)

See 'Steps to reproduce'


Discussion / hints:
Maybe related with #90030 and #90017. 

Similar situation is also found when a NoiseTexture Resource is modified every frame by a script (I know it is not recommended). 
In my understanding, the NoiseTexture will create a new texture when the parameter is modified. So this bug might be triggered when the window is minimized. 
A workaround should be checking if the game window is minimized before creating resources from `RenderingDevice`. 

Such as: 
``` gdscript
if Engine.get_main_loop().root.mode == Window.MODE_MINIMIZED:
   return
# ... original code
```
Do the resources get freed once the window is no longer minimized?
> Do the resources get freed once the window is no longer minimized?

Probably NO, it seems only part of them will be freed. 

In my project, I find this bug in a compute shader which creates a texture each frame. In this case, the memory leak is more obvious than the example code above. 

The memory leak of the example code above is too severe, so I cannot observe the same behavior before my compute's memory used up  :(

![image](https://github.com/user-attachments/assets/49cffb45-7130-4c56-b913-7e8efb2e2918)
This is observed in my compute shader case, I just reopen the window three times which corresponds to the three sudden decline. 
When the free_rid() method is called the resources are not released immediately, but are queued to be freed in the next frame, but when the window is minimized the gpu cycle is paused which means it doesn't process frames anymore, this causes resources to be queued but never released. And when the window is opened up again it starts to process the frames and clears out the queue. 

As a test, I added this inside add_frame_delay method in os.cpp,

```
if (!p_can_draw) {
	RD::get_singleton()->_flush_and_stall_for_all_frames();
}
```

It solved this problem, #90030 and also #90017, but I'm not sure what would be the proper implementation because _flush_and_stall_for_all_frames is a private method (I changed it to public for testing) and calling it every frame might be problematic. 
@Uumutunal Great work! That seems like an acceptable workaround. We could probably store a variable to ensure that we only call `_flush_and_stall_for_all_frames()` once when the window is first minimized. 

However, the way I understand things is that resources _should_ get cleaned up on the next frame, which _should_ happen once the window is opened back up. However Hippogyz' [comment](https://github.com/godotengine/godot/issues/98044#issuecomment-2404133395) Seems to indicate that, even after the GPU starts processing frames again, it doesn't clean up the resources. Which makes me think that maybe there is still a problem somewhere else and the resources are somehow getting orphaned before they are actually cleaned up. 

@DarioSamo what do you think?
> We could probably store a variable to ensure that we only call `_flush_and_stall_for_all_frames()` once when the window is first minimized.

I'm guessing calling it once won't work because new resources keep getting created while the window is minimized. We would have to call it at regular intervals at least, something like once a second.

>even after the GPU starts processing frames again, it doesn't clean up the resources.

After testing it more, yes you are right, it seems that only part of the resources are freed unless it is very small amount like 1-2 mb. I have no idea why this happens though.
Resources are not freed until there's a guarantee the GPU is no longer executing the frame. It only knows that if it has to wait for the fence, which happens when it wants to begin drawing on that frame in the queue. Therefore destruction of resources is delayed until that frame begins recording.

```cpp
void RenderingDevice::_begin_frame() {
	if (frames[frame].fence_signaled) {
		driver->fence_wait(frames[frame].fence);
		frames[frame].fence_signaled = false;
	}

        ...

	_free_pending_resources(frame);

       ...
}
```

Calling _flush_and_stall_for_all_frames() does indeed fix most of the issue as the trick the method does is wait on the current frame to finish executing and begins the next one.

We're indeed lacking a way to force this. There's other methods such as `buffer_get_data()` that will also trigger it, but we don't have a method publicly exposed for it. `sync()` is the closest we have, but the use of `submit()` and `sync()` is reserved for Local Devices only.

It'd seem the only way to go about it is making the sync() method accepted or to add a different method altogether.
Asking just to learn because I'm pretty much new to this, but why can't we simply add a public method and call `_flush_and_stall_for_all_frames()` or even `_free_pending_resources()`? Because the reason `buffer_get_data()`, `sync()` and `submit()` work is that they all call `_free_pending_resources()`, and I'm guessing we'll have to call it one way or another.
Adding public methods to certain classes is usually done with extreme care because these classes are exposed to users either via GDScript or Extensions and require documentation. It also presents a commitment to support this new method in future versions and we must make sure to not break compatibility if possible.

If we come to situations like this where there's no other solution, we can discuss it and add the method, but it's not usually something that we should think of as the first solution if possible.

As for free pending resources, it's unlikely we'd ever want to expose that. That's an internal implementation detail that end users should not concern themselves with, as from their perspective, resources are deleted when they call free().
Alright, that makes sense. Thanks for the explanation.
@DarioSamo What do you think about the fact that not all resources are getting freed once the window is opened again? It looks like in hippogyz' case a lot of the memory is never freed, even once the window is opened again
This might be the same issue as https://github.com/godotengine/godot/issues/84212
> [@DarioSamo](https://github.com/DarioSamo) What do you think about the fact that not all resources are getting freed once the window is opened again? It looks like in hippogyz' case a lot of the memory is never freed, even once the window is opened again

Stale references perhaps? Could be a problem caused by either RenderingDevice or something else. I think that'd warrant investigation separate from the issue that was suspected.
I'm a bit confused about the possible implications on how to solve this properly. There's something that rings wrong to me about the fact that we allow GDScript to call code on instances where the frame queue is not active because it's no longer presenting to the screen.

While the OP has shown a fairly simple script that demonstrates the issue, it falls to reason there'll be many other errors like for example attempting to draw something using RD while the window is minimized. Even that kind of situation will lead to an endless amount of commands being queued and never generated, eventually leading to an overflow of the maximum command list size allowed by the GPU.

The only solution I see to that is running the frame queue on the background regardless of that, but if present is skipped, that means the resources will be saturated while the window is minimized and the GPU will keep doing a ton of work.

We'd need some sort of mixed solution where we allow running frames only if the user has requested something from RenderingDevice while the window is minimized, and of course we don't present to the swap chain. Do we want to go for a solution like that?
After looking into it further, I see we got some logic that already handles the fact viewports are _not_ drawn while the window is minimized. We might be able to find some path from that to make RD execute the command queue if it's not empty (e.g. RS has a 'has_changes' method that might come in handy).
Adding something here that has a method for RenderingServer to check whether it has any queued commands to process is probably our best bet here.

```cpp
bool buffers_swapped = false; // New
if ((DisplayServer::get_singleton()->can_any_window_draw() || DisplayServer::get_singleton()->has_additional_outputs()) &&
		RenderingServer::get_singleton()->is_render_loop_enabled()) {
	if ((!force_redraw_requested) && OS::get_singleton()->is_in_low_processor_usage_mode()) {
		if (RenderingServer::get_singleton()->has_changed()) {
			RenderingServer::get_singleton()->draw(true, scaled_step); // flush visual commands
			Engine::get_singleton()->increment_frames_drawn();
			buffers_swapped = true; // New
		}
	} else {
		RenderingServer::get_singleton()->draw(true, scaled_step); // flush visual commands
		Engine::get_singleton()->increment_frames_drawn();
		force_redraw_requested = false;
		buffers_swapped = true; // New
	}
}

if (!buffers_swapped) {
	// New
	RenderingServer::get_singleton()->command_that_flushes_rd();
}
```

The script posted by the OP has no interaction with RenderingServer whatsoever to find out if there's anything queued up, so we'd basically need some way for RenderingServer to redirect and perform a check on RD if there's anything that needs to be processed or freed.

Like I said, this issue is not entirely about the fact that resources are not freed, but a script that constantly fills and submits drawing lists will also run into the same issue and eventually run out of command buffer memory.

Fix patch:
diff --git a/main/main.cpp b/main/main.cpp
index 2c2ef94e0eaa..51b26a63f2c4 100644
--- a/main/main.cpp
+++ b/main/main.cpp
@@ -4441,15 +4441,20 @@ bool Main::iteration() {
 
 	RenderingServer::get_singleton()->sync(); //sync if still drawing from previous frames.
 
-	if ((DisplayServer::get_singleton()->can_any_window_draw() || DisplayServer::get_singleton()->has_additional_outputs()) &&
-			RenderingServer::get_singleton()->is_render_loop_enabled()) {
+	const bool has_pending_resources_for_processing = RD::get_singleton() && RD::get_singleton()->has_pending_resources_for_processing();
+	bool wants_present = (DisplayServer::get_singleton()->can_any_window_draw() ||
+								 DisplayServer::get_singleton()->has_additional_outputs()) &&
+			RenderingServer::get_singleton()->is_render_loop_enabled();
+
+	if (wants_present || has_pending_resources_for_processing) {
+		wants_present |= force_redraw_requested;
 		if ((!force_redraw_requested) && OS::get_singleton()->is_in_low_processor_usage_mode()) {
 			if (RenderingServer::get_singleton()->has_changed()) {
-				RenderingServer::get_singleton()->draw(true, scaled_step); // flush visual commands
+				RenderingServer::get_singleton()->draw(wants_present, scaled_step); // flush visual commands
 				Engine::get_singleton()->increment_frames_drawn();
 			}
 		} else {
-			RenderingServer::get_singleton()->draw(true, scaled_step); // flush visual commands
+			RenderingServer::get_singleton()->draw(wants_present, scaled_step); // flush visual commands
 			Engine::get_singleton()->increment_frames_drawn();
 			force_redraw_requested = false;
 		}
diff --git a/servers/rendering/dummy/rasterizer_dummy.h b/servers/rendering/dummy/rasterizer_dummy.h
index 6205193d9a99..33b4c38018b4 100644
--- a/servers/rendering/dummy/rasterizer_dummy.h
+++ b/servers/rendering/dummy/rasterizer_dummy.h
@@ -90,8 +90,8 @@ class RasterizerDummy : public RendererCompositor {
 
 	void gl_end_frame(bool p_swap_buffers) override {}
 
-	void end_frame(bool p_swap_buffers) override {
-		if (p_swap_buffers) {
+	void end_frame(bool p_present) override {
+		if (p_present) {
 			DisplayServer::get_singleton()->swap_buffers();
 		}
 	}
diff --git a/servers/rendering/renderer_compositor.h b/servers/rendering/renderer_compositor.h
index dbc8f155bfbd..e52cab740e01 100644
--- a/servers/rendering/renderer_compositor.h
+++ b/servers/rendering/renderer_compositor.h
@@ -99,7 +99,7 @@ class RendererCompositor {
 	virtual void blit_render_targets_to_screen(DisplayServer::WindowID p_screen, const BlitToScreen *p_render_targets, int p_amount) = 0;
 
 	virtual void gl_end_frame(bool p_swap_buffers) = 0;
-	virtual void end_frame(bool p_swap_buffers) = 0;
+	virtual void end_frame(bool p_present) = 0;
 	virtual void finalize() = 0;
 	virtual uint64_t get_frame_number() const = 0;
 	virtual double get_frame_delta_time() const = 0;
diff --git a/servers/rendering/renderer_rd/renderer_compositor_rd.cpp b/servers/rendering/renderer_rd/renderer_compositor_rd.cpp
index ba47508700fa..05dda5e74c55 100644
--- a/servers/rendering/renderer_rd/renderer_compositor_rd.cpp
+++ b/servers/rendering/renderer_rd/renderer_compositor_rd.cpp
@@ -112,10 +112,8 @@ void RendererCompositorRD::begin_frame(double frame_step) {
 	scene->set_time(time, frame_step);
 }
 
-void RendererCompositorRD::end_frame(bool p_swap_buffers) {
-	if (p_swap_buffers) {
-		RD::get_singleton()->swap_buffers();
-	}
+void RendererCompositorRD::end_frame(bool p_present) {
+	RD::get_singleton()->swap_buffers(p_present);
 }
 
 void RendererCompositorRD::initialize() {
@@ -264,7 +262,7 @@ void RendererCompositorRD::set_boot_image(const Ref<Image> &p_image, const Color
 
 	RD::get_singleton()->draw_list_end();
 
-	RD::get_singleton()->swap_buffers();
+	RD::get_singleton()->swap_buffers(true);
 
 	texture_storage->texture_free(texture);
 	RD::get_singleton()->free(sampler);
diff --git a/servers/rendering/renderer_rd/renderer_compositor_rd.h b/servers/rendering/renderer_rd/renderer_compositor_rd.h
index 6821fa737e4e..41c11113fefa 100644
--- a/servers/rendering/renderer_rd/renderer_compositor_rd.h
+++ b/servers/rendering/renderer_rd/renderer_compositor_rd.h
@@ -128,7 +128,7 @@ class RendererCompositorRD : public RendererCompositor {
 	void blit_render_targets_to_screen(DisplayServer::WindowID p_screen, const BlitToScreen *p_render_targets, int p_amount);
 
 	void gl_end_frame(bool p_swap_buffers) {}
-	void end_frame(bool p_swap_buffers);
+	void end_frame(bool p_present);
 	void finalize();
 
 	_ALWAYS_INLINE_ uint64_t get_frame_number() const { return frame; }
diff --git a/servers/rendering/rendering_device.cpp b/servers/rendering/rendering_device.cpp
index 72984e5abf8b..85b5759d1246 100644
--- a/servers/rendering/rendering_device.cpp
+++ b/servers/rendering/rendering_device.cpp
@@ -5660,6 +5660,8 @@ void RenderingDevice::_free_internal(RID p_id) {
 		ERR_PRINT("Attempted to free invalid ID: " + itos(p_id.get_id()));
 #endif
 	}
+
+	frames_pending_resources_for_processing = uint32_t(frames.size());
 }
 
 // The full list of resources that can be named is in the VkObjectType enum.
@@ -5762,11 +5764,11 @@ String RenderingDevice::get_device_pipeline_cache_uuid() const {
 	return driver->get_pipeline_cache_uuid();
 }
 
-void RenderingDevice::swap_buffers() {
+void RenderingDevice::swap_buffers(bool p_present) {
 	ERR_RENDER_THREAD_GUARD();
 
 	_end_frame();
-	_execute_frame(true);
+	_execute_frame(p_present);
 
 	// Advance to the next frame and begin recording again.
 	frame = (frame + 1) % frames.size();
@@ -5868,6 +5870,10 @@ void RenderingDevice::_free_pending_resources(int p_frame) {
 
 		frames[p_frame].buffers_to_dispose_of.pop_front();
 	}
+
+	if (frames_pending_resources_for_processing > 0u) {
+		--frames_pending_resources_for_processing;
+	}
 }
 
 uint32_t RenderingDevice::get_frame_delay() const {
diff --git a/servers/rendering/rendering_device.h b/servers/rendering/rendering_device.h
index 6b19d05fca48..2595aed62d7b 100644
--- a/servers/rendering/rendering_device.h
+++ b/servers/rendering/rendering_device.h
@@ -1400,6 +1400,17 @@ class RenderingDevice : public RenderingDeviceCommons {
 	TightLocalVector<Frame> frames;
 	uint64_t frames_drawn = 0;
 
+	// Whenever logic/physics request a graphics operation (not just deleting a resource) that requires
+	// us to flush all graphics commands, we must set frames_pending_resources_for_processing = frames.size().
+	// This is important for when the user requested for the logic loop to still be updated while
+	// graphics should not (e.g. headless Multiplayer servers, minimized windows that need to still
+	// process something on the background).
+	uint32_t frames_pending_resources_for_processing = 0u;
+
+public:
+	bool has_pending_resources_for_processing() const { return frames_pending_resources_for_processing != 0u; }
+
+private:
 	void _free_pending_resources(int p_frame);
 
 	uint64_t texture_memory = 0;
@@ -1444,7 +1455,7 @@ class RenderingDevice : public RenderingDeviceCommons {
 
 	uint64_t limit_get(Limit p_limit) const;
 
-	void swap_buffers();
+	void swap_buffers(bool p_present);
 
 	uint32_t get_frame_delay() const;
 
diff --git a/servers/rendering/rendering_server_default.cpp b/servers/rendering/rendering_server_default.cpp
index 2ec693cbbfe7..b5d0d014791f 100644
--- a/servers/rendering/rendering_server_default.cpp
+++ b/servers/rendering/rendering_server_default.cpp
@@ -406,15 +406,15 @@ void RenderingServerDefault::sync() {
 	}
 }
 
-void RenderingServerDefault::draw(bool p_swap_buffers, double frame_step) {
+void RenderingServerDefault::draw(bool p_present, double frame_step) {
 	ERR_FAIL_COND_MSG(!Thread::is_main_thread(), "Manually triggering the draw function from the RenderingServer can only be done on the main thread. Call this function from the main thread or use call_deferred().");
 	// Needs to be done before changes is reset to 0, to not force the editor to redraw.
 	RS::get_singleton()->emit_signal(SNAME("frame_pre_draw"));
 	changes = 0;
 	if (create_thread) {
-		command_queue.push(this, &RenderingServerDefault::_draw, p_swap_buffers, frame_step);
+		command_queue.push(this, &RenderingServerDefault::_draw, p_present, frame_step);
 	} else {
-		_draw(p_swap_buffers, frame_step);
+		_draw(p_present, frame_step);
 	}
 }
 
diff --git a/servers/rendering/rendering_server_default.h b/servers/rendering/rendering_server_default.h
index 29b1e163c78a..4a0df4a0cf3b 100644
--- a/servers/rendering/rendering_server_default.h
+++ b/servers/rendering/rendering_server_default.h
@@ -1124,7 +1124,7 @@ class RenderingServerDefault : public RenderingServer {
 
 	virtual void request_frame_drawn_callback(const Callable &p_callable) override;
 
-	virtual void draw(bool p_swap_buffers, double frame_step) override;
+	virtual void draw(bool p_present, double frame_step) override;
 	virtual void sync() override;
 	virtual bool has_changed() const override;
 	virtual void init() override;


Test patch: