Repository: godotengine/godot (language: cpp)

Issue:
Godot crash on Linux when compiled with GCC 12
### Tested versions

Launch Godot.
I compiled Godot master from git using this settings:

```
       _,met$$$$$gg.          binogure@binogure 
    ,g$$$$$$$$$$$$$$$P.       ----------------- 
  ,g$$P"     """Y$$.".        OS: Debian GNU/Linux trixie/sid x86_64 
 ,$$P'              `$$$.     Host: A520 AORUS ELITE 
',$$P       ,ggs.     `$$b:   Kernel: 6.11-amd64 
`d$$'     ,$P"'   .    $$$    Uptime: 8 hours, 35 mins 
 $$P      d$'     ,    $$P    Packages: 4712 (dpkg) 
 $$:      $$.   -    ,d$$'    Shell: bash 5.2.37 
 $$;      Y$b._   _,d$P'      Resolution: 2560x1440, 2560x1440 
 Y$$.    `.`"Y$$$$P"'         DE: Cinnamon 6.2.9 
 `$$b      "-.__              WM: Mutter (Muffin) 
  `Y$$                        WM Theme: cinnamon (Default) 
   `Y$$.                      Theme: Adwaita [GTK2/3] 
     `$$b.                    Icons: gnome [GTK2/3] 
       `Y$$b.                 Terminal: terminator 
          `"Y$b._             CPU: AMD Ryzen 7 5800X (16) @ 4.851GHz 
              `"""            GPU: NVIDIA GeForce RTX 3070 
                              Memory: 18014MiB / 64226MiB 
gcc (Debian 12.2.0-14) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
gcc (Debian 12.2.0-14) 12.2.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```

Using git bisect, it seems that this commit causes the crash:
be86ce3103d92e4f7a625cbf11866266473ac99c

### System information

Linux binogure 6.11-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.11-1~exp1 (2024-09-19) x86_64 GNU/Linux

### Issue description

Launch Godot and it crashes on a SIGSEGV:
```
handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.4.dev.custom_build (be86ce3103d92e4f7a625cbf11866266473ac99c)
Dumping the backtrace. Please include this when reporting the bug to the project developer.
```

### Steps to reproduce

Launch Godot on Linux

https://github.com/user-attachments/assets/3406bc5a-0f10-402b-9353-ccde9003e852


### Minimal reproduction project (MRP)

N/A


Discussion / hints:
> Using git bisect, it seems that this commit causes the crash:
> be86ce3

That sounds surprising, this commit is just removing some unused headers. If the code compiles, it should be equivalent to what was before.
Would you mind trying to build the binary on your computer using our own linux toolchain?

https://github.com/godotengine/buildroot

You can just get a pre-build version from the "obtaining a toolchain" part of the document.

This will tell us if this is a Debian problem, or a Debian GCC problem.

Thanks!
Just sharing some more info:
```
ldd --version
ldd (Debian GLIBC 2.36-9+deb12u7) 2.36
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
```

I pulled the latest master, and compiled it. Still crashing on Linux, just opening Godot crashes.

Since I've started compiling the master branch and reverted this commit: be86ce3103d92e4f7a625cbf11866266473ac99c
I'm gonna wait for it to end before trying anything else. If it works, then I'll tell you, if it does not, I'm gonna try with gcc 13.
Reverting be86ce3103d92e4f7a625cbf11866266473ac99c fix the crash on Linux.
Since it's mostly headers that's been removed/moved/replaced I think it may be another underlying problem. 
> Would you mind trying to build the binary on your computer using our own linux toolchain?
> 
> https://github.com/godotengine/buildroot
> 
> You can just get a pre-build version from the "obtaining a toolchain" part of the document.
> 
> This will tell us if this is a Debian problem, or a Debian GCC problem.
> 
> Thanks!

Using this toolchain solve the issue too. So I guess I can't use gcc older than v13. It might be useful to update the documentation about building Godot for Linux and to make it matches with the toolchain tools versions:
https://docs.godotengine.org/en/stable/contributing/development/compiling/compiling_for_linuxbsd.html
I suspect it might be an incremental build issue, where SCons might not properly trigger rebuilding all files that need to be built, and some old object files are reused and linked into a broken executable.

Does the problem fix itself if you do a clean build from scratch with Debian's GCC? You can clean everything in a Git clone with `git clean -fxd`, though note it will remove any `.gitignore`d files, including potential IDE config files in that folder, etc. Alternatively you can clean with `scons [options] -c`.
> I suspect it might be an incremental build issue, where SCons might not properly trigger rebuilding all files that need to be built, and some old object files are reused and linked into a broken executable.
> 
> Does the problem fix itself if you do a clean build from scratch with Debian's GCC? You can clean everything in a Git clone with `git clean -fxd`, though note it will remove any `.gitignore`d files, including potential IDE config files in that folder, etc. Alternatively you can clean with `scons [options] -c`.

The first build was a clean build not an incremental one, still crashing.
While the others were incremental, just pulling the faulty commit fixed the issue.

To be clear, I'm using a VM dedicated to build Godot. So libs and binaries never get updated what so ever.
I've building Godot 4 since Godot 4.1, using this VM. Usually when Godot crashes, the first thing I do, is removing the scons DB, so it triggers a clean build. Also, the git repository is clean:
```sh
Sur la branche master
Fichiers non suivis:
  (utilisez "git add <fichier>..." pour inclure dans ce qui sera validé)
	build-consolidate.sh
	build-linux.sh
	build-windows.bat
	godot.gdkey
	modules/godotdiscord/
	modules/godoterrorhandler/
	modules/godotsteam/

aucune modification ajoutée à la validation mais des fichiers non suivis sont présents (utilisez "git add" pour les suivre)
```
I do have to say tho, there are indeed 3 modules that I use, and I haven't tried building Godot without them. Shell scripts are there because I use them to build Godot, and the `godot.gdkey` contains the encryption key I use to build Godot.

I do think that adding a link to the toolchain used by the github action to the documentation might be  the solution to this issue. Saying, if it crashes but you're not using this toolchain to build Godot, then "we cannot help much". Please use the specific tools we use to build Godot.
As requested, a `git clean` and `scons -c` to produce a fresh build (sorry for the delay):
![Image](https://github.com/user-attachments/assets/1a47d731-e9c0-4ae1-a93d-62b9a4635b72)

Still crashing indeed and I don't think it's an incremental build issue.

Also, I added the following commands before compiling, just to be sure to produce a clean build
```sh
      find . -name "*.lib" -not -path "*/sdk/*" -print -delete
      find . -name "*.o" -not -path "*/sdk/*" -print -delete
      find . -name "*.obj" -not -path "*/sdk/*" -print -delete
      find . -name "*.ox" -not -path "*/sdk/*" -print -delete
      find . -name "*.a" -not -path "*/sdk/*" -print -delete
      find . -name "*.ax" -not -path "*/sdk/*" -print -delete
      find . -name "*.d" -not -path "*/sdk/*" -print -delete
      find . -name "*.so" -not -path "*/sdk/*" -print -delete
      find . -name "*.os" -not -path "*/sdk/*" -print -delete
      find . -name "*.Plo" -not -path "*/sdk/*" -print -delete
      find . -name "*.lo" -not -path "*/sdk/*" -print -delete
      find . -name "*.gen.*" -not -path "*/sdk/*" -print -delete
```

I'm also seeing crashes when godot is built on debian (`gcc (Debian 12.2.0-14) 12.2.0`), also tried with a clean build, same thing.

Stack trace for project settings:

```
Thread 1 "godot.linuxbsd." received signal SIGSEGV, Segmentation fault.
0x0000555558ac55ae in Node::is_inside_tree (this=0x0) at scene/main/node.h:489
489		_FORCE_INLINE_ bool is_inside_tree() const { return data.inside_tree; }
(gdb) bt
#0  0x0000555558ac55ae in Node::is_inside_tree (this=0x0) at ./scene/main/node.h:489
#1  0x000055555b73ad0d in Window::set_title (this=0x0, p_title=...) at scene/main/window.cpp:281
#2  0x000055555a73e253 in ProjectManager::ProjectManager (this=0x5555640d1b60) at editor/project_manager.cpp:1142
#3  0x0000555558867679 in Main::start () at main/main.cpp:4290
#4  0x000055555878d7b0 in main (argc=1, argv=0x7fffffffdd68) at platform/linuxbsd/godot_linuxbsd.cpp:84
```

Stack trace with `-e --path path/to/project`:

```
0x0000555558799c22 in std::__atomic_base<unsigned long>::fetch_sub (__m=std::memory_order_acq_rel, __i=1, this=0x3f7ffffffffffff0) at /usr/include/c++/12/bits/atomic_base.h:628
628	      { return __atomic_fetch_sub(&_M_i, __i, int(__m)); }
(gdb) bt
#0  0x0000555558799c22 in std::__atomic_base<unsigned long>::fetch_sub (__m=std::memory_order_acq_rel, __i=1, this=0x3f7ffffffffffff0) at /usr/include/c++/12/bits/atomic_base.h:628
#1  SafeNumeric<unsigned long>::decrement (this=0x3f7ffffffffffff0) at ./core/templates/safe_refcount.h:87
#2  CowData<char>::_unref (this=0x5555637a8e18) at ./core/templates/cowdata.h:273
#3  0x00005555587ced4d in CowData<char>::operator= (this=0x5555637a8e18, p_from=...) at ./core/templates/cowdata.h:180
#4  0x00005555587c91aa in CharString::operator= (this=0x5555637a8e18, p_str=...) at ./core/string/ustring.h:244
#5  0x000055555cfbc693 in ShaderRD::initialize (this=0x5555637a8e10, p_variant_defines=..., p_general_defines=..., r_immutable_samplers=...) at servers/rendering/renderer_rd/shader_rd.cpp:774
#6  0x000055555cffe73f in RendererRD::GI::init (this=0x5555637a84b8, p_sky=0x5555637a7b80) at servers/rendering/renderer_rd/environment/gi.cpp:3443
#7  0x000055555cfb37a4 in RendererSceneRenderRD::init (this=0x5555637a7970) at servers/rendering/renderer_rd/renderer_scene_render_rd.cpp:1471
#8  0x000055555cfa7ca7 in RendererCompositorRD::RendererCompositorRD (this=0x555561fae800) at servers/rendering/renderer_rd/renderer_compositor_rd.cpp:348
#9  0x00005555587cea3d in RendererCompositorRD::_create_current () at ./servers/rendering/renderer_rd/renderer_compositor_rd.h:142
#10 0x000055555cda9215 in RendererCompositor::create () at servers/rendering/renderer_compositor.cpp:42
#11 0x000055555ceba9eb in RenderingServerDefault::_init (this=0x555561825140) at servers/rendering/rendering_server_default.cpp:216
#12 0x000055555cebae46 in RenderingServerDefault::init (this=0x555561825140) at servers/rendering/rendering_server_default.cpp:255
#13 0x000055555885e1f9 in Main::setup2 (p_show_boot_logo=true) at main/main.cpp:3156
#14 0x000055555885b81e in Main::setup (execpath=0x7fffffffe0d5 "/media/SSD/Code/git/godot-dev/bin/godot.linuxbsd.editor.dev.x86_64", argc=3, argv=0x7fffffffdd40, p_second_phase=true) at main/main.cpp:2685
#15 0x000055555878d77c in main (argc=4, argv=0x7fffffffdd38) at platform/linuxbsd/godot_linuxbsd.cpp:74
```

Using llvm (`Debian clang version 14.0.6`) works fine, so indeed, it seems we broke builds with gcc 12.
The stacktrace sounds relevant for you @RandomShaper @hpvb 

Still very puzzled at how a simple cleanup PR removing unused headers would cause this, but if it's a compiler bug, who knows... Maybe it was only working up until now by pure chance

Would be good to try to narrow down further what part of be86ce3 triggers this. From reading the diff I'd guess the removal of Semaphore in ResourceLoader + its use of some forward declares may be playing nasty?
@xsellier can you tell us how exactly you build the engine? The exact scons command you use, and the versions for gcc, linker, etc you use?

This does seem like a toolchain bug of some kind, but I'd like to know exactly what the bug is, we might be triggering it elsewhere too.
> [@xsellier](https://github.com/xsellier) can you tell us how exactly you build the engine? The exact scons command you use, and the versions for gcc, linker, etc you use?
> 
> This does seem like a toolchain bug of some kind, but I'd like to know exactly what the bug is, we might be triggering it elsewhere too.

```shell
#!/bin/bash
# This script is intended to run on Linux.

set -eo pipefail

export BUILD_REVISION=official

export NB_CPU=16
export CXX=g++
export CC=gcc

echo ""
echo "BUILDING BINARY FOR Linux 64"
echo ""
scons -j${NB_CPU} CXX=$CXX CC=$CC platform=linuxbsd target=editor debug_symbols=yes arch=x86_64 use_static_cpp=yes
echo ""
echo "BUILD DONE FOR Linux 64"
echo ""
```
Tested on Fedora 41 with our older buildroot using GCC 10.2.0, and it seems to work fine.
https://github.com/godotengine/buildroot/releases/tag/godot-2020.11.x-2

So at least it's not that we broke support for all GCC < 13, the problem seems to be related to Debian's version or other related dependencies specifically. Time to set up a Debian VM.

*Edit:* I can reproduce it in the `debian:12` Docker image (with `podman`), and I confirm that a local revert of be86ce3 works it around.
So far I've narrowed it down to the changes in `core/os/` from be86ce3. Reverting only those fixes the issue in my first tests.

I thought the culprit might be the change to `core/os/thread_local.h` that's actually wrong - it needs `Mutex` to be functional, but static analysis missed it as it's only used in macros. But only reverting that file doesn't seem to fix the issue.

I'll check further when I have time.

Fix patch:
diff --git a/core/os/thread.cpp b/core/os/thread.cpp
index afc74364f652..3992eb31bf5f 100644
--- a/core/os/thread.cpp
+++ b/core/os/thread.cpp
@@ -29,13 +29,13 @@
 /**************************************************************************/
 
 #include "platform_config.h"
-#ifndef PLATFORM_THREAD_OVERRIDE // See details in thread.h
+
+#ifndef PLATFORM_THREAD_OVERRIDE // See details in thread.h.
 
 #include "thread.h"
 
 #ifdef THREADS_ENABLED
 #include "core/object/script_language.h"
-#include "core/templates/safe_refcount.h"
 
 SafeNumeric<uint64_t> Thread::id_counter(1); // The first value after .increment() is 2, hence by default the main thread ID should be 1.
 
diff --git a/core/os/thread.h b/core/os/thread.h
index 066c4befa77d..9042c6c01716 100644
--- a/core/os/thread.h
+++ b/core/os/thread.h
@@ -29,19 +29,28 @@
 /**************************************************************************/
 
 #include "platform_config.h"
+
 // Define PLATFORM_THREAD_OVERRIDE in your platform's `platform_config.h`
-// to use a custom Thread implementation defined in `platform/[your_platform]/platform_thread.h`
-// Overriding the platform implementation is required in some proprietary platforms
+// to use a custom Thread implementation defined in `platform/[your_platform]/platform_thread.h`.
+// Overriding the Thread implementation is required in some proprietary platforms.
+
 #ifdef PLATFORM_THREAD_OVERRIDE
+
 #include "platform_thread.h"
+
 #else
 
 #ifndef THREAD_H
 #define THREAD_H
 
-#include "core/templates/safe_refcount.h"
 #include "core/typedefs.h"
 
+#ifdef THREADS_ENABLED
+
+#include "core/templates/safe_refcount.h"
+
+#include <new> // IWYU pragma: keep // For hardware interference size.
+
 #ifdef MINGW_ENABLED
 #define MINGW_STDTHREAD_REDUNDANCY_WARNING
 #include "thirdparty/mingw-std-threads/mingw.thread.h"
@@ -53,8 +62,6 @@
 
 class String;
 
-#ifdef THREADS_ENABLED
-
 class Thread {
 public:
 	typedef void (*Callback)(void *p_userdata);
@@ -143,6 +150,8 @@ class Thread {
 
 #else // No threads.
 
+class String;
+
 class Thread {
 public:
 	typedef void (*Callback)(void *p_userdata);
diff --git a/core/os/thread_safe.h b/core/os/thread_safe.h
index 316b0c96876f..f49592a4ecb0 100644
--- a/core/os/thread_safe.h
+++ b/core/os/thread_safe.h
@@ -31,6 +31,8 @@
 #ifndef THREAD_SAFE_H
 #define THREAD_SAFE_H
 
+#include "core/os/mutex.h" // IWYU pragma: keep // Used in macro.
+
 #define _THREAD_SAFE_CLASS_ mutable Mutex _thread_safe_;
 #define _THREAD_SAFE_METHOD_ MutexLock _thread_safe_method_(_thread_safe_);
 #define _THREAD_SAFE_LOCK_ _thread_safe_.lock();


Test patch: