Repository: bitcoin/bitcoin (language: cpp)

Issue:
ci: ccache does not work for macOS cross-compiling builds
https://cirrus-ci.com/task/6268895175639040?logs=ci#L3869:
```
ccache version 3.7.7
cache directory                     /tmp/ccache_dir
primary config                      /tmp/ccache_dir/ccache.conf
secondary config      (readonly)    /etc/ccache.conf
stats updated                       Tue Mar 30 01:48:58 2021
stats zeroed                        Tue Mar 30 01:21:25 2021
cache hit (direct)                     0
cache hit (preprocessed)               0
cache miss                             0
cache hit rate                      0.00 %
called for link                       10
multiple source files                636
cleanups performed                     0
files in cache                         0
cache size                           0.0 kB
max cache size                     200.0 MB
```

UPD 2023-08-25. See https://github.com/ccache/ccache/issues/1326.


Discussion / hints:
This line
```
multiple source files                636
```
describes a little. According to `ccache` [docs](https://ccache.dev/manual/4.2.1.html#_cache_statistics):
> The compiler was called to compile multiple source files in one go. This is not supported by ccache.
@dongcarl 

`ccache` behavior was broken in 300733921863c176535806c40afdc813b99e7459 (#19683).
Was this not the case before 300733921863c176535806c40afdc813b99e7459? I'm not entirely sure how that commit makes it so that "the compiler was called to compile multiple source files in one go"
> I'm not entirely sure how that commit makes it so that "the compiler was called to compile multiple source files in one go"

I believe this is because `ccache` do not support invoking a compiler that prefixed with `env` command.

FWIW, if I inject `ccache` between `env` and `clang`, it works fine.
> FWIW, if I inject `ccache` between `env` and `clang`, it works fine.

What is the ccache stats output if you inject `ccache` between `env` and `clang`?
@dongcarl 

With the following patch
```diff
diff --git a/depends/hosts/darwin.mk b/depends/hosts/darwin.mk
index dd71697f0..8a7c3a791 100644
--- a/depends/hosts/darwin.mk
+++ b/depends/hosts/darwin.mk
@@ -98,6 +98,7 @@ $(foreach TOOL,$(cctools_TOOLS),$(eval darwin_$(TOOL) = $$(build_prefix)/bin/$$(
 darwin_CC=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
               -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \
               -u LIBRARY_PATH \
+            /usr/bin/ccache \
             $(clang_prog) --target=$(host) -mmacosx-version-min=$(OSX_MIN_VERSION) \
               -B$(build_prefix)/bin -mlinker-version=$(LD64_VERSION) \
               --sysroot=$(OSX_SDK) \
@@ -106,6 +107,7 @@ darwin_CC=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
 darwin_CXX=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
                -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \
                -u LIBRARY_PATH \
+            /usr/bin/ccache \
              $(clangxx_prog) --target=$(host) -mmacosx-version-min=$(OSX_MIN_VERSION) \
                -B$(build_prefix)/bin -mlinker-version=$(LD64_VERSION) \
                --sysroot=$(OSX_SDK) \
```

- the first run
```
$ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin18/share/config.site ./configure --disable-ccache
$ ccache --zero-stats
$ make
$ ccache --show-stats 
cache directory                     /home/hebasto/.ccache
primary config                      /home/hebasto/.ccache/ccache.conf
secondary config      (readonly)    /etc/ccache.conf
stats updated                       Fri Apr  9 14:03:35 2021
stats zeroed                        Fri Apr  9 13:59:19 2021
cache hit (direct)                   521
cache hit (preprocessed)               1
cache miss                           647
cache hit rate                     44.65 %
called for link                       12
cleanups performed                     2
files in cache                     24070
cache size                          13.0 GB
max cache size                      15.0 GB
```
- the second run
```
$ make clean
$ ccache --zero-stats
$ make
$ ccache --show-stats 
cache directory                     /home/hebasto/.ccache
primary config                      /home/hebasto/.ccache/ccache.conf
secondary config      (readonly)    /etc/ccache.conf
stats updated                       Fri Apr  9 14:04:47 2021
stats zeroed                        Fri Apr  9 14:04:15 2021
cache hit (direct)                   644
cache hit (preprocessed)               2
cache miss                             1
cache hit rate                     99.85 %
called for link                       11
cleanups performed                     0
files in cache                     24073
cache size                          13.0 GB
max cache size                      15.0 GB
```
Urgh... That kinda sucks... I think the easiest solution is perhaps to wait until ccache 4.1 is available on all distros and set `compiler_type`/`CCACHE_COMPILERTYPE`?
> to wait until ccache 4.1 is available on all distros

@dongcarl What do you mean by "all distros"?
Sorry, probably better rephrased as "available on stable versions of major distros"... Sometimes I think we might need a configure script for depends lol
> "available on stable versions of major distros"

* Debian 10 (old stable) has ccache 3.6
* Debian 11 (stable) has ccache 4.2
* Ubuntu 20.04 (latest LTS) has ccache 3.7.7
* Ubuntu 21.04 has ccache 4.2
* Ubuntu 21.10 has ccache 4.2.1
* Ubuntu 22.04 (upcoming LTS) has ccache 4.5.1

src: https://repology.org/project/ccache/versions
@dongcarl 
> Urgh... That kinda sucks... I think the easiest solution is perhaps to wait until ccache 4.1 is available on all distros and set `compiler_type`/`CCACHE_COMPILERTYPE`?

I've tested this approach on Ubuntu 22.04 with ccache 4.5.1, and it doesn't work.

OTOH, another [approach](https://github.com/bitcoin/bitcoin/issues/21552#issuecomment-813709017) has been implemented in #24620.
Not just when cross-compiling. I made a separate issue #25387, because I'm not sure if it's the same (ccache 4.6.1).

(nvm: it does work as far as I can tell, but doesn't use it by default)
Given it's not in the description, can someone summarize the current state of this issue?

* What doesn't work?
* What ccache options have been tried (and don't work)? Taking into account recent ccache releases.
* Have we opened an issue upstream to either report that ccache is broken (for us), or that we have a usecase which isn't satified by it's current feature/option set, to see if they are open to adding a new feature/option?
* If there is any (related) upstream discussions/issues, can somebody link to them here?
> Given it's not in the description, can someone summarize the current state of this issue?
> 
>    * What doesn't work?
> 
>    * Have we opened an issue upstream to either report that ccache is broken (for us), or that we have a usecase which isn't satified by it's current feature/option set, to see if they are open to adding a new feature/option?
> 
>    * If there is any (related) upstream discussions/issues, can somebody link to them here?

See https://github.com/ccache/ccache/issues/1326.
> Urgh... That kinda sucks... I think the easiest solution is perhaps to wait until ccache 4.1 is available on all distros and set `compiler_type`/`CCACHE_COMPILERTYPE`?

Tested with ccache 4.8.2. It does not work.

Fix patch:
diff --git a/contrib/guix/libexec/build.sh b/contrib/guix/libexec/build.sh
index 9bc8c0e75d7cc..fa6933b3328f2 100755
--- a/contrib/guix/libexec/build.sh
+++ b/contrib/guix/libexec/build.sh
@@ -71,11 +71,9 @@ unset OBJCPLUS_INCLUDE_PATH
 
 export C_INCLUDE_PATH="${NATIVE_GCC}/include"
 export CPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
-export OBJC_INCLUDE_PATH="${NATIVE_GCC}/include"
-export OBJCPLUS_INCLUDE_PATH="${NATIVE_GCC}/include/c++:${NATIVE_GCC}/include"
 
 case "$HOST" in
-    *darwin*) export LIBRARY_PATH="${NATIVE_GCC}/lib" ;;
+    *darwin*) export LIBRARY_PATH="${NATIVE_GCC}/lib" ;; # Required for qt/qmake
     *mingw*) export LIBRARY_PATH="${NATIVE_GCC}/lib" ;;
     *)
         NATIVE_GCC_STATIC="$(store_path gcc-toolchain static)"
@@ -180,6 +178,14 @@ make -C depends --jobs="$JOBS" HOST="$HOST" \
                                    x86_64_linux_NM=x86_64-linux-gnu-gcc-nm \
                                    x86_64_linux_STRIP=x86_64-linux-gnu-strip
 
+case "$HOST" in
+    *darwin*)
+        # Unset now that Qt is built
+        unset C_INCLUDE_PATH
+        unset CPLUS_INCLUDE_PATH
+        unset LIBRARY_PATH
+        ;;
+esac
 
 ###########################
 # Source Tarball Building #
diff --git a/depends/hosts/darwin.mk b/depends/hosts/darwin.mk
index 564381d1e9308..a50e36110dc02 100644
--- a/depends/hosts/darwin.mk
+++ b/depends/hosts/darwin.mk
@@ -50,17 +50,11 @@ darwin_STRIP=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-strip")
 #         Disable adhoc codesigning (for now) when using LLVM tooling, to avoid
 #         non-determinism issues with the Identifier field.
 
-darwin_CC=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
-              -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \
-              -u LIBRARY_PATH \
-              $(clang_prog) --target=$(host) \
+darwin_CC=$(clang_prog) --target=$(host) \
               -isysroot$(OSX_SDK) -nostdlibinc \
               -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks
 
-darwin_CXX=env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH \
-               -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH \
-               -u LIBRARY_PATH \
-               $(clangxx_prog) --target=$(host) \
+darwin_CXX=$(clangxx_prog) --target=$(host) \
                -isysroot$(OSX_SDK) -nostdlibinc \
                -iwithsysroot/usr/include/c++/v1 \
                -iwithsysroot/usr/include -iframeworkwithsysroot/System/Library/Frameworks
diff --git a/depends/packages/qt.mk b/depends/packages/qt.mk
index d35139dd2de3e..b133539495982 100644
--- a/depends/packages/qt.mk
+++ b/depends/packages/qt.mk
@@ -23,6 +23,7 @@ $(package)_patches += memory_resource.patch
 $(package)_patches += clang_18_libpng.patch
 $(package)_patches += utc_from_string_no_optimize.patch
 $(package)_patches += windows_lto.patch
+$(package)_patches += darwin_no_libm.patch
 $(package)_patches += zlib-timebits64.patch
 
 $(package)_qttranslations_file_name=qttranslations-$($(package)_suffix)
@@ -236,6 +237,7 @@ define $(package)_preprocess_cmds
   patch -p1 -i $($(package)_patch_dir)/utc_from_string_no_optimize.patch && \
   patch -p1 -i $($(package)_patch_dir)/guix_cross_lib_path.patch && \
   patch -p1 -i $($(package)_patch_dir)/windows_lto.patch && \
+  patch -p1 -i $($(package)_patch_dir)/darwin_no_libm.patch && \
   patch -p1 -i $($(package)_patch_dir)/zlib-timebits64.patch && \
   mkdir -p qtbase/mkspecs/macx-clang-linux &&\
   cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\
diff --git a/depends/patches/qt/darwin_no_libm.patch b/depends/patches/qt/darwin_no_libm.patch
new file mode 100644
index 0000000000000..38a94beeb7a69
--- /dev/null
+++ b/depends/patches/qt/darwin_no_libm.patch
@@ -0,0 +1,17 @@
+build: remove explicit -lm link from qttools
+    
+This causes issues with at least the macOS cross build, and shouldn't
+actually be required anywhere else. GCC with libstdc++ will already get libm.
+
+--- a/qtbase/src/corelib/tools/tools.pri
++++ b/qtbase/src/corelib/tools/tools.pri
+@@ -111,9 +111,6 @@ qtConfig(easingcurve) {
+         tools/qtimeline.cpp
+ }
+ 
+-# Note: libm should be present by default becaue this is C++
+-unix:!macx-icc:!vxworks:!haiku:!integrity:!wasm: LIBS_PRIVATE += -lm
+-
+ TR_EXCLUDE += ../3rdparty/*
+ 
+ # MIPS DSP


Test patch: