Repository: single-cell-data/TileDB-SOMA (language: python)

Issue:
[Bug] Can't install tiledbsoma for Python on Fedora 40
**Describe the bug**

Hi all!

Unfortunately, I don't manage to install the latest version of tiledbsoma on my Fedora 40 system. I managed to preproduce it with a Fedora 40 container. I didn't try on other platforms.

Thanks in advance!

**To Reproduce**

```bash
docker run --rm -it fedora:40

dnf install -y python3-devel python3-pip git g++ make

pip install tiledbsoma
```


[Full log](https://github.com/user-attachments/files/17172139/pip-install-tiledbsoma.txt)


      ...
          [ 89%] Linking CXX static library libCatch2.a
          [ 89%] Built target Catch2
          [ 89%] Building CXX object _deps/catch2-build/src/CMakeFiles/Catch2WithMain.dir/catch2/internal/catch_main.cpp.o
          [ 90%] Linking CXX static library libCatch2Main.a
          [ 90%] Built target Catch2WithMain
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:286: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/logger.cc.o] Error 1
          gmake[5]: *** Waiting for unfinished jobs....
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:328: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/version.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:188: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/soma/soma_context.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:244: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/soma/array_buffers.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:76: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/reindexer/reindexer.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:258: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/soma/column_buffer.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:90: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/soma/managed_query.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:314: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/util.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:272: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/utils/arrow_adapter.cc.o] Error 1
          cc1plus: all warnings being treated as errors
          gmake[5]: *** [src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/build.make:104: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/soma/soma_array.cc.o] Error 1
          gmake[4]: *** [CMakeFiles/Makefile2:211: src/CMakeFiles/TILEDB_SOMA_OBJECTS.dir/all] Error 2
          gmake[3]: *** [Makefile:146: all] Error 2
          gmake[2]: *** [CMakeFiles/libtiledbsoma.dir/build.make:86: libtiledbsoma-prefix/src/libtiledbsoma-stamp/libtiledbsoma-build] Error 2
          gmake[1]: *** [CMakeFiles/Makefile2:142: CMakeFiles/libtiledbsoma.dir/all] Error 2
          gmake: *** [Makefile:91: all] Error 2
          Checking: /tmp/pip-install-6wj9pvw9/tiledbsoma_afd25373037d471e8556541bbc3a3ce2/dist_links/dist/lib/libtiledbsoma.so exists: False
          Checking: /tmp/pip-install-6wj9pvw9/tiledbsoma_afd25373037d471e8556541bbc3a3ce2/dist_links/dist/lib64/libtiledbsoma.so exists: False
          Checking: /tmp/pip-install-6wj9pvw9/tiledbsoma_afd25373037d471e8556541bbc3a3ce2/dist_links/dist/lib/x86_64-linux-gnu/libtiledbsoma.so exists: False
          libtiledbsoma.so: cannot open shared object file: No such file or directory
          Traceback (most recent call last):
            File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
              main()
            File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
              json_out['return_val'] = hook(**hook_input['kwargs'])
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
              return _build_backend().build_wheel(wheel_directory, config_settings,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 421, in build_wheel
              return self._build_with_temp_dir(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
              self.run_setup()
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
              exec(code, locals())
            File "<string>", line 269, in <module>
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
              return distutils.core.setup(**attrs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 183, in setup
              return run_commands(dist)
                     ^^^^^^^^^^^^^^^^^^
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
              dist.run_commands()
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
              self.run_command(cmd)
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 950, in run_command
              super().run_command(command)
            File "/tmp/pip-build-env-mm81khxq/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
              cmd_obj.run()
            File "<string>", line 224, in run
            File "<string>", line 186, in find_or_build_package_data
            File "/usr/lib64/python3.12/subprocess.py", line 571, in run
              raise CalledProcessError(retcode, process.args,
          subprocess.CalledProcessError: Command '['./bld', '--tiledb=/tmp/pip-install-6wj9pvw9/tiledbsoma_afd25373037d471e8556541bbc3a3ce2/dist_links/dist']' returned non-zero exit status 2.
          [end of output]
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      ERROR: Failed building wheel for tiledbsoma
    Failed to build tiledbsoma
    ERROR: Could not build wheels for tiledbsoma, which is required to install pyproject.toml-based projects




**Versions (please complete the following information):**
 - TileDB-SOMA version: 1.14.2
 - Language and language version (e.g. Python 3.9, R 4.3.2): Python 3.12
 - OS (e.g. MacOS, Ubuntu Linux): Fedora 40
 - Note: you can use `tiledbsoma.show_package_versions()` (Python) or `tiledbsoma::show_package_versions()` (R)

**Additional context**
Add any other context about the problem here.



Discussion / hints:
@rcannood taking a look!
Thanks @johnkerl !
@rcannood the issue seems to be errors of the form:

```
/tmp/TileDB-SOMA/build/externals/install/include/spdlog/fmt/bundled/core.h:1872:38:
required from ‘std::string fmt::v8::format(format_string<T ...>, T&& ...)
[with T = {std::basic_string_view<char, std::char_traits<char> >, long int}; std::string =
std::__cxx11::basic_string<char>; format_string<T ...> = basic_format_string<char,
std::basic_string_view<char, std::char_traits<char> >, long int>]’
```

-- we do use some known-good `fmt` versions within our Conda-feedstock builds, and IIRC have used v9 at some point in the past. I do believe v8 has always been "too old" for us.

The versions that are known-good for our Conda feedstock are

* `fmt` 9 `spdlog` 1.11
* `fmt` 11 `spdlog` 1.14

I'm a little puzzled here since with `docker run` of `fedora40` I see

```
# dnf list | grep spdlog
spdlog.i686         1.12.0-4.fc40 updates
spdlog.x86_64       1.12.0-4.fc40 updates
spdlog-devel.i686   1.12.0-4.fc40 updates
spdlog-devel.x86_64 1.12.0-4.fc40 updates
```

and 1.12 is _In between_ 1.11 and 1.14. And I get the same build error after `dnf install spdlog`.

And the `v8` is from here:

```
/tmp/TileDB-SOMA/build/externals/install/include/spdlog/fmt/bundled/core.h
```

At any rate -- I'll check with the team and circle back to see what recommendations we have.
@rcannood here is the issue. We are using some third-party code (which we don't control) and yet for our own code we want to compile with `-Werror`. It turns out that in that third-party code, there are some things that some compilers warn about & others don't. In this case the Fedora 40 compiler is emitting a warning, which is then fatal.

Here is a quick workaround for kick-the-tires mode:

* `cd /tmp`
* `git clone https://github.com/single-cell-data/TileDB-SOMA`
* Apply this diff:
```
diff --git a/libtiledbsoma/CMakeLists.txt b/libtiledbsoma/CMakeLists.txt
index 5d3d6d90..36361825 100644
--- a/libtiledbsoma/CMakeLists.txt
+++ b/libtiledbsoma/CMakeLists.txt
@@ -200,7 +200,8 @@ else()
   set(TILEDBSOMA_COMPILE_OPTIONS -Wall -Wextra)

   if(TILEDBSOMA_ENABLE_WERROR)
-    set(TILEDBSOMA_WERROR_OPTION -Werror)
+    ####set(TILEDBSOMA_WERROR_OPTION -Werror)
+    set(TILEDBSOMA_WERROR_OPTION "")
   else()
     set(TILEDBSOMA_WERROR_OPTION "")
   endif()
```
* `pip install apis/python`

This worked for me in a `docker run` of `fedora:40`.

We do need this to work out of the box, though, on a plain-vanilla `pip install`. I'll keep you posted.
Thanks! This workaround allows me to install tiledbsoma :relaxed: 

```bash
pip install apis/python
```

    Installing collected packages: tiledbsoma
      Attempting uninstall: tiledbsoma
        Found existing installation: tiledbsoma 1.12.0rc0.post203.dev1088558980
        Uninstalling tiledbsoma-1.12.0rc0.post203.dev1088558980:
          Successfully uninstalled tiledbsoma-1.12.0rc0.post203.dev1088558980

However, I do notice that cellxgene-census requires tiledbsoma>=1.12.3. Am I on the wrong branch or should I manually set `apis/python/RELEASE-VERSION` to something like 1.12.4rc0?
Update: I checked out tag 1.14.2 and applied the diff to that.

I'll mark this comment and the previous one as off-topic
@rcannood your previous two comments were IMO completely on-topic! :) 

My apologies, indeed I should have mentioned checking out tag 1.14.2, rather than the default which gave you tip of `main`.
cc @jp-dark @nguyenv using docker-run as above with `fedora:40` I applied the following test-patch:

```
diff --git a/libtiledbsoma/CMakeLists.txt b/libtiledbsoma/CMakeLists.txt
index 5d3d6d90..e85f67a7 100644
--- a/libtiledbsoma/CMakeLists.txt
+++ b/libtiledbsoma/CMakeLists.txt
@@ -200,7 +200,7 @@ else()
   set(TILEDBSOMA_COMPILE_OPTIONS -Wall -Wextra)

   if(TILEDBSOMA_ENABLE_WERROR)
-    set(TILEDBSOMA_WERROR_OPTION -Werror)
+    set(TILEDBSOMA_WERROR_OPTION "")
   else()
     set(TILEDBSOMA_WERROR_OPTION "")
   endif()
diff --git a/scripts/bld b/scripts/bld
index 5008e188..19bb9568 100755
--- a/scripts/bld
+++ b/scripts/bld
@@ -11,7 +11,7 @@ arg() { echo "$1" | sed "s/^${2-[^=]*=}//" | sed "s/:/;/g"; }
 build="Release"
 prefix=""
 tiledb=""
-cmake_verbose="false"
+cmake_verbose="true"
 no_tiledb_deprecated="false"

 while test $# != 0; do
```

and got a successful `pip install -e apis/python`

```
[root@629d17eb941a TileDB-SOMA]# uname -a
Linux 629d17eb941a 6.8.0-1016-aws #17-Ubuntu SMP Mon Sep  2 13:48:07 UTC 2024 x86_64 GNU/Linux
[root@629d17eb941a TileDB-SOMA]# python
bash: python: command not found
[root@629d17eb941a TileDB-SOMA]# python3
Python 3.12.6 (main, Sep  9 2024, 00:00:00) [GCC 14.2.1 20240801 (Red Hat 14.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tiledbsoma
>>> tiledbsoma.show_package_versions()
tiledbsoma.__version__              1.15.0rc0.post12.dev3893317782
TileDB core version (libtiledbsoma) 2.26.2
python version                      3.12.6.final.0
OS version                          Linux 6.8.0-1016-aws
```

While the above test-patch is of course not the final solution, it does prove that `-Werror` on third-party source is indeed the issue.

This suggests that our work on #2408 #2664 #2763 was not complete.

Fix patch:
diff --git a/.github/workflows/libtiledb-ci.yml b/.github/workflows/libtiledb-ci.yml
index fa8f2d619d..9bbfa562fc 100644
--- a/.github/workflows/libtiledb-ci.yml
+++ b/.github/workflows/libtiledb-ci.yml
@@ -19,7 +19,7 @@ jobs:
     - name: Checkout TileDB-SOMA
       uses: actions/checkout@v4
     - name: Build libTileDB-SOMA
-      run: TILEDBSOMA_COVERAGE="--coverage" ./scripts/bld --no-tiledb-deprecated=true
+      run: TILEDBSOMA_COVERAGE="--coverage" ./scripts/bld --no-tiledb-deprecated=true --werror=true
     - name: Run libTileDB-SOMA unittests
       run: ctest --test-dir build/libtiledbsoma -C Release --verbose --rerun-failed --output-on-failure
     - name: Upload coverage to Codecov
diff --git a/.github/workflows/python-ci-packaging.yml b/.github/workflows/python-ci-packaging.yml
index c470117b00..a85db66d09 100644
--- a/.github/workflows/python-ci-packaging.yml
+++ b/.github/workflows/python-ci-packaging.yml
@@ -92,6 +92,7 @@ jobs:
             -D OVERRIDE_INSTALL_PREFIX=OFF \
             -D DOWNLOAD_TILEDB_PREBUILT=OFF \
             -D TILEDB_REMOVE_DEPRECATIONS=ON \
+            -D TILEDBSOMA_ENABLE_WERROR=ON \
             -D FORCE_BUILD_TILEDB=OFF
           cmake --build build-libtiledbsoma -j $(nproc)
           cmake --build build-libtiledbsoma --target install-libtiledbsoma
diff --git a/libtiledbsoma/CMakeLists.txt b/libtiledbsoma/CMakeLists.txt
index 5d3d6d90be..7ee8e227ec 100644
--- a/libtiledbsoma/CMakeLists.txt
+++ b/libtiledbsoma/CMakeLists.txt
@@ -39,12 +39,17 @@ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake/Mo
 
 set(TILEDBSOMA_CMAKE_INPUTS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/cmake/inputs")
 
+# Check for and set environment variables.
+if (NOT DEFINED TILEDBSOMA_ENABLE_WERROR AND DEFINED $ENV{TILEDBSSOMA_ENABLE_WERROR})
+  set(TILEDBSOMA_ENABLE_WERROR "$ENV{TILEDBSOMA_ENABLE_WERROR}")
+endif()
+
 # TileDB-SOMA Options
 option(TILEDBSOMA_BUILD_PYTHON "Build Python API bindings" ON)
 option(TILEDBSOMA_BUILD_CLI "Build tiledbsoma CLI tool" ON)
 option(TILEDBSOMA_BUILD_STATIC "Build a static library; otherwise, shared library" OFF)
 option(TILEDBSOMA_ENABLE_TESTING "Enable tests" ON)
-option(TILEDBSOMA_ENABLE_WERROR "Enables the -Werror flag during compilation." ON)
+option(TILEDBSOMA_ENABLE_WERROR "Enables the -Werror flag during compilation." OFF)
 
 # Superbuild option must be on by default.
 option(SUPERBUILD "If true, perform a superbuild (builds all missing dependencies)." ON)
@@ -168,7 +173,7 @@ if(MSVC)
   set(TILEDBSOMA_COMPILE_OPTIONS /W4 /wd4101 /wd4146 /wd4244 /wd4251 /wd4456 /wd4457 /wd4702 /wd4800 /wd4996)
 
   # Warnings as errors:
-  if(TILEDBSOMA_ENABLE_WERROR)
+  if(${TILEDBSOMA_ENABLE_WERROR})
     set(TILEDBSOMA_WERROR_OPTION /WX)
   else()
     set(TILEDBSOMA_WERROR_OPTION "")
@@ -199,7 +204,7 @@ else()
 
   set(TILEDBSOMA_COMPILE_OPTIONS -Wall -Wextra)
 
-  if(TILEDBSOMA_ENABLE_WERROR)
+  if(${TILEDBSOMA_ENABLE_WERROR})
     set(TILEDBSOMA_WERROR_OPTION -Werror)
   else()
     set(TILEDBSOMA_WERROR_OPTION "")
diff --git a/scripts/bld b/scripts/bld
index 5008e188dc..a1355b302d 100755
--- a/scripts/bld
+++ b/scripts/bld
@@ -13,6 +13,7 @@ prefix=""
 tiledb=""
 cmake_verbose="false"
 no_tiledb_deprecated="false"
+werror="false"
 
 while test $# != 0; do
   case "$1" in
@@ -21,6 +22,7 @@ while test $# != 0; do
   --tiledb=*) tiledb=$(arg "$1");;
   --cmake-verbose=*) cmake_verbose=$(arg "$1");;
   --no-tiledb-deprecated=*) no_tiledb_deprecated=$(arg "$1");;
+  --werror=*) werror=$(arg "$1");;
   esac
   shift
 done
@@ -46,6 +48,8 @@ if [ "$(uname -m)" == "aarch64" ]; then
   extra_opts+=" -DDOWNLOAD_TILEDB_PREBUILT=OFF"
 fi
 
+
+
 # NOTE: set to true to debug the cmake build
 if [ "$cmake_verbose" = "true" ]; then
   # This is _incredibly_ helpful in that it reveals the actual compile lines etc which make itself
@@ -61,12 +65,16 @@ if [ "$cmake_verbose" = "true" ]; then
 
   # Also (pro-tip), set nproc=1 to get a more deterministic ordering of output lines.
   nproc=1
+elif [ "$werror" = "true" ]; then
+    extra_opts+=" -DTILEDBSOMA_ENABLE_WERROR=ON"
+
 fi
 
 if [ "$no_tiledb_deprecated" = "true" ]; then
     extra_opts+=" -DTILEDB_REMOVE_DEPRECATIONS=ON"
 fi
 
+
 # set installation path
 if [ -n "${prefix}"  ]; then
   extra_opts+=" -DCMAKE_INSTALL_PREFIX=${prefix} -DOVERRIDE_INSTALL_PREFIX=OFF"


Test patch:
diff --git a/.github/workflows/r-python-interop-testing.yml b/.github/workflows/r-python-interop-testing.yml
index 3cb006f4c7..e481205615 100644
--- a/.github/workflows/r-python-interop-testing.yml
+++ b/.github/workflows/r-python-interop-testing.yml
@@ -80,7 +80,7 @@ jobs:
       #  run: cd apis/r && Rscript -e "options(bspm.version.check=TRUE); install.packages('tiledb', repos = c('https://eddelbuettel.r-universe.dev/bin/linux/jammy/4.3/', 'https://cloud.r-project.org'))"
 
       - name: Build and install libtiledbsoma
-        run: sudo scripts/bld --prefix=/usr/local --no-tiledb-deprecated=true && sudo ldconfig
+        run: sudo scripts/bld --prefix=/usr/local --no-tiledb-deprecated=true --werror=true && sudo ldconfig
 
       - name: Install R-tiledbsoma
         run: |
@@ -100,7 +100,7 @@ jobs:
           cache-dependency-path: ./apis/python/setup.py
 
       - name: Install tiledbsoma
-        run: pip -v install -e apis/python[dev] -C "--build-option=--no-tiledb-deprecated"
+        run: pip -v install -e apis/python[dev] -C "--build-option=--no-tiledb-deprecated "
 
       - name: Show Python package versions
         run: |