Repository: conan-io/conan

Issue:
Conan fails to detect the compiler although `CC` and `CXX` are set
### Environment details

* Operating System+version: ubuntu-22.04
* Compiler+version: 13.1.0
* Conan version: 2.0.9
* Python version: 3.11.4


### Steps to reproduce

I am creating a Conda package of my software, as part of a Github C/I workflow. My software depends on other software (e.g.: Boost). For dependencies for which there isn't a Conda package available I use Conan to install them.

Here is the idea, scattered over the various files (`.github/workflows/linux-conda.yml`, `environment/conda/build.sh`):
- Create a Conda environment
- Install the boa Conda package builder and execute: `conda mambabuild environment/conda --override-channels --channel conda-forge`

Then `build.sh` is executed which includes:

```bash
conan profile detect                                                                                             
conan install . \                                                           
    --profile=default \                                                                                          
    --settings:host compiler.cppstd=17 \                                                                         
    --settings:build compiler.cppstd=17 \                                                                        
    --settings:build build_type=Release \                                                                        
    --build=missing \                                                                                            
    --output-folder=build
```

This results in an error (see log below). Conan is not able to correctly detect the compiler. The `CC` and `CXX` variables are set, so I would expect that those are used to detect the compiler.

I tried passing in the compiler explicitly, when calling `conan install`, using `--setting compiler=$CXX`, but that failed with `ERROR: 'settings.compiler' value not defined`. Maybe because the whole compiler section is missing from the profile?

I think this is a bug, given that the compiler is installed and pointed to by the standard variables. If not, then my question is how to get Conan to detect the compiler correctly in this situation.

I use the same conan commands to install Conan packages outside of the Conda build environment (on Linux, macOS, Windows), and those work fine. It seems that the Conda build environment is confusing Conan. Maybe because of the atypical setting of `CC` and `CXX`?

### Logs

```
2023-08-24T16:10:09.7983374Z CC and CXX: /usr/share/miniconda/envs/test/conda-bld/lue_1692893106258/_build_env/bin/x86_64-conda-linux-gnu-cc, /usr/share/miniconda/envs/test/conda-bld/lue_1692893106258/_build_env/bin/x86_64-conda-linux-gnu-c++ 
2023-08-24T16:10:09.7984827Z ERROR: Not able to automatically detect '/usr/share/miniconda/envs/test/conda-bld/lue_1692893106258/_build_env/bin/x86_64-conda-linux-gnu-cc' version
2023-08-24T16:10:09.7985898Z No compiler was detected (one may not be needed)
2023-08-24T16:10:09.7986712Z WARN: This profile is a guess of your environment, please check it.
2023-08-24T16:10:09.7987819Z WARN: The output of this command is not guaranteed to be stable and can change in future Conan versions.
2023-08-24T16:10:09.7988301Z WARN: Use your own profile files for stability.
2023-08-24T16:10:09.7988673Z Saving detected profile to /home/runner/.conan2/profiles/default
2023-08-24T16:10:09.8382941Z + conan install . --profile=default --settings:host compiler.cppstd=17 --settings:build compiler.cppstd=17 --settings:build build_type=Release --build=missing --output-folder=build
2023-08-24T16:10:10.1836158Z ERROR: 'settings.compiler' value not defined
```

Somewhat later on in the log:

```
2023-08-24T16:10:10.3898835Z Detected profile:
2023-08-24T16:10:10.3899230Z [settings]
2023-08-24T16:10:10.3899457Z arch=x86_64
2023-08-24T16:10:10.3899668Z build_type=Release
2023-08-24T16:10:10.3900101Z os=Linux
```


Discussion:
Hi @kordejong 

It seems that the command ``<compiler> --dumpversion`` is failing in this setup. Could you please execute the command and paste here the output?

The relevant code is:
```python
ret, out = detect_runner('%s -dumpversion' % compiler_exe)
if ret != 0:
    return None
compiler = "gcc"
installed_version = re.search(r"([0-9]+(\.[0-9])?)", out).group()
```

In any case, the right approach is not to rely on the auto-detect, not pass it in the settings, but define the input ``-s compiler.version=7`` right version in your command (better put it in a profile file than use the command line), and forget about the conan autodetect profile. As suggested in the docs, the output of this command should not be trusted in production.
Hi @memsharded,

Thanks for the feedback!

> It seems that the command `<compiler> --dumpversion` is failing in this setup. Could you please execute the command and paste here the output?

The output is: `13.1.0`

> In any case, the right approach is not to rely on the auto-detect, not pass it in the settings, but define the input `-s compiler.version=7` right version in your command (better put it in a profile file than use the command line), and forget about the conan autodetect profile. As suggested in the docs, the output of this command should not be trusted in production.

I understand, but I don't control the Conda build environment. The compiler version can change at any time, for example. My current strategy is to call `conan detect` and then pass knowledge I do know (`compiler.cppstd`, `compiler.build_type`) to `conan install`. I think the auto-detection of the environment is useful in this case.

If I have to I can add logic to my `build.sh` to write a profile, like you suggest. The Conda build environment has various variables I can use for that. I would be replicating some of the logic from `conan profile detect` though ;-) .
> The output is: 13.1.0

Uhm, I was expecting something different. Is this the whole output? No other characters or text? It is output to stdout or stderr?
It seems the code above works for this return, and the regex is valid, ``installed_version`` should be "13.1", not sure why it is failing for your case

Do you know how to run Conan from source in that environment? Adding a few prints() in that code might give some more hints.
I will try to find out what is going on.
I have the same problem. The autodetect is broken within a conda environment (Ubuntu LTS 22.04, x86) with the cxx-compiler package (this is a generic way to ensure that the end-user has a modern C++ compiler installed even if they don't have any sudo access on the system)

**My settings :** 

* Conan version 2.0.13
* Ubuntu LTS 22.04, x86
* conda 1.4.2 
* mamba 23.3.1
* gcc (installed via mamba cxx-compiler) : 12.3.0-2

In a [previous project](https://github.com/GDelevoye/cigarfilter/blob/main/CMakeLists.txt) , I used conan 1.59 with autodetect and it worked fine within conda/cxx-compiler (still works today), which was really great. But today I tried to modernize my build toolchain and it fails with conan v2.0

The workaround that I found was to check my gcc versionmanually and edit the ~/.conan/profiles/XXXX manually to specify the right parameters by hand like : 

```
[settings]
arch=x86_64
build_type=Release
compiler=gcc # This was missing
compiler.cppstd=gnu20 # This was missing
compiler.libcxx=libstdc++11 # This was missing
compiler.version=12 # This was missing
os=Linux # This was there
```

Then it worked. 

I know and understand that the autodetect is not necessarily 100% reliable on all systems, and I truly understand that but I still really upvote this issue. If it can work on Ubuntu, Windows and MacOS with conda, that would be really great for the scientific community. The end-users must often install the software on clusters, remote servers, or professional laptops where they don't have any sudo access; nor necessarly a modern compiler installed, and many of them are not programmers. Developpers, on the other hand, are often single +/- isolated programmers who want to broadcast their software quick&dirty with conda, AND use conan to spare themselves the nightmare of the dependencies. Using conda+conan allows to bypass all these issues very easily for everyone

Hi @GDelevoye 

Thanks for the feedback.
We are definitely wanting to try to fix it and make it work. It would be great to have some help and understand why it would be failing in those environment. The current code to detect compiler is:
```python
def _gcc_compiler(compiler_exe="gcc"):
        if platform.system() == "Darwin":
            # In Mac OS X check if gcc is a fronted using apple-clang
            _, out = detect_runner("%s --version" % compiler_exe)
            out = out.lower()
            if "clang" in out:
                return None, None

        ret, out = detect_runner('%s -dumpversion' % compiler_exe)
        if ret != 0:
            return None, None
        compiler = "gcc"
        installed_version = re.search(r"([0-9]+(\.[0-9])?)", out).group()
        # Since GCC 7.1, -dumpversion return the major version number
        # only ("7"). We must use -dumpfullversion to get the full version
        # number ("7.1.1").
        if installed_version:
            ConanOutput(scope="detect_api").info("Found %s %s" % (compiler, installed_version))
            return compiler, Version(installed_version)
```

What is the output of running ``gcc -dumpversion`` in your setup?

Never heard back from @kordejong, did you manage to check what was happening in your setup?
> Never heard back from @kordejong, did you manage to check what was happening in your setup?

No, still on my todo list to look into this, but flooded with other stuff right now. I wrote a script which writes a profile, replicating some of Conan's autodectect logic.

@memsharded Here is the dumpversion on my system :

```console
(base) delevoye@Dell-G15-5530:~$ which gcc
/home/delevoye/miniforge3/bin/gcc
(base) delevoye@Dell-G15-5530:~$ gcc -dumpversion
12.3.0
(base) delevoye@Dell-G15-5530:~$ conda deactivate
delevoye@Dell-G15-5530:~$ which gcc
/usr/bin/gcc
delevoye@Dell-G15-5530:~$ gcc -dumpversion
11
delevoye@Dell-G15-5530:~$ gcc -dumpfullversion
11.4.0
delevoye@Dell-G15-5530:~$ 
```

Regarding CC and CXX

```console
(base) delevoye@Dell-G15-5530:~$ echo $CC$
/home/delevoye/miniforge3/bin/x86_64-conda-linux-gnu-cc$
(base) delevoye@Dell-G15-5530:~$ echo $CXX$
/home/delevoye/miniforge3/bin/x86_64-conda-linux-gnu-c++$
(base) delevoye@Dell-G15-5530:~$ conda deactivate
delevoye@Dell-G15-5530:~$ echo $CC$
$
delevoye@Dell-G15-5530:~$ echo $CXX$
$
```
Whereis/Which

```console
(base) delevoye@Dell-G15-5530:~$ whereis gcc
gcc: /usr/bin/gcc /usr/lib/gcc /usr/share/gcc /home/delevoye/miniforge3/bin/gcc /usr/share/man/man1/gcc.1.gz
(base) delevoye@Dell-G15-5530:~$ echo $CX$
$
(base) delevoye@Dell-G15-5530:~$ echo $CXX$
/home/delevoye/miniforge3/bin/x86_64-conda-linux-gnu-c++$
(base) delevoye@Dell-G15-5530:~$ conda deactivate
delevoye@Dell-G15-5530:~$ whereis gcc
gcc: /usr/bin/gcc /usr/lib/gcc /usr/share/gcc /usr/share/man/man1/gcc.1.gz
```

Maybe what I can suggest here is to always use fulldumpversion, and do something without a regular expression like : 

```python
stdout_gcc = my_magic_function_dumpfullversion()
major_version = stdout_gcc.split('.')[0] # Should work with both long and and short versions
if major_version:
    try:
        minor_version = ".".join(stdout_gcc.split('.')[1:])
    except IndexError:
        minor_version = 0
```

Note also that the conda environment uses an abstraction layer with symlinks as was visible in OP

```
2023-08-24T16:10:09.7984827Z ERROR: Not able to automatically detect '/usr/share/miniconda/envs/test/conda-bld/lue_1692893106258/_build_env/bin/x86_64-conda-linux-gnu-cc' version
```

I have no idea how much this symlink system varies between windows, Ubuntu and MacOS for conda, and I have no idea of how much it impacts the way you autodetect

> Maybe what I can suggest here is to always use fulldumpversion,

There are flavors of ``gcc`` that don't support ``-dumpfullversion``, we found that ``-dumpversion`` was the most common one.
Regarding the CC, CXX variables the code is:
```python
 cc = os.environ.get("CC", "")
    cxx = os.environ.get("CXX", "")
    if cc or cxx:  # Env defined, use them
        output.info("CC and CXX: %s, %s " % (cc or "None", cxx or "None"))
        command = cc or cxx
```
So it seems you are checking the wrong env-var ``CX`` is incorrect, need to check ``CC``.
Then it would be good to know what would be the output of:
```
/usr/share/miniconda/envs/test/conda-bld/lue_1692893106258/_build_env/bin/x86_64-conda-linux-gnu-cc -dumpversion
```
I think that is the one that the detect is failing to parse. So far it seems that the detection process is correctly using that compiler from the environment, calling it, and just failing to understand its output.

> There are flavors of gcc that don't support -dumpfullversion, we found that -dumpversion was the most common one.

[this SO post](https://stackoverflow.com/questions/45168516/gcc-7-1-1-on-fedora-26-dumpversion-now-only-includes-major-version-by-default) suggests an easy way around it that works for older versions too

```console
gcc -dumpfullversion -dumpversion
```

> So it seems you are checking the wrong env-var CX is incorrect, need to check CC.

I changed CX by CC in my previous post and I replaced the output with the correct result

>  Then it would be good to know what would be the output of:
> ```console
>    /usr/share/miniconda/envs/test/conda-bld/lue_1692893106258/_build_env/bin/x86_64-conda-linux-gnu-cc --dumpversion
>

Maybe you meant a single dash instead of double dash ?

If I try both :

```console
(base) delevoye@Dell-G15-5530:~$ whereis x86_64-conda-linux-gnu-cc
x86_64-conda-linux-gnu-cc: /home/delevoye/miniforge3/bin/x86_64-conda-linux-gnu-cc
(base) delevoye@Dell-G15-5530:~$ x86_64-conda-linux-gnu-cc -dumpversion
12.3.0
(base) delevoye@Dell-G15-5530:~$ x86_64-conda-linux-gnu-cc --dumpversion
x86_64-conda-linux-gnu-cc: error: unrecognized command-line option '--dumpversion'; did you mean '-dumpversion'?
x86_64-conda-linux-gnu-cc: fatal error: no input files
compilation terminated.
(base) delevoye@Dell-G15-5530:~$ x86_64-conda-linux-gnu-cc -dumpfullversion
12.3.0
(base) delevoye@Dell-G15-5530:~$ x86_64-conda-linux-gnu-cc --dumpfullversion
x86_64-conda-linux-gnu-cc: error: unrecognized command-line option '--dumpfullversion'; did you mean '-dumpfullversion'?
x86_64-conda-linux-gnu-cc: fatal error: no input files
compilation terminated.
```
> Maybe you meant a single dash instead of double dash ?

Yes, definitely, my mistake, fixed.
I cannot see why ``12.3.0`` would fail to be parsed. I am trying to understand the possible source. I assume that it is printed to ``stdout`` and not ``stderr``, is this correct?

A this point, the best option could be to add some ``prints()`` to the detect function:
```python
def _gcc_compiler(compiler_exe="gcc"):
    try:
        if platform.system() == "Darwin":
            # In Mac OS X check if gcc is a fronted using apple-clang
            _, out = detect_runner("%s --version" % compiler_exe)
            out = out.lower()
            if "clang" in out:
                return None, None

        ret, out = detect_runner('%s -dumpversion' % compiler_exe)
        if ret != 0:
            return None, None
        compiler = "gcc"
        installed_version = re.search(r"([0-9]+(\.[0-9])?)", out).group()
        # Since GCC 7.1, -dumpversion return the major version number
        # only ("7"). We must use -dumpfullversion to get the full version
        # number ("7.1.1").
        if installed_version:
            ConanOutput(scope="detect_api").info("Found %s %s" % (compiler, installed_version))
            return compiler, Version(installed_version)
    except (Exception,):  # to disable broad-except
        return None, None
```
And debug what is happening. If you need guidance or help, I might be able to connect like right now or on monday. I am also available in the CppLang slack, so we could discuss and debug it live, that might be the fastest way to sort this out.
Hi @GDelevoye 

Did you manage to add those prints and check what is happening? That would help to move and fix this.

Resolving patch:
diff --git a/conan/internal/api/detect_api.py b/conan/internal/api/detect_api.py
--- a/conan/internal/api/detect_api.py
+++ b/conan/internal/api/detect_api.py
@@ -276,7 +276,7 @@ def detect_default_compiler():
         command = cc or cxx
         if "clang" in command.lower():
             return _clang_compiler(command)
-        if "gcc" in command or "g++" in command or "c++" in command:
+        if "gnu-cc" in command or "gcc" in command or "g++" in command or "c++" in command:
             gcc, gcc_version, compiler_exe = _gcc_compiler(command)
             if platform.system() == "Darwin" and gcc is None:
                 output.error("%s detected as a frontend using apple-clang. "


Test patch: