Repository: conan-io/conan
Pull request: Keep only one flavor of the same libs in `collect_libs()`
closes https://github.com/conan-io/conan/issues/11282
closes https://github.com/conan-io/conan/issues/11526

Changelog: Fix: In `conans.tools.collect_libs` and `conan.tools.files.collect_libs`, avoids to list other flavors (version, soversion) of the same `dylib` on macOS, so that behavior is consistent on Linux & macOS.
Docs: https://github.com/conan-io/docs/pull/2610

- [x] Refer to the issue that supports this Pull Request.
- [x] If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
- [x] I've read the [Contributing guide](https://github.com/conan-io/conan/blob/develop/.github/CONTRIBUTING.md).
- [x] I've followed the PEP8 style guides for Python code.
- [x] I've opened another PR in the Conan docs repo to the ``develop`` branch, documenting this one. 

<sup>**Note:** By default this PR will skip the slower tests and will use a limited set of python versions. Check [here](https://github.com/conan-io/conan/blob/develop/.github/PR_INCREASE_TESTING.md) how to increase the testing level by writing some tags in the current PR body text.</sup>


Problem statement:
[bug] tool.collect_libs behaves differently on mac and linux
Mac and Linux both creaete symlinks without package version and libs with package version.

Linux: libmylib.so -> llibmylib.so.X.Y.Z
Mac: libmylib.dylib -> llibmylib.X.Y.Z.dylib

tools.collect_libs with make a os.path.splitext and check if the extension is a valid lib extension. Both symlinks are fine:
 libmylib.so  -> ext = .so (valid and listed)
 libmylib.dylib -> ext = .dylib (valid and listed)
 
The actualy libs however behave defferntly:
 libmylib.so.X.Y.Z -> ext = .Z (not valid is discarded
 libmylib.X.Y.Z.dylib -> ext = .dylib (valid and listed)

Not sure what the corect behaviour should be probaly both should be listed in both cases. But behaving like this is clearly a bug


[bug] `collect_libs()` collects all flavors (version, soversion) of the same dylib
<!--
  Please don't forget to update the issue title.
  Include all applicable information to help us reproduce your problem.

  To help us debug your issue please explain:
-->

### Environment Details (include every applicable attribute)
  * Operating System+version: macOS Montery
  * Compiler+version: apple-clang 13
  * Conan version: 1.49.0
  * Python version: 3.9.13

### Steps to reproduce (Include if Applicable)

- call conan install with some generator on a recipe relying on `tools.collect_libs()` in which version and soversion are installed. For example `conan install proj/9.0.1@ -g CMakeDeps`
- there are multiple references of the same real lib due to various symlinks in lib folder, which are not reduced to the canonical lib name. Indeed, on macOS, version & soversion are added before `.dylib` extension, so `collect_libs()` also lists them.

### Logs (Executed commands with output) (Include/Attach if Applicable)

In lib folder of proj:

```bash
$ ls -la
total 10704
drwxr-xr-x  5 spaceim  staff      160 16 jui 10:34 .
drwxr-xr-x  9 spaceim  staff      288 16 jui 10:34 ..
-rwxr-xr-x  1 spaceim  staff  5480152 16 jui 10:34 libproj.25.9.0.1.dylib
lrwxr-xr-x  1 spaceim  staff       22 16 jui 10:34 libproj.25.dylib -> libproj.25.9.0.1.dylib
lrwxr-xr-x  1 spaceim  staff       16 16 jui 10:34 libproj.dylib -> libproj.25.dylib
```

`tools.collect_libs()` returns `proj`, `proj.25` & `proj.25.9.0.1`, while it could be smart and only returns `proj` by grouping symlinks and real file together, and keeping only the shortest name in each group.

Hints from issue discussion:
Jeah, `collect_lib` should collect all libs (even symlinks) and also it should report
a list of (`fileName`, `path`) instead of only the filenames, to allow the user to check other stuff on these files... discarding symlinks.

On linux the additional check:

```python
".so." in name
```
should be done...
https://github.com/conan-io/conan/pull/11527 implements a consistent behavior. But it keeps only the most generic lib name since `collect_libs` usually aims to populate `self.cpp_info.libs` in `package_info()`.

Commit message: no more warnings if same lib found several times

Indeed, it happens all the time with symlinks

Patch to review:
diff --git a/conan/tools/files/files.py b/conan/tools/files/files.py
index 7eca6af893f..9aebe7970dd 100644
--- a/conan/tools/files/files.py
+++ b/conan/tools/files/files.py
@@ -494,24 +494,29 @@ def collect_libs(conanfile, folder=None):
     else:
         lib_folders = [os.path.join(conanfile.package_folder, folder)
                        for folder in conanfile.cpp_info.libdirs]
-    result = []
+
+    ref_libs = {}
     for lib_folder in lib_folders:
         if not os.path.exists(lib_folder):
             conanfile.output.warn("Lib folder doesn't exist, can't collect libraries: "
                                   "{0}".format(lib_folder))
             continue
+        # In case of symlinks, only keep shortest file name in the same "group"
         files = os.listdir(lib_folder)
         for f in files:
             name, ext = os.path.splitext(f)
             if ext in (".so", ".lib", ".a", ".dylib", ".bc"):
-                if ext != ".lib" and name.startswith("lib"):
-                    name = name[3:]
-                if name in result:
-                    conanfile.output.warn("Library '%s' was either already found in a previous "
-                                          "'conanfile.cpp_info.libdirs' folder or appears several "
-                                          "times with a different file extension" % name)
-                else:
-                    result.append(name)
+                real_lib = os.path.basename(os.path.realpath(os.path.join(lib_folder, f)))
+                if real_lib not in ref_libs or len(f) < len(ref_libs[real_lib]):
+                    ref_libs[real_lib] = f
+
+    result = []
+    for f in ref_libs.values():
+        name, ext = os.path.splitext(f)
+        if ext != ".lib" and name.startswith("lib"):
+            name = name[3:]
+        if name not in result:
+            result.append(name)
     result.sort()
     return result
 
diff --git a/conans/client/tools/files.py b/conans/client/tools/files.py
index 199581c0ee6..a8444b23c07 100644
--- a/conans/client/tools/files.py
+++ b/conans/client/tools/files.py
@@ -351,24 +351,29 @@ def collect_libs(conanfile, folder=None):
     else:
         lib_folders = [os.path.join(conanfile.package_folder, folder)
                        for folder in conanfile.cpp_info.libdirs]
-    result = []
+
+    ref_libs = {}
     for lib_folder in lib_folders:
         if not os.path.exists(lib_folder):
             conanfile.output.warn("Lib folder doesn't exist, can't collect libraries: "
                                   "{0}".format(lib_folder))
             continue
+        # In case of symlinks, only keep shortest file name in the same "group"
         files = os.listdir(lib_folder)
         for f in files:
             name, ext = os.path.splitext(f)
             if ext in VALID_LIB_EXTENSIONS:
-                if ext != ".lib" and name.startswith("lib"):
-                    name = name[3:]
-                if name in result:
-                    conanfile.output.warn("Library '%s' was either already found in a previous "
-                                          "'conanfile.cpp_info.libdirs' folder or appears several "
-                                          "times with a different file extension" % name)
-                else:
-                    result.append(name)
+                real_lib = os.path.basename(os.path.realpath(os.path.join(lib_folder, f)))
+                if real_lib not in ref_libs or len(f) < len(ref_libs[real_lib]):
+                    ref_libs[real_lib] = f
+
+    result = []
+    for f in ref_libs.values():
+        name, ext = os.path.splitext(f)
+        if ext != ".lib" and name.startswith("lib"):
+            name = name[3:]
+        if name not in result:
+            result.append(name)
     result.sort()
     return result
 
diff --git a/conans/test/unittests/tools/files/collect_lib_test.py b/conans/test/unittests/tools/files/collect_lib_test.py
index e862ce75ba0..3d72419ea5c 100644
--- a/conans/test/unittests/tools/files/collect_lib_test.py
+++ b/conans/test/unittests/tools/files/collect_lib_test.py
@@ -43,21 +43,6 @@ def test_collect_libs():
     result = collect_libs(conanfile, folder="custom_folder")
     assert ["customlib"] == result
 
-    # Warn same lib different folders
-    conanfile = ConanFileMock()
-    conanfile.folders.set_base_package(temp_folder())
-    conanfile.cpp_info = CppInfo(conanfile.name, "")
-    custom_mylib_path = os.path.join(conanfile.package_folder, "custom_folder", "mylib.lib")
-    lib_mylib_path = os.path.join(conanfile.package_folder, "lib", "mylib.lib")
-    save(custom_mylib_path, "")
-    save(lib_mylib_path, "")
-    conanfile.cpp_info.libdirs = ["lib", "custom_folder"]
-    result = collect_libs(conanfile)
-    assert ["mylib"] == result
-    assert "Library 'mylib' was either already found in a previous "\
-           "'conanfile.cpp_info.libdirs' folder or appears several times with a "\
-           "different file extension" in conanfile.output
-
     # Warn lib folder does not exist with correct result
     conanfile = ConanFileMock()
     conanfile.folders.set_base_package(temp_folder())
diff --git a/conans/test/unittests/util/tools_test.py b/conans/test/unittests/util/tools_test.py
index 2be7e00e8e2..4930b568632 100644
--- a/conans/test/unittests/util/tools_test.py
+++ b/conans/test/unittests/util/tools_test.py
@@ -817,21 +817,6 @@ def test_collect_libs(self):
         result = tools.collect_libs(conanfile, folder="custom_folder")
         self.assertEqual(["customlib"], result)
 
-        # Warn same lib different folders
-        conanfile = ConanFileMock()
-        conanfile.folders.set_base_package(temp_folder())
-        conanfile.cpp_info = CppInfo(conanfile.name, "")
-        custom_mylib_path = os.path.join(conanfile.package_folder, "custom_folder", "mylib.lib")
-        lib_mylib_path = os.path.join(conanfile.package_folder, "lib", "mylib.lib")
-        save(custom_mylib_path, "")
-        save(lib_mylib_path, "")
-        conanfile.cpp_info.libdirs = ["lib", "custom_folder"]
-        result = tools.collect_libs(conanfile)
-        self.assertEqual(["mylib"], result)
-        self.assertIn("Library 'mylib' was either already found in a previous "
-                      "'conanfile.cpp_info.libdirs' folder or appears several times with a "
-                      "different file extension", conanfile.output)
-
         # Warn lib folder does not exist with correct result
         conanfile = ConanFileMock()
         conanfile.folders.set_base_package(temp_folder())
@@ -880,21 +865,6 @@ def test_self_collect_libs(self):
         result = conanfile.collect_libs(folder="custom_folder")
         self.assertEqual(["customlib"], result)
 
-        # Warn same lib different folders
-        conanfile = ConanFileMock()
-        conanfile.folders.set_base_package(temp_folder())
-        conanfile.cpp_info = CppInfo("", "")
-        custom_mylib_path = os.path.join(conanfile.package_folder, "custom_folder", "mylib.lib")
-        lib_mylib_path = os.path.join(conanfile.package_folder, "lib", "mylib.lib")
-        save(custom_mylib_path, "")
-        save(lib_mylib_path, "")
-        conanfile.cpp_info.libdirs = ["lib", "custom_folder"]
-        result = conanfile.collect_libs()
-        self.assertEqual(["mylib"], result)
-        self.assertIn("Library 'mylib' was either already found in a previous "
-                      "'conanfile.cpp_info.libdirs' folder or appears several times with a "
-                      "different file extension", conanfile.output)
-
         # Warn lib folder does not exist with correct result
         conanfile = ConanFileMock()
         conanfile.folders.set_base_package(temp_folder())


Code review comments:
diff_hunk: @@ -43,21 +43,6 @@ def test_collect_libs():
     result = collect_libs(conanfile, folder="custom_folder")
     assert ["customlib"] == result
 
-    # Warn same lib different folders
-    conanfile = ConanFileMock()
-    conanfile.folders.set_base_package(temp_folder())
-    conanfile.cpp_info = CppInfo(conanfile.name, "")
-    custom_mylib_path = os.path.join(conanfile.package_folder, "custom_folder", "mylib.lib")
-    lib_mylib_path = os.path.join(conanfile.package_folder, "lib", "mylib.lib")
-    save(custom_mylib_path, "")
-    save(lib_mylib_path, "")
-    conanfile.cpp_info.libdirs = ["lib", "custom_folder"]
-    result = collect_libs(conanfile)
-    assert ["mylib"] == result
-    assert "Library 'mylib' was either already found in a previous "\
path: conans/test/unittests/tools/files/collect_lib_test.py
text: @user1:
Why this test is being removed? Maybe it is fine to remove the warning check, but this test should still work and ``result == ["mylib"]``?
Also, it is also that necessary to remove the warning? If ``collect_libs()`` is decided to use one specific name in a group of related ones, isn't it good to show some information about this?

@author:
Yes, the first assert is still valid. The second assert fails if I keep the warning, because now there is a first pass grouping the same libs in all folders, either symlinks or real file, and it loses the capability to track duplicated real files (but it can still detect libs with different extensions). Displaying a warning for symlinks is unnecessary noise. I can try to track duplicated real files, but I'm not sure it's worth the effort and overhead.

@author:
I'll add tests for symlinks

@user1:
Good, lets keep the first asset, and I agree, not worth the effort to keep the warning, it can be removed.