Repository: conda/conda-build

Issue:
Unexpected `binary is None` in `liefldd.py`
### Checklist

- [X] I added a descriptive title
- [X] I searched open reports and couldn't find a duplicate

### What happened?

With the last conda-smithy release, conda-forge is now back to using lief 0.12.3, and failing in https://github.com/conda-forge/compiler-rt-feedstock/pull/61 with:
```
  File "/Users/runner/miniforge3/lib/python3.10/site-packages/conda_build/os_utils/liefldd.py", line 406, in inspect_linkages_lief
    if binary.format == lief.EXE_FORMATS.ELF:
AttributeError: 'NoneType' object has no attribute 'format'
```

### Conda Info

```shell
conda-forge CI for osx-64 / osx-arm64
```


### Conda Config

```shell
conda-forge CI for osx-64 / osx-arm64
```


### Conda list

host env:
```shell
environment location: /Users/runner/miniforge3/conda-bld/compiler-rt-packages_1677355103841/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol


The following NEW packages will be INSTALLED:

    clang:           16.0.0.rc3-h9d310f0_0         conda-forge/label/llvm_rc
    clang-16:        16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clang-format:    16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clang-format-16: 16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clang-tools:     16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clangdev:        16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    clangxx:         16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    icu:             70.1-h96cf925_0               conda-forge              
    libclang:        16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    libclang-cpp:    16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    libclang-cpp16:  16.0.0.rc3-default_h5b99a12_0 conda-forge/label/llvm_rc
    libclang13:      16.0.0.rc3-default_h4e9855e_0 conda-forge/label/llvm_rc
    libcxx:          14.0.6-hccf4f1f_0             conda-forge              
    libiconv:        1.17-hac89ed1_0               conda-forge              
    libllvm16:       16.0.0.rc3-h8456b07_1         conda-forge/label/llvm_rc
    libxml2:         2.10.3-hb9e07b5_0             conda-forge              
    libzlib:         1.2.13-hfd90126_4             conda-forge              
    llvm-tools:      16.0.0.rc3-h8456b07_1         conda-forge/label/llvm_rc
    llvmdev:         16.0.0.rc3-h8456b07_1         conda-forge/label/llvm_rc
    xz:              5.2.6-h775f41a_0              conda-forge              
    zlib:            1.2.13-hfd90126_4             conda-forge              
    zstd:            1.5.2-hbc0c0cd_6              conda-forge 
```
build env:
```
  environment location: /Users/runner/miniforge3/conda-bld/compiler-rt-packages_1677355103841/_build_env

The following NEW packages will be INSTALLED:

    bzip2:                  1.0.8-h0d85af4_4          conda-forge
    c-ares:                 1.18.1-h0d85af4_0         conda-forge
    ca-certificates:        2022.12.7-h033912b_0      conda-forge
    clang_bootstrap_osx-64: 14.0.6-h6ddedbb_4         conda-forge
    cmake:                  3.25.2-h4032537_0         conda-forge
    expat:                  2.5.0-hf0c8a7f_0          conda-forge
    krb5:                   1.20.1-h049b76e_0         conda-forge
    libcurl:                7.88.1-h6df9250_0         conda-forge
    libcxx:                 14.0.6-hccf4f1f_0         conda-forge
    libedit:                3.1.20191231-h0678c8f_2   conda-forge
    libev:                  4.33-haf1e3a3_1           conda-forge
    libffi:                 3.4.2-h0d85af4_5          conda-forge
    libnghttp2:             1.51.0-he2ab024_0         conda-forge
    libsqlite:              3.40.0-ha978bb4_0         conda-forge
    libssh2:                1.10.0-h47af595_3         conda-forge
    libuv:                  1.44.2-hac89ed1_0         conda-forge
    libzlib:                1.2.13-hfd90126_4         conda-forge
    ncurses:                6.3-h96cf925_1            conda-forge
    ninja:                  1.11.1-hb8565cd_0         conda-forge
    openssl:                3.0.8-hfd90126_0          conda-forge
    python:                 3.11.0-he7542f4_1_cpython conda-forge
    readline:               8.1.2-h3899abd_0          conda-forge
    rhash:                  1.4.3-hac89ed1_0          conda-forge
    tk:                     8.6.12-h5dbffcc_0         conda-forge
    tzdata:                 2022g-h191b570_0          conda-forge
    xz:                     5.2.6-h775f41a_0          conda-forge
    zlib:                   1.2.13-hfd90126_4         conda-forge
    zstd:                   1.5.2-hbc0c0cd_6          conda-forge
```


### Additional Context

Failing CI run [here](https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=664408&view=results)


Discussion:
Welp, just hit this as well, when it worked on Linux and Windows. Is it possible to work around it by downgrading to an earlier version of conda-build?
Hi @h-vetinari,

Thanks for filing this bug. I dug a little deeper on this, and it turns out our usage of this particular library in conda-build is a bit fragile. In my opinion, I think it's probably to time refactor https://github.com/conda/conda-build/blob/main/conda_build/os_utils/liefldd.py and reconsider the way we use the `lief` package as a dependency in that module.

I've added the https://github.com/conda/conda-build/labels/good-first-issue label because this could be a reasonable task for someone new to the repository. Otherwise, we unfortunately cannot promise when this will be addressed.
@1player,

I don't think it's possible to work around this with an earlier version of conda-build. After viewing a git blame, I could see this code was added five years ago.

I think the work around fix for this is to somehow force install the version of `lief` that conda-build works with. I am currently not aware of which version this is though. This is a very undesirable route to go though, in my opinion.
Looks like there's an upstream bug to track for this: https://github.com/lief-project/LIEF/issues/873
The easiest way to work-around this would be to skip files ending with `.a` at https://github.com/conda/conda-build/blob/c5146a02c1b86033a7989718b1511969838db5dd/conda_build/post.py#L738
Collecting of dependencies of `.a` static libraries have never worked and is not expected to work as they do not have dependency info on them.
Thanks a lot for the insight @isuruf! I opened a PR: https://github.com/conda/conda-build/pull/4808
> The easiest way to work-around this would be to skip files ending with `.a` at
> 
> https://github.com/conda/conda-build/blob/c5146a02c1b86033a7989718b1511969838db5dd/conda_build/post.py#L738
> 
> Collecting of dependencies of `.a` static libraries have never worked and is not expected to work as they do not have dependency info on them.

I don't think that's a good approach.

I've been hitting this issue because we are packaging LLVM, and it is crucial the `clang_rt-builtins.a` file is present in the final package as it is needed for LLVM users. This issue affects only macOS, because we have no issues on Linux and that .a file.
I think you misunderstand (or I do!)... This is not a question whether we package static libs (there are cases that are unavoidable, like compiler-builtins that you mention), but whether we run the lief-machinery on them that inspects the artefacts for their dependencies (and fails).
Any update? We're stuck because we need to build compiler-rt, and there are two PR that fix this exact issue.
yes, I agree. I ran into same issue with llvm/clang 16 building ... I patched lief/conda-build for it
I will share a PR with conda-build on the changes I made to it. In general it would be pretty good to have a switch in conda-build, which could be used to turn off the lief checks
There are two unmerged PRs that solve this bug already. 

They just need someone to review and merge one or the other. 

On Wed, 24 May 2023, at 18:12, Kai Tietz wrote:
> 
> 
> I will share a PR with conda-build on the changes I made to it. In general it would be pretty good to have a switch in conda-build, which could be used to turn off the lief checks
> 
> 
> —
> Reply to this email directly, view it on GitHub <https://github.com/conda/conda-build/issues/4787#issuecomment-1561638320>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFIPSGP5ZJNQK3B7RH5YLLXHY6P5ANCNFSM6AAAAAAVII4MKI>.
> You are receiving this because you were mentioned.Message ID: ***@***.***>
> 

I just read through them.  I think they both fix the issue here in particular, but there might be a way to resolve it more generic.  
Actually we should try to stop to run on something binary is  'NoneType' . But yes, looking and filtering out specific file-endings would be good.

Btw why this filtering of file-endings was removed anyway.  IIRC there was such a feature in the past, and now we attempt to run over any files in PREFIX with a lot of noise

Resolving patch:
diff --git a/conda_build/os_utils/liefldd.py b/conda_build/os_utils/liefldd.py
--- a/conda_build/os_utils/liefldd.py
+++ b/conda_build/os_utils/liefldd.py
@@ -232,6 +232,8 @@ def get_runpaths_or_rpaths_raw(file):
 
 def set_rpath(old_matching, new_rpath, file):
     binary = ensure_binary(file)
+    if not binary:
+        return
     if binary.format == lief.EXE_FORMATS.ELF and (
         binary.type == lief.ELF.ELF_CLASS.CLASS32
         or binary.type == lief.ELF.ELF_CLASS.CLASS64
@@ -343,7 +345,9 @@ def _get_path_dirs(prefix):
 
 def get_uniqueness_key(file):
     binary = ensure_binary(file)
-    if binary.format == lief.EXE_FORMATS.MACHO:
+    if not binary:
+        return lief.EXE_FORMATS.UNKNOWN
+    elif binary.format == lief.EXE_FORMATS.MACHO:
         return binary.name
     elif binary.format == lief.EXE_FORMATS.ELF and (  # noqa
         binary.type == lief.ELF.ELF_CLASS.CLASS32
@@ -463,7 +467,9 @@ def inspect_linkages_lief(
     sysroot = _trim_sysroot(sysroot)
 
     default_paths = []
-    if binary.format == lief.EXE_FORMATS.ELF:
+    if not binary:
+        default_paths = []
+    elif binary.format == lief.EXE_FORMATS.ELF:
         if binary.type == lief.ELF.ELF_CLASS.CLASS64:
             default_paths = [
                 "$SYSROOT/lib64",
@@ -491,6 +497,8 @@ def inspect_linkages_lief(
             filename2 = element[0]
             binary = element[1]
             uniqueness_key = get_uniqueness_key(binary)
+            if not binary:
+                continue
             if uniqueness_key not in already_seen:
                 parent_exe_dirname = None
                 if binary.format == lief.EXE_FORMATS.PE:


Test patch: