Repository: pypa/cibuildwheel

Issue:
remove .so files when copying source into the container
Trying to build wheels on travis and I'm getting the following error:

```
auditwheel: error: cannot repair "/tmp/built_wheel/pydantic-0.27a1-cp37-cp37m-linux_x86_64.whl" to "manylinux1_x86_64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.
```

See [this build](https://travis-ci.org/samuelcolvin/pydantic/jobs/537258176) for details.

I don't get this error when building locally with `cibuildwheel --platform linux --output-dir dist`.

Any idea what I'm doing wrong?


Discussion:
possibly the same as https://github.com/pypa/auditwheel/issues/36#issuecomment-217965612 since the first image build goes fine, it's the second that fails.
sorted this at last, found the answer at https://github.com/pypa/manylinux/issues/214#issuecomment-409375930 - I was building the package on the host before running `cibuildwheel`.

`cibuildwheel` is copying the `.so` files along with the source into the container thus the problem.

`cibuildwheel` should exclude `.so` (and perhaps `.c`?) files when copying the source into the contianer.
Ah, thanks for tracking this down. Maybe a simple way around this would be to delete the 'build' directory inside the docker container before each build? Anything in that folder can be regenerated as part of the build, as I understand it.
That wouldn't help in my case, during testing I install/build with `pip install -e .` which means lots of `*.so` files along side `*.py` files.

Ideally `cibuildwheel` would either:
* delete all `.so` files in src before copying (with a warning)
* or better, filter out such files when copying with some kind of glob,
* or delete `.so` files inside the container before building
* or just fail with a clear error message if such files exist in the source directory
* or document this potential pitfall

Or some combination of the above. I'm afraid I can't be much help on this at the moment - I'm already spending far too much time on open source stuff, but thanks a lot for cibuildwheel, even with this problem it's saved me a lot of time.
> `cibuildwheel` is copying the `.so` files along with the source into the container thus the problem.
> 
> `cibuildwheel` should exclude `.so` (and perhaps `.c`?) files when copying the source into the contianer.

Maybe I don't fully understand the problem, but this seems like a problem on the user-side of `cibuildwheel`? `cibuildwheel` just makes sure the builds inside the container have access to all files in the `project` folder passed at invocation. It feels rather dangerous to start making assumptions and exclude certain types of files, if you ask me.

Correct me if I'm wrong, but the actual problem seems to be that the project is built twice? Which is in principle not a property of `cibuildwheel`, but a consequence of how it's used in the CI config?

Is there a possibility to clean up the build? Or maybe to run `cibuildwheel` in a separate job, such that your builds do not conflict?
> Is there a possibility to clean up the build? Or maybe to run cibuildwheel in a separate job, such that your builds do not conflict?

Yes that's what I'm doing now.

Maybe just a warning then and a note in docs?
Yeah, sure, if this can be confusing, it should be clarified. I'm just not sure where to add it and how general it should be. What kind of note are you thinking about? Something related to this specific problem, or a more general note on `cibuildwheel` actually making a full independent build?
question is what can cause the "presence of too-recent versioned symbols" error? Is it just `.so` dynamic library files or some other extensions? I'm guessing 99.99% of the time it'll be `.so` files.

I would say:
* add to the docs, "make sure your source directory doesn't include any build artifacts like `*.c` or `*.so` files before running `cibuildwheel`"
* if there are any `*.so` files in the build directory issue a warning during build saying something like "Build artifacts (`*.so` files) were found in your source directory "/path/to/src", this may cause cibuildwheel builds to fail. Consider deleting these files before calling cibuildwheel"

Resolving patch:
diff --git a/cibuildwheel/linux.py b/cibuildwheel/linux.py
--- a/cibuildwheel/linux.py
+++ b/cibuildwheel/linux.py
@@ -1,5 +1,5 @@
 from __future__ import print_function
-import os, subprocess, sys, uuid
+import os, subprocess, sys, uuid, textwrap
 from collections import namedtuple
 from .util import prepare_command, get_build_verbosity_extra_flags
 
@@ -173,8 +173,36 @@ def run_docker(command, stdin_str=None):
             run_docker(['cp', os.path.abspath(project_dir) + '/.', container_name + ':/project'])
             run_docker(['start', '-i', '-a', container_name], stdin_str=bash_script)
             run_docker(['cp', container_name + ':/output/.', os.path.abspath(output_dir)])
-        except subprocess.CalledProcessError:
+        except subprocess.CalledProcessError as error:
+            troubleshoot(project_dir, error)
             exit(1)
         finally:
             # Still gets executed, even when 'exit(1)' gets called
             run_docker(['rm', '--force', '-v', container_name])
+
+
+def troubleshoot(project_dir, error):
+    if (isinstance(error, subprocess.CalledProcessError) and 'start' in error.cmd):
+        # the bash script failed
+        print('Checking for common errors...')
+        so_files = []
+        for root, dirs, files in os.walk(project_dir):
+            for name in files:
+                _, ext = os.path.splitext(name)
+                if ext == '.so':
+                    so_files.append(os.path.join(root, name))
+
+        if so_files:
+            print(textwrap.dedent('''
+                NOTE: Shared object (.so) files found in this project.
+
+                  These files might be built against the wrong OS, causing problems with
+                  auditwheel.
+
+                  If you're using Cython and have previously done an in-place build,
+                  remove those build files (*.so and *.c) before starting cibuildwheel.
+            '''))
+
+            print('  Files detected:')
+            print('\n'.join(['    '+f for f in so_files]))
+            print('')


Test patch: