Question: pip install . fails, while python3 ./setup.py install succeeds. I am trying to compile and install a module that is purely local for now, written in C++. The name of the module is tcore. The module uses python C API and numpy C API. pip install complains it cannot find numpy which is imported by my setup.py, however the import is actually correct and the setup.py import is very trivial. numpy is correctly installed and can be imported: pierre@pierre-maxm3 tcore % pip install numpy Requirement already satisfied: numpy in /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages (1.26.2) pierre@pierre-maxm3 tcore % python3 Python 3.12.3 (v3.12.3:f6650f9ad7, Apr 9 2024, 08:18:47) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy_path = numpy.get_include() >>> print(numpy_path) /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/numpy/core/include  Of course python3 setup.py install works and install my module properly. But pip install fails to load numpy. It looks into /var/private/folders directory that doesn't exist... this is pip 24.3.1. Any idea? What is the problem with pip? pierre@pierre-maxm3 tcore % pip install . DEPRECATION: Loading egg at /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg is deprecated. pip 25.1 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330 Processing /Users/pierre/src/trade/t-bot/c/modules/tcore Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [33 lines of output] 3.12.3 (v3.12.3:f6650f9ad7, Apr 9 2024, 08:18:47) [Clang 13.0.0 (clang-1300.0.29.30)] system paths: /Users/pierre/src/trade/t-bot/c/modules/tcore /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process /private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/site /Library/Frameworks/Python.framework/Versions/3.12/lib/python312.zip /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12 /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload /private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/overlay/lib/python3.12/site-packages /private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/normal/lib/python3.12/site-packages /private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/overlay/lib/python3.12/site-packages/setuptools/_vendor *******will import numpy Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module> main() File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) ^^^^^^^^^^^^^^^^^^^^^ File "/private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=[]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires self.run_setup() File "/private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 522, in run_setup super().run_setup(setup_script=setup_script) File "/private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip-build-env-1m0z9fie/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 320, in run_setup exec(code, locals()) File "<string>", line 13, in <module> ModuleNotFoundError: No module named 'numpy' [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. pierre@pierre-maxm3 tcore % ls -la /private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip* zsh: no matches found: /private/var/folders/9g/wy6m2s4d705fpzyr78yx0s9w0000gn/T/pip* pierre@pierre-maxm3 tcore %  pip setup.py install : pierre@pierre-maxm3 tcore % python3 setup.py install ['/Users/pierre/src/trade/t-bot/c/modules/tcore', '/Library/Frameworks/Python.framework/Versions/3.12/lib/python312.zip', '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12', '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload', '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages', '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg'] 3.12.3 (v3.12.3:f6650f9ad7, Apr 9 2024, 08:18:47) [Clang 13.0.0 (clang-1300.0.29.30)] /Users/pierre/src/trade/t-bot/c/modules/tcore /Library/Frameworks/Python.framework/Versions/3.12/lib/python312.zip /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12 /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg *******will import numpy done numpy root_path /Users/pierre/src/trade/t-bot/c/modules/tcore src_path /Users/pierre/src/trade/t-bot/c/modules/tcore main() root_path /Users/pierre/src/trade/t-bot/c/modules/tcore running install /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` directly. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. ******************************************************************************** !! self.initialize_options() /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !! ******************************************************************************** Please avoid running ``setup.py`` and ``easy_install``. Instead, use pypa/build, pypa/installer or other standards-based tools. See https://github.com/pypa/setuptools/issues/917 for details. ******************************************************************************** !! self.initialize_options() running bdist_egg running egg_info writing tcore.egg-info/PKG-INFO writing dependency_links to tcore.egg-info/dependency_links.txt writing top-level names to tcore.egg-info/top_level.txt dependency /Users/pierre/src/trade/t-bot/c/src/ticker_database/src/TickerDatabase.h won't be automatically included in the manifest: the path must be relative dependency /Users/pierre/src/trade/t-bot/c/src/ticker_database/src/Tickert.h won't be automatically included in the manifest: the path must be relative dependency /Users/pierre/src/trade/t-bot/c/src/logger/src/TmonLogger.h won't be automatically included in the manifest: the path must be relative reading manifest file 'tcore.egg-info/SOURCES.txt' writing manifest file 'tcore.egg-info/SOURCES.txt' installing library code to build/bdist.macosx-10.9-universal2/egg running install_lib running build_ext building 'tcore' extension clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c /Users/pierre/src/trade/t-bot/c/src/logger/src/TmonLogger.cpp -o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/logger/src/TmonLogger.o -O3 -std=c++23 -I/Users/pierre/src/trade/t-bot/c/src/ticker_database/src -I/Users/pierre/src/trade/t-bot/c/src/logger/src -I/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/numpy/core/include clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c /Users/pierre/src/trade/t-bot/c/src/ticker_database/src/Ticker.cpp -o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/ticker_database/src/Ticker.o -O3 -std=c++23 -I/Users/pierre/src/trade/t-bot/c/src/ticker_database/src -I/Users/pierre/src/trade/t-bot/c/src/logger/src -I/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/numpy/core/include clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c /Users/pierre/src/trade/t-bot/c/src/ticker_database/src/TickerDatabase.cpp -o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/ticker_database/src/TickerDatabase.o -O3 -std=c++23 -I/Users/pierre/src/trade/t-bot/c/src/ticker_database/src -I/Users/pierre/src/trade/t-bot/c/src/logger/src -I/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/numpy/core/include clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c /Users/pierre/src/trade/t-bot/c/src/ticker_database/src/day_offsets.cpp -o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/ticker_database/src/day_offsets.o -O3 -std=c++23 -I/Users/pierre/src/trade/t-bot/c/src/ticker_database/src -I/Users/pierre/src/trade/t-bot/c/src/logger/src -I/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/numpy/core/include clang -fno-strict-overflow -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -O3 -Wall -arch arm64 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.12/include/python3.12 -c tcore.cpp -o build/temp.macosx-10.9-universal2-cpython-312/tcore.o -O3 -std=c++23 -I/Users/pierre/src/trade/t-bot/c/src/ticker_database/src -I/Users/pierre/src/trade/t-bot/c/src/logger/src -I/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/numpy/core/include clang++ -bundle -undefined dynamic_lookup -arch arm64 -arch x86_64 -g build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/logger/src/TmonLogger.o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/ticker_database/src/Ticker.o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/ticker_database/src/TickerDatabase.o build/temp.macosx-10.9-universal2-cpython-312/Users/pierre/src/trade/t-bot/c/src/ticker_database/src/day_offsets.o build/temp.macosx-10.9-universal2-cpython-312/tcore.o -o build/lib.macosx-10.9-universal2-cpython-312/tcore.cpython-312-darwin.so -std=c++23 creating build/bdist.macosx-10.9-universal2/egg copying build/lib.macosx-10.9-universal2-cpython-312/tcore.cpython-312-darwin.so -> build/bdist.macosx-10.9-universal2/egg creating stub loader for tcore.cpython-312-darwin.so byte-compiling build/bdist.macosx-10.9-universal2/egg/tcore.py to tcore.cpython-312.pyc creating build/bdist.macosx-10.9-universal2/egg/EGG-INFO copying tcore.egg-info/PKG-INFO -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO copying tcore.egg-info/SOURCES.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO copying tcore.egg-info/dependency_links.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO copying tcore.egg-info/top_level.txt -> build/bdist.macosx-10.9-universal2/egg/EGG-INFO writing build/bdist.macosx-10.9-universal2/egg/EGG-INFO/native_libs.txt zip_safe flag not set; analyzing archive contents... __pycache__.tcore.cpython-312: module references __file__ creating 'dist/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg' and adding 'build/bdist.macosx-10.9-universal2/egg' to it removing 'build/bdist.macosx-10.9-universal2/egg' (and everything under it) Processing tcore-1.0.0-py3.12-macosx-10.9-universal2.egg removing '/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg' (and everything under it) creating /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg Extracting tcore-1.0.0-py3.12-macosx-10.9-universal2.egg to /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages Adding tcore 1.0.0 to easy-install.pth file Installed /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/tcore-1.0.0-py3.12-macosx-10.9-universal2.egg Processing dependencies for tcore==1.0.0 Finished processing dependencies for tcore==1.0.0 pierre@pierre-maxm3 tcore %  Here is my setup.py: from setuptools import setup, Extension import os import sys import site print(sys.version) print("system paths:") for p in sys.path: print(p) print("\n*******will import numpy") import numpy print("done numpy") root_path = os.path.abspath(".") print(f"root_path {root_path}") p = root_path # look for top path of the .git repo while ".git" not in os.listdir(p): p = os.path.dirname(p) src_path = os.path.join(p, "c", "src") print(f"src_path {root_path}") src_files = ["tcore.cpp",\ os.path.join(src_path, "ticker_database", "src", "TickerDatabase.cpp"),\ os.path.join(src_path, "ticker_database", "src", "day_offsets.cpp"),\ os.path.join(src_path, "ticker_database", "src", "Ticker.cpp"),\ os.path.join(src_path, "logger", "src", "TmonLogger.cpp"),\ ] h_files = [os.path.join(src_path, "ticker_database", "src", "TickerDatabase.h"),\ os.path.join(src_path, "ticker_database", "src", "Tickert.h"),\ os.path.join(src_path, "logger", "src", "TmonLogger.h"),\ ] sp = site.getsitepackages()[0] numpy_path = numpy.get_include() include_path = [os.path.join(src_path, "ticker_database", "src"),\ os.path.join(src_path, "logger", "src"), numpy_path] include_options = [f"-I{p}" for p in include_path] compile_options = ["-O3", "-std=c++23"] compile_options += include_options def main(): print(f"main() root_path {root_path}") src_path = "../" setup(name="tcore", version="1.0.0", description="Python interface for tcore C++", author="Pierre Vandwalle", include_package_data=True, ext_modules=[Extension("tcore", src_files, extra_compile_args=compile_options, extra_link_args=["-std=c++23"], depends=h_files, )], setup_requires=['numpy'] ) if __name__ == "__main__": main()  Added pyproject.toml: [project] name = "tcore" dependencies = [ "numpy==1.26.2", ] 

Answer: Modern pip uses build isolation, it uses a transient virtual env to build a wheel and then installs the wheel into the target environment; this transient virtual env is your absent temporary directory; pip removes it after success or failure so you cannot find it. There are two ways to work around the problem:  Install numpy into the current environment and install the source code without build isolation:  pip install numpy pip install --no-build-isolation .   Create pyproject.toml and declare numpy as a build-time dependency:  [build-system] build-backend = "setuptools.build_meta" requires = ["setuptools", "numpy==1.26.2"]