Repository: yt-project/yt

Issue:
Fresh yt installation not importing
<!--To help us understand and resolve your issue, please fill out the form to
the best of your ability.-->
<!--You can feel free to delete the sections that do not apply.-->

### Bug report

**Bug summary**

yt fails to import on a fresh development installation. The error is that `ModuleNotFoundError: No module named 'yt.utilities.lib.misc_utilities'` Similar to [Issue 2685](https://github.com/yt-project/yt/issues/2685)
<!--A short 1-2 sentences that succinctly describes the bug-->

**Code for reproduction**
Installed using `pip install git+git://github.com/yt-project/yt.git`
<!--A minimum code snippet required to reproduce the bug, also minimizing the
number of dependencies required.-->

<!-- If you need to use a data file to trigger the issue you're having, consider
using one of the datasets from the yt data hub (http://yt-project.org/data). If
your issue cannot be triggered using a public dataset, you can use the yt
curldrop (https://docs.hub.yt/services.html#curldrop) to share data
files. Please include a link to the dataset in the issue if you use the
curldrop.-->

```python
import yt
```

**Actual outcome**

<!--The output produced by the above code, which may be a screenshot, console
output, etc.-->

```
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-2d2292a375dc> in <module>
----> 1 import yt

~/.conda/envs/ytgit/lib/python3.8/site-packages/yt/__init__.py in <module>
     63 )
     64 
---> 65 from yt.fields.api import (
     66     field_plugins,
     67     DerivedField,

~/.conda/envs/ytgit/lib/python3.8/site-packages/yt/fields/api.py in <module>
      1 # from . import species_fields
----> 2 from . import (
      3     angular_momentum,
      4     astro_fields,
      5     cosmology_fields,

~/.conda/envs/ytgit/lib/python3.8/site-packages/yt/fields/angular_momentum.py in <module>
      1 import numpy as np
      2 
----> 3 from yt.utilities.lib.misc_utilities import (
      4     obtain_position_vector,
      5     obtain_relative_velocity_vector,

ModuleNotFoundError: No module named 'yt.utilities.lib.misc_utilities'
```

**Expected outcome**

<!--A description of the expected outcome from the code snippet-->
<!--If this used to work in an earlier version of yt, please note the
version it used to work on-->

**Version Information**
<!--Please specify your platform and versions of the relevant libraries you are
using:-->
  * Operating System: CentOS Linux 7 (Core)x86_64
  * Python Version: 3.8.5
  * yt version: 4.0.dev0
  * Other Libraries (if applicable): N/A

<!--Please tell us how you installed yt and python e.g., from source,
pip, conda. If you installed from conda, please specify which channel you used
if not the default-->

Installed using `pip install git+git://github.com/yt-project/yt.git`


Thanks!


Discussion:
Hi!  One reason this often happens is if it isn't built.  But, pip
installing it like this should build.  Can you provide the full log output
from the pip install?

On Fri, Sep 4, 2020 at 10:32 AM Raziq Noorali <notifications@github.com>
wrote:

> Bug report
>
> *Bug summary*
>
> yt fails to import on a fresh development installation. The error is that ModuleNotFoundError:
> No module named 'yt.utilities.lib.misc_utilities'
>
> *Code for reproduction*
> Installed using pip install git+git://github.com/yt-project/yt.git
>
> # Paste your code here##import yt
>
> *Actual outcome*
>
> # If applicable, paste the console output here
> #
> #
>
> *Expected outcome*
>
> *Version Information*
>
>    - Operating System: CentOS Linux 7 (Core)x86_64
>    - Python Version: 3.8.5
>    - yt version: 4.0.dev0
>    - Other Libraries (if applicable): N/A
>
> Installed using pip install git+git://github.com/yt-project/yt.git
>
> Thanks!
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/yt-project/yt/issues/2892>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAAVXO7JCM3FCUAY7FJ3CGTSEECCRANCNFSM4QYVEXEQ>
> .
>

Hi @matthewturk. I apologize for the delay. In solidarity with the [graduate student strike at the University of Michigan](https://www.geo3550.org/2020/09/08/geo-is-on-strike/), I have not crossed imposed picket lines and am getting to this now. Please let me know if you need anything else!

log from pip install: 
```
$ pip install git+git://github.com/yt-project/yt.git
Collecting git+git://github.com/yt-project/yt.git
  Cloning git://github.com/yt-project/yt.git to /tmp/pip-req-build-m9_kw_3f
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting IPython>=1.0
  Downloading ipython-7.18.1-py3-none-any.whl (786 kB)
     |████████████████████████████████| 786 kB 5.6 MB/s 
Collecting unyt>=2.7.2
  Using cached unyt-2.7.2-py2.py3-none-any.whl (97 kB)
Collecting numpy>=1.10.4
  Downloading numpy-1.19.2-cp38-cp38-manylinux2010_x86_64.whl (14.5 MB)
     |████████████████████████████████| 14.5 MB 53.4 MB/s 
Requirement already satisfied: setuptools>=19.6 in ./.conda/envs/ytgit_test/lib/python3.8/site-packages (from yt==4.0.dev0) (49.6.0.post20200814)
Collecting matplotlib>=1.5.3
  Downloading matplotlib-3.3.2-cp38-cp38-manylinux1_x86_64.whl (11.6 MB)
     |████████████████████████████████| 11.6 MB 85.2 MB/s 
Collecting sympy>=1.2
  Using cached sympy-1.6.2-py3-none-any.whl (5.8 MB)
Collecting decorator
  Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting pexpect>4.3; sys_platform != "win32"
  Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Collecting pygments
  Downloading Pygments-2.7.1-py3-none-any.whl (944 kB)
     |████████████████████████████████| 944 kB 83.9 MB/s 
Collecting traitlets>=4.2
  Downloading traitlets-5.0.4-py3-none-any.whl (98 kB)
     |████████████████████████████████| 98 kB 5.3 MB/s 
Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0
  Downloading prompt_toolkit-3.0.7-py3-none-any.whl (355 kB)
     |████████████████████████████████| 355 kB 87.6 MB/s 
Collecting pickleshare
  Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Collecting jedi>=0.10
  Using cached jedi-0.17.2-py2.py3-none-any.whl (1.4 MB)
Collecting backcall
  Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)
Collecting python-dateutil>=2.1
  Using cached python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting cycler>=0.10
  Using cached cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting pillow>=6.2.0
  Using cached Pillow-7.2.0-cp38-cp38-manylinux1_x86_64.whl (2.2 MB)
Collecting kiwisolver>=1.0.1
  Using cached kiwisolver-1.2.0-cp38-cp38-manylinux1_x86_64.whl (92 kB)
Requirement already satisfied: certifi>=2020.06.20 in ./.conda/envs/ytgit_test/lib/python3.8/site-packages (from matplotlib>=1.5.3->yt==4.0.dev0) (2020.6.20)
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Processing ./.cache/pip/wheels/29/2c/1c/d2e4580cde2743b0aef389e936ac21a2db92921ddbca53faa1/mpmath-1.1.0-py3-none-any.whl
Collecting ptyprocess>=0.5
  Using cached ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting ipython-genutils
  Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Collecting wcwidth
  Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Collecting parso<0.8.0,>=0.7.0
  Using cached parso-0.7.1-py2.py3-none-any.whl (109 kB)
Collecting six>=1.5
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Building wheels for collected packages: yt
  Building wheel for yt (PEP 517) ... done
  Created wheel for yt: filename=yt-4.0.dev0-py3-none-any.whl size=1857377 sha256=648b2cef24207a5d158b0c61c76a0f8723c97e6a235036c578870ae2e4197267
  Stored in directory: /tmp/pip-ephem-wheel-cache-p6pus6eo/wheels/de/5c/c8/8c939833284c1c193ca0f3dda036b9ea66c6e4089174dace55
Successfully built yt
Installing collected packages: decorator, ptyprocess, pexpect, pygments, ipython-genutils, traitlets, wcwidth, prompt-toolkit, pickleshare, parso, jedi, backcall, IPython, numpy, mpmath, sympy, unyt, six, python-dateutil, cycler, pillow, kiwisolver, pyparsing, matplotlib, yt
Successfully installed IPython-7.18.1 backcall-0.2.0 cycler-0.10.0 decorator-4.4.2 ipython-genutils-0.2.0 jedi-0.17.2 kiwisolver-1.2.0 matplotlib-3.3.2 mpmath-1.1.0 numpy-1.19.2 parso-0.7.1 pexpect-4.8.0 pickleshare-0.7.5 pillow-7.2.0 prompt-toolkit-3.0.7 ptyprocess-0.6.0 pygments-2.7.1 pyparsing-2.4.7 python-dateutil-2.8.1 six-1.15.0 sympy-1.6.2 traitlets-5.0.4 unyt-2.7.2 wcwidth-0.2.5 yt-4.0.dev0
``` 


What is the output of the following two commands?
```
which pip
which python
```
Also, it may prove useful to provide the log using `pip install --verbose git+git://github.com/yt-project/yt.git`.
Sure!
```
(ytgit_test) [rnoorali@gl-login2 ~]$ which pip
~/.conda/envs/ytgit_test/bin/pip
(ytgit_test) [rnoorali@gl-login2 ~]$ which python
~/.conda/envs/ytgit_test/bin/python
```

The verbose log is quite long, soft the sake of readability I attached it here
[pip_log.txt](https://github.com/yt-project/yt/files/5263879/pip_log.txt)

> Sure!
> 
> ```
> (ytgit_test) [rnoorali@gl-login2 ~]$ which pip
> ~/.conda/envs/ytgit_test/bin/pip
> (ytgit_test) [rnoorali@gl-login2 ~]$ which python
> ~/.conda/envs/ytgit_test/bin/python
> ```
> 
> The verbose log is quite long, soft the sake of readability I attached it here
> [pip_log.txt](https://github.com/yt-project/yt/files/5263879/pip_log.txt)

Alright, this looks definitely like a bug on yt's side: yt relies on some files that need to be compiled (for higher efficiency). However, for some reason they do not get compiled when one installs using the command you tried (I can reproduce this on my side).

In the meantime, you should be able to install with the following commands:
```bash
$ cd /path/to/where/yt/will/be
$ git clone https://github.com/yt-project/yt.git
$ cd yt
$ pip install -e .
```
So after some debugging, it turns out that yt cannot be installed unless `-e` is specified in the pip command, _i.e._
```bash
pip install -e git+https://github.com/yt-project/yt.git#egg=yt   # works
pip install git+https://github.com/yt-project/yt.git#egg=yt      # does not compile the cython files
```
This stems from the fact that `pip install whatever` without `-e` does not call our custom version of `build_ext.build_extensions`.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -5,7 +5,7 @@
 from distutils.version import LooseVersion
 
 import pkg_resources
-from setuptools import find_packages, setup
+from setuptools import Distribution, find_packages, setup
 
 from setupext import (
     check_for_openmp,
@@ -84,6 +84,16 @@
 # This overrides using lib_exts, so it has to happen after lib_exts is fully defined
 build_ext, sdist = create_build_ext(lib_exts, cythonize_aliases)
 
+# Force setuptools to consider that there are ext modules, even if empty.
+# See https://github.com/yt-project/yt/issues/2922 and
+# https://stackoverflow.com/a/62668026/2601223 for the fix.
+class BinaryDistribution(Distribution):
+    """Distribution which always forces a binary package with platform name."""
+
+    def has_ext_modules(self):
+        return True
+
+
 if __name__ == "__main__":
     setup(
         name="yt",
@@ -141,6 +151,7 @@
         license="BSD 3-Clause",
         zip_safe=False,
         scripts=["scripts/iyt"],
+        distclass=BinaryDistribution,
         ext_modules=[],  # !!! We override this inside build_ext above
         python_requires=">=3.6",
     )


Test patch: