Repository: iterative/dvc

Issue:
Failing to install due to the lack of GCC, required for psutil
**Please provide information about your setup**
DVC 0.5.2, Linux (docker)


```

ERROR: Failed building wheel for psutil
509 | ERROR: intake 0.5.0 requires holoviews, which is not installed.
510 | ERROR: intake 0.5.0 requires msgpack-python, which is not installed.
511 | ERROR: dvc 0.52.1 has requirement ruamel.yaml>=0.15.91, but you'll have ruamel-yaml 0.15.46 which is incompatible.
512 | ERROR: Complete output from command /opt/conda/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-d1ijh8kc/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-hpo__0h3/install-record.txt --single-version-externally-managed --compile:
513 | ERROR: running install
514 | running build
515 | running build_py
516 | creating build
517 | creating build/lib.linux-x86_64-3.7
518 | creating build/lib.linux-x86_64-3.7/psutil
519 | copying psutil/__init__.py -> build/lib.linux-x86_64-3.7/psutil
520 | copying psutil/_common.py -> build/lib.linux-x86_64-3.7/psutil
521 | copying psutil/_compat.py -> build/lib.linux-x86_64-3.7/psutil
522 | copying psutil/_psaix.py -> build/lib.linux-x86_64-3.7/psutil
523 | copying psutil/_psbsd.py -> build/lib.linux-x86_64-3.7/psutil
524 | copying psutil/_pslinux.py -> build/lib.linux-x86_64-3.7/psutil
525 | copying psutil/_psosx.py -> build/lib.linux-x86_64-3.7/psutil
526 | copying psutil/_psposix.py -> build/lib.linux-x86_64-3.7/psutil
527 | copying psutil/_pssunos.py -> build/lib.linux-x86_64-3.7/psutil
528 | copying psutil/_pswindows.py -> build/lib.linux-x86_64-3.7/psutil
529 | creating build/lib.linux-x86_64-3.7/psutil/tests
530 | copying psutil/tests/__init__.py -> build/lib.linux-x86_64-3.7/psutil/tests
531 | copying psutil/tests/__main__.py -> build/lib.linux-x86_64-3.7/psutil/tests
532 | copying psutil/tests/runner.py -> build/lib.linux-x86_64-3.7/psutil/tests
533 | copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-3.7/psutil/tests
534 | copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-3.7/psutil/tests
535 | copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-3.7/psutil/tests
536 | copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-3.7/psutil/tests
537 | copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-3.7/psutil/tests
538 | copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-3.7/psutil/tests
539 | copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-3.7/psutil/tests
540 | copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-3.7/psutil/tests
541 | copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-3.7/psutil/tests
542 | copying psutil/tests/test_process.py -> build/lib.linux-x86_64-3.7/psutil/tests
543 | copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-3.7/psutil/tests
544 | copying psutil/tests/test_system.py -> build/lib.linux-x86_64-3.7/psutil/tests
545 | copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-3.7/psutil/tests
546 | copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-3.7/psutil/tests
547 | running build_ext
548 | building 'psutil._psutil_linux' extension
549 | creating build/temp.linux-x86_64-3.7
550 | creating build/temp.linux-x86_64-3.7/psutil
551 | gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=562 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/opt/conda/include/python3.7m -c psutil/_psutil_common.c -o build/temp.linux-x86_64-3.7/psutil/_psutil_common.o
552 | unable to execute 'gcc': No such file or directory
553 | error: command 'gcc' failed with exit status 1
554 | ----------------------------------------
555 | ERROR: Command "/opt/conda/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-d1ijh8kc/psutil/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-hpo__0h3/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-d1ijh8kc/psutil/
556 |  
557 | The command '/bin/sh -c conda env update --name root -f environment.yml && conda clean -a && rm -rf /root/.cache/pip' returned a non-zero code: 1
558 | time="2019-07-16T22:03:59Z" level=fatal msg="exit status 1"


```


Discussion:
Hi @Casyfill !

Thanks for reporting, we'll take a look ASAP.
Ok, we need to wait or help out these guys to provide manylinux wheels https://github.com/giampaolo/psutil/issues/824
@Casyfill 0.53.1 has psutil excluded from the mandatory list of dependencies, so dvc should not require gcc. That is a temporary workaround until psutil's wheels are sorted-out. Please let us know it it works for you so far.
Thanks! I surrendered and added gcc as a dependency, but looking forward to
drop it.
Thanks!

On Fri, Jul 26, 2019 at 9:24 AM Ruslan Kuprieiev <notifications@github.com>
wrote:

> @Casyfill <https://github.com/Casyfill> 0.53.1 has psutil excluded from
> the mandatory list of dependencies, so dvc should not require gcc. That is
> a temporary workaround until psutil's wheels are sorted-out. Please let us
> know it it works for you so far.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/iterative/dvc/issues/2284?email_source=notifications&email_token=AAUNHJLR6AFGXT3JB5JU73DQBL3ITA5CNFSM4IEIPWQKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD24SJHQ#issuecomment-515450014>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAUNHJPNO7KJNR3J65YDUW3QBL3ITANCNFSM4IEIPWQA>
> .
>
-- 
Philipp Kats,
data journalism, analysis and visualization

philipp-kats.squarespace.com <http://cargocollective.com/Cityfish/>


email: casyfill@gmail.com
cellphone: +7(916)438 68 04

- [ ] Please remember to update/remove the note in [static/docs/command-reference/version.md](https://github.com/iterative/dvc.org/blob/master/static/docs/command-reference/version.md)
  
  > Note that if you've installed dvc using pip...

  when this is done.
~@efiop , I think they are already providing wheels for linux :thinking:~
https://pypi.org/project/psutil/#files

UPDATE: They aren't
`psutil` yesterday became an _unconditional_ dependency of all existing DVC releases at least since 0.92.1, because `flufl.lock` released version 4.0 which is in range of the dependencies of dvc in those existing releases (constrained away at the tip of dvc master yesterday) and which unconditionally depends on `psutil`. Thus existing DVC versions now fail to install where gcc is not available, breaking our CI build and probably lots of others. :-(
@gthb We've fixed the flufl version requirements to be <4 yesterday, 1.1.2 should work fine now. Could you give 1.1.2 a try, please?
Already did :) but that fix was the next commit _after_ the 1.1.2 release. So 1.1.2 still has the problem.

A quick and sufficient workaround for existing CI builds is to add a direct requirement `flufl.lock<4.0`
@gthb Oops, indeed. Releasing 1.1.3 right now, should be out on pypi in an hour or so. Thanks for the heads up!
@gthb dvc 1.1.3 is out, please upgrade :slightly_smiling_face: 
Btw, looks like psutil is now building wheels https://github.com/giampaolo/psutil/issues/824 but doesn't publish them on pypi yet. So looks very promising, hopefully, the next release will have the wheels.
Yep, dvc 1.1.3 resolves it, so I can take that `flufl.lock` dependency workaround right back out of my CI build. :)
seems this issue can be closed
@auvipy We need to add psutil to dependencies first, but you are right, psutil wheels are finally out.

Resolving patch:
diff --git a/dvc/info.py b/dvc/info.py
--- a/dvc/info.py
+++ b/dvc/info.py
@@ -4,6 +4,8 @@
 import platform
 import uuid
 
+import psutil
+
 from dvc.exceptions import DvcException, NotDvcRepoError
 from dvc.repo import Repo
 from dvc.scm.base import SCMError
@@ -13,11 +15,6 @@
 from dvc.utils.pkg import PKG
 from dvc.version import __version__
 
-try:
-    import psutil
-except ImportError:
-    psutil = None
-
 if PKG is None:
     package = ""
 else:
@@ -44,9 +41,8 @@ def get_dvc_info():
             info.append(
                 "Cache types: {}".format(_get_linktype_support_info(repo))
             )
-            if psutil:
-                fs_type = get_fs_type(repo.cache.local.cache_dir)
-                info.append(f"Cache directory: {fs_type}")
+            fs_type = get_fs_type(repo.cache.local.cache_dir)
+            info.append(f"Cache directory: {fs_type}")
         else:
             info.append("Cache types: " + error_link("no-dvc-cache"))
 
@@ -60,9 +56,8 @@ def get_dvc_info():
         info.append("Repo: dvc, git (broken)")
     else:
         root_directory = repo.root_dir
-        if psutil:
-            fs_root = get_fs_type(os.path.abspath(root_directory))
-            info.append(f"Workspace directory: {fs_root}")
+        fs_root = get_fs_type(os.path.abspath(root_directory))
+        info.append(f"Workspace directory: {fs_root}")
         info.append("Repo: {}".format(_get_dvc_repo_info(repo)))
     return "\n".join(info)
 
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -73,6 +73,7 @@ def run(self):
     "win-unicode-console>=0.5; sys_platform == 'win32'",
     "pywin32>=225; sys_platform == 'win32'",
     "networkx>=2.1",
+    "psutil>=5.8.0",
     "pydot>=1.2.4",
     "speedcopy>=2.0.1; python_version < '3.8' and sys_platform == 'win32'",
     "dataclasses==0.7; python_version < '3.7'",


Test patch: