Repository: iterative/dvc

Issue #9512: import: inconsistently Errno 95 "no more link types left to try out"
Created: 2023-05-25T17:22:29Z
Bug Report
sorry for the imprecision, I just wanted to make a public note of this as it is messing up deployments and can not be reliably reproduced
any help or guidance is appreciated. or at least some better error message could be added
Description
When importing from a local repository sometimes dvc produces the following error:
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/repo/imp.py", line 6, in imp
    return self.imp_url(path, out=out, fname=fname, erepo=erepo, frozen=True, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/repo/__init__.py", line 65, in wrapper
    return f(repo, *args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/repo/scm_context.py", line 151, in run
    return method(repo, *args, **kw)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/repo/imp_url.py", line 93, in imp_url
    stage.run(jobs=jobs, no_download=no_download)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/funcy/decorators.py", line 47, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/stage/decorators.py", line 43, in rwlocked
    return call()
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/funcy/decorators.py", line 68, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/stage/__init__.py", line 609, in run
    self._sync_import(dry, force, kwargs.get("jobs", None), no_download)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/funcy/decorators.py", line 47, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/stage/decorators.py", line 43, in rwlocked
    return call()
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/funcy/decorators.py", line 68, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/stage/__init__.py", line 646, in _sync_import
    sync_import(self, dry, force, jobs, no_download)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/stage/imports.py", line 61, in sync_import
    stage.deps[0].download(
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/dependency/base.py", line 53, in download
    fs_download(self.fs, self.fs_path, to, jobs=jobs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/fs/__init__.py", line 56, in download
    fs.get(fs_path, to.fs_path, batch_size=jobs, callback=cb)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 660, in get
    list(executor.imap_unordered(get_file, from_infos, to_infos))
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/executors.py", line 56, in imap_unordered
    yield fut.result()
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/callbacks.py", line 69, in func
    return wrapped(path1, path2, **kw)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/callbacks.py", line 41, in wrapped
    res = fn(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 624, in get_file
    self.fs.get_file(rpath, lpath, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc/fs/dvc.py", line 388, in get_file
    return dvc_fs.get_file(dvc_path, lpath, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/base.py", line 550, in get_file
    self.fs.get_file(from_info, to_info, callback=callback, **kwargs)
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_data/fs.py", line 147, in get_file
    transfer(
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/generic.py", line 306, in transfer
    _try_links(
  File "/opt/hostedtoolcache/Python/3.10.11/x64/lib/python3.10/site-packages/dvc_objects/fs/generic.py", line 265, in _try_links
    raise OSError(errno.ENOTSUP, "no more link types left to try out") from error

Reproduce
as it is inconsistent I can't provide easy reproduction, but here is proof, as this is the same test run for the same commit hash on the same repo 5 times: https://github.com/sscu-budapest/datazimmer/actions/runs/5067371335/attempts/4
first 4 times fails at different points, 5th time succeeds
also, please note that in the repo I run dvc from the script, but I reimplemented since then the whole library to always use subprocesses and the issue remains
Expected
do the import consistently
Environment information
a brand new dvc (2.58.1) install on a github actions ubuntu-latest instance

Duplicate issue #9640: dvc pull on github actions: unexpected error - Response payload is not completed
Created: 2023-06-21T17:10:41Z
Bug Report

Running dvc pull on github is flaky and sometimes errors with unexpected error - Response payload is not completed
Description

We are using DVC to manage our zero-knowledge fixtures files from S3 storage, we are using it on our github CI and recently we see that the CI fails because the dvc pull itself failed.
Reproduce
Try to use DVC on Github CI and install it using the official action here: iterative/setup-dvc@v1


Expected
It should work from the first time, not after keep restarting the CI like 6 times until it works!

Environment information
Github Actions CI (Linux Ubuntu)

Output of dvc doctor:
$ dvc doctor
Additional Information (if any):
See Examples of the Error here:
https://github.com/webb-tools/relayer/actions/runs/5314336714/jobs/9641401063
https://github.com/webb-tools/relayer/actions/runs/5325350790/jobs/9647750332?pr=526
https://github.com/webb-tools/protocol-substrate/actions/runs/5312647626/jobs/9617443898

Closing comment (2023-06-22T01:53:56Z): First one looks similar to #9512.
Second issue is aio-libs/aiohttp#4581, which does not seem like we can do anything without getting it fixed upstream.