Repository: datalad/datalad

Issue:
test_sibling_enable_sameas started to FAIL on appveyor (Ubu) recently
spotted while reviewing https://github.com/datalad/datalad/pull/7318 .

looked like
```
FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
[{'action': 'get',
  'annexkey': 'SHA256E-s1--5feceb66ffc86f38d952786c6d696c79c2dbc239dd4e91b46729d73a27fb57e9',
  'error_message': 'rsync failed\nrsync failed\nrsync failed',
  'path': '/home/appveyor/DLTMP/datalad_temp_test_sibling_enable_sameast7tzb2z1/f0',
  'refds': '/home/appveyor/DLTMP/datalad_temp_test_sibling_enable_sameast7tzb2z1',
  'status': 'error',
  'type': 'file'}]

```

and apparently not for the first time but started recently:

```
$> git grep 'FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas'
08/pr/7317/d3a47bc/appveyor-9170-failed/1o7ovh42loqs1udv.txt:[00:25:10] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
08/pr/7317/d3a47bc/appveyor-9170-failed/1o7ovh42loqs1udv.txt:[00:27:06] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
08/pr/7317/d3a47bc/appveyor-9170-failed/ulnb0prc4odiwjcl.txt:[00:23:38] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
08/pr/7317/d3a47bc/appveyor-9170-failed/ulnb0prc4odiwjcl.txt:[00:25:24] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
08/push/maint/901c8f4/appveyor-9167-failed/ni3la8616gomqukn.txt:[00:24:30] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
08/push/maint/901c8f4/appveyor-9167-failed/ni3la8616gomqukn.txt:[00:26:13] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
08/push/maint/901c8f4/appveyor-9167-failed/td81tnkiw3rfc5f6.txt:[00:22:52] [gw0] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
08/push/maint/901c8f4/appveyor-9167-failed/td81tnkiw3rfc5f6.txt:[00:24:51] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
08/push/maint/c1b079f/appveyor-9166-failed/hxrjorsunge47o1d.txt:[00:23:21] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
08/push/maint/c1b079f/appveyor-9166-failed/hxrjorsunge47o1d.txt:[00:25:13] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
08/push/maint/c1b079f/appveyor-9166-failed/pe7etil04hh9e2ux.txt:[00:25:07] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
08/push/maint/c1b079f/appveyor-9166-failed/pe7etil04hh9e2ux.txt:[00:27:07] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7317/03da886/appveyor-9178-failed/9v3v9awdts4ybclk.txt:[00:25:55] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7317/03da886/appveyor-9178-failed/9v3v9awdts4ybclk.txt:[00:27:48] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7317/03da886/appveyor-9178-failed/n5emh379373mrc5m.txt:[00:23:53] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7317/03da886/appveyor-9178-failed/n5emh379373mrc5m.txt:[00:25:38] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7317/43cb9f0/appveyor-9176-failed/9y2trs40gvv5edpw.txt:[00:23:15] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7317/43cb9f0/appveyor-9176-failed/9y2trs40gvv5edpw.txt:[00:24:58] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7317/43cb9f0/appveyor-9176-failed/ob9l82pjtbmfk6w2.txt:[00:25:40] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7317/43cb9f0/appveyor-9176-failed/ob9l82pjtbmfk6w2.txt:[00:27:43] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7317/ff72c14/appveyor-9173-failed/cc39wnjusvhy0og7.txt:[00:25:24] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7317/ff72c14/appveyor-9173-failed/cc39wnjusvhy0og7.txt:[00:27:17] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7317/ff72c14/appveyor-9173-failed/difgngng03j2vqsq.txt:[00:23:41] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7317/ff72c14/appveyor-9173-failed/difgngng03j2vqsq.txt:[00:25:27] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7318/e0acb7e/appveyor-9175-failed/1h871v79409t03a1.txt:[00:25:08] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7318/e0acb7e/appveyor-9175-failed/1h871v79409t03a1.txt:[00:27:08] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/pr/7318/e0acb7e/appveyor-9175-failed/kskka21kr42q3kpn.txt:[00:22:47] [gw0] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/pr/7318/e0acb7e/appveyor-9175-failed/kskka21kr42q3kpn.txt:[00:24:50] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/push/maint/6c42103/appveyor-9171-failed/awlarnbxu4865miy.txt:[00:25:59] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/push/maint/6c42103/appveyor-9171-failed/awlarnbxu4865miy.txt:[00:27:53] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
09/push/maint/6c42103/appveyor-9171-failed/r7wd7ljfhc6cabqw.txt:[00:23:37] [gw1] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas 
09/push/maint/6c42103/appveyor-9171-failed/r7wd7ljfhc6cabqw.txt:[00:25:27] FAILED ../datalad/distribution/tests/test_siblings.py::test_sibling_enable_sameas - datalad.support.exceptions.IncompleteResultsError: Command did not complete successfully. 1 failed:
```


Discussion:
codecov[bot]: ## [Codecov](https://codecov.io/gh/datalad/datalad/pull/7342?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad) Report
Patch coverage has no change and project coverage change: **`+1.80`** :tada:
> Comparison is base [(`f392b4b`)](https://codecov.io/gh/datalad/datalad/commit/f392b4b39544321016594ef33fcc97a08aea3bcd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad) 88.13% compared to head [(`3701d39`)](https://codecov.io/gh/datalad/datalad/pull/7342?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad) 89.94%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##            maint    #7342      +/-   ##
==========================================
+ Coverage   88.13%   89.94%   +1.80%     
==========================================
  Files         327      327              
  Lines       44556    44538      -18     
  Branches        0     5907    +5907     
==========================================
+ Hits        39270    40059     +789     
+ Misses       5286     4464     -822     
- Partials        0       15      +15     
```


[see 94 files with indirect coverage changes](https://codecov.io/gh/datalad/datalad/pull/7342/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad)

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad)

</details>

[:umbrella: View full report in Codecov by Sentry](https://codecov.io/gh/datalad/datalad/pull/7342?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=datalad).


Patch:
diff --git a/changelog.d/pr-7329.md b/changelog.d/pr-7329.md
deleted file mode 100644
index 37650c437..000000000
--- a/changelog.d/pr-7329.md
+++ /dev/null
@@ -1,6 +0,0 @@
-### 🐛 Bug Fixes
-
-- Fixed a bug where changing DataLad's log level could lead to failing git-annex calls.
-  Fixes [#7328](https://github.com/datalad/datalad/issues/7328) via
-  [PR #7329](https://github.com/datalad/datalad/pull/7329)
-  (by [@bpoldrack](https://github.com/bpoldrack))
diff --git a/changelog.d/pr-7342.md b/changelog.d/pr-7342.md
new file mode 100644
index 000000000..314841401
--- /dev/null
+++ b/changelog.d/pr-7342.md
@@ -0,0 +1,3 @@
+### 🧪 Tests
+
+- Skip with_sameas_remote when rsync and annex are incompatible.  Fixes [#7320](https://github.com/datalad/datalad/issues/7320) via [PR #7342](https://github.com/datalad/datalad/pull/7342) (by [@bpoldrack](https://github.com/bpoldrack))
diff --git a/datalad/support/annexrepo.py b/datalad/support/annexrepo.py
index b746bd0e1..a92903945 100644
--- a/datalad/support/annexrepo.py
+++ b/datalad/support/annexrepo.py
@@ -1166,7 +1166,7 @@ class AnnexRepo(GitRepo, RepoInterface):
         list(dict)
           List of parsed result records.
         """
-        protocol_class = GeneratorAnnexJsonProtocol
+        protocol_class = GeneratorAnnexJsonNoStderrProtocol
 
         args = args[:] + ['--json', '--json-error-messages']
         if progress:
diff --git a/datalad/support/external_versions.py b/datalad/support/external_versions.py
index 3771ff44e..36c1797b1 100644
--- a/datalad/support/external_versions.py
+++ b/datalad/support/external_versions.py
@@ -150,6 +150,33 @@ def _get_system_7z_version():
     lgr.debug("Could not determine version of 7z from stdout. %s", out)
 
 
+def get_rsync_version():
+
+    # This does intentionally not query the version of rsync itself, but
+    # that of the debian package it's installed with. Reason is in gh-7320,
+    # which results in the need to detect a patched-by-ubuntu version of rsync
+    # and therefore the package version, not the result of `rsync --version`.
+    from datalad.utils import on_linux, get_linux_distribution
+    if on_linux:
+        dist = get_linux_distribution()[0]
+        if dist in ['debian', 'ubuntu']:
+            out = _runner.run(['apt-cache', 'policy', 'rsync'],
+                              protocol=StdOutErrCapture)
+            for line in out['stdout'].splitlines():
+                parts = line.split()
+                if parts[0] == 'Installed:':
+                    ver = LooseVersion(parts[1])
+                    break
+        # If we have a debian package version, use this as rsync version.
+        # Otherwise report what `rsync --version` itself has to say.
+        if ver:
+            return ver
+    out = _runner.run(['rsync', '--version'], protocol=StdOutErrCapture)
+    # Expected first line:
+    # rsync version x protocol version y
+    return LooseVersion(out['stdout'].splitlines()[0].split()[2])
+
+
 class ExternalVersions(object):
     """Helper to figure out/use versions of the externals (modules, cmdline tools, etc).
 


Test patch:
diff --git a/datalad/distribution/tests/test_siblings.py b/datalad/distribution/tests/test_siblings.py
index c6fab81a6..fec54b84f 100644
--- a/datalad/distribution/tests/test_siblings.py
+++ b/datalad/distribution/tests/test_siblings.py
@@ -397,7 +397,6 @@ def test_arg_missing(path=None, path2=None):
         annex_groupwanted='whatever')
 
 
-@pytest.mark.xfail(on_appveyor, reason="https://github.com/datalad/datalad/issues/7320")
 @with_sameas_remote
 @with_tempfile(mkdir=True)
 def test_sibling_enable_sameas(repo=None, clone_path=None):
diff --git a/datalad/tests/test_tests_utils_pytest.py b/datalad/tests/test_tests_utils_pytest.py
index af234f7b8..a357b7bf8 100644
--- a/datalad/tests/test_tests_utils_pytest.py
+++ b/datalad/tests/test_tests_utils_pytest.py
@@ -665,27 +665,3 @@ def test_ok_file_under_git_symlinks(path=None):
         ok_file_under_git(op.join(lpath, 'notingit'))
     with assert_raises(AssertionError):
         ok_file_under_git(op.join(lpath, 'nonexisting'))
-
-
-def test_assert_raises():
-    # rudimentary test of assert_raises shim prompted by suspicion in
-    # https://github.com/datalad/datalad/issues/6846#issuecomment-1363878497
-    def raise_ValueError():
-        raise ValueError("exc ValueError")
-
-    def raise_TypeError():
-        raise TypeError("exc TypeError")
-
-    # Test both forms of use
-    with assert_raises(ValueError):
-        raise_ValueError()
-    assert_raises(ValueError, raise_ValueError)
-
-    # can we specify multiple in a tuple?
-    with assert_raises((ValueError, TypeError)):
-        raise_ValueError()
-    with assert_raises((ValueError, TypeError)):
-        raise_TypeError()
-
-    assert_raises((ValueError, TypeError), raise_TypeError)
-    assert_raises((ValueError, TypeError), raise_ValueError)
\ No newline at end of file
diff --git a/datalad/tests/utils_pytest.py b/datalad/tests/utils_pytest.py
index 4f2af0f1f..681786f7b 100644
--- a/datalad/tests/utils_pytest.py
+++ b/datalad/tests/utils_pytest.py
@@ -36,7 +36,10 @@ from datalad import utils
 from datalad.consts import ARCHIVES_TEMP_DIR
 from datalad.dochelpers import borrowkwargs
 
-from datalad.support.external_versions import external_versions
+from datalad.support.external_versions import (
+    external_versions,
+    get_rsync_version,
+)
 from datalad.support.keyring_ import MemoryKeyring
 from datalad.support.network import RI
 from datalad.support.vcr_ import *
@@ -1124,6 +1127,16 @@ def with_sameas_remote(func, autoenabled=False):
            '8.20200522' < external_versions['cmd:annex'] < '8.20200720':
             pytest.skip("Test known to hang")
 
+        # A fix in rsync 3.2.4 broke compatibility with older annex versions.
+        # To make things a bit more complicated, ubuntu pulled that fix into
+        # their rsync package for 3.1.3-8.
+        # Issue: gh-7320
+        rsync_ver = get_rsync_version()
+        rsync_fixed = rsync_ver >= "3.1.3-8ubuntu" or rsync_ver >= "3.2.4"
+        if rsync_fixed and external_versions['cmd:annex'] < "10.20220504":
+            pytest.skip(f"rsync {rsync_ver} and git-annex "
+                        f"{external_versions['cmd:annex']} incompatible")
+
         sr_path, repo_path = args[-2:]
         fn_args = args[:-2]
         repo = AnnexRepo(repo_path)