Repository: borgbackup/borg

Issue:
borg tests broken on python 3.7
https://travis-ci.org/borgbackup/borg/jobs/360736109

note: 3.7.0a4+ might be rather old, like Jan 2018, currently python 3.7 is at b4 (soon rc1).

https://docs.python.org/3.7/whatsnew/3.7.html


Discussion:
https://github.com/travis-ci/travis-ci/issues/9580
i did a local test with the stretch64 machine in master branch and a pyenv-built 3.7.0rc1 and still see some of these errors. either something in py37 is broken or borg and py37 are somehow not compatible.
Arch Linux is currently rebuilding the world for the final python 3.7 release. Here's a log of the testsuite: https://paste.xinu.at/wEIaO/

Errors excerpt:

```
=================================== FAILURES ===================================
_____________________ ArchiverTestCase.test_common_options _____________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_common_options>

    def test_common_options(self):
        self.create_test_files()
        self.cmd('init', '--encryption=repokey', self.repository_location)
        log = self.cmd('--debug', 'create', self.repository_location + '::test', 'input')
>       assert 'security: read previous location' in log
E       AssertionError: assert 'security: read previous location' in 'using builtin fallback logging configuration\n'

borg/testsuite/archiver.py:2025: AssertionError
__________________ ArchiverTestCase.test_corrupted_repository __________________

self = <borg.testsuite.archiver.ArchiverTestCase testMethod=test_corrupted_repository>

    def test_corrupted_repository(self):
        self.cmd('init', '--encryption=repokey', self.repository_location)
        self.create_src_archive('test')
        self.cmd('extract', '--dry-run', self.repository_location + '::test')
        output = self.cmd('check', '--show-version', self.repository_location)
        self.assert_in('borgbackup version', output)  # implied output even without --info given
        self.assert_not_in('Starting repository check', output)  # --info not given for root logger
    
        name = sorted(os.listdir(os.path.join(self.tmpdir, 'repository', 'data', '0')), reverse=True)[1]
        with open(os.path.join(self.tmpdir, 'repository', 'data', '0', name), 'r+b') as fd:
            fd.seek(100)
            fd.write(b'XXXX')
        output = self.cmd('check', '--info', self.repository_location, exit_code=1)
>       self.assert_in('Starting repository check', output)  # --info given for root logger
E       AssertionError: 'Starting repository check' not found in 'Data integrity error: Segment entry checksum mismatch [segment 4, offset 8]\nIndex object count mismatch.\ncommitted index: 126 objects\nrebuilt index:   0 objects\nID                                                               rebuilt index    committed index \n0000000000000000000000000000000000000000000000000000000000000000 <not found>      (4, 3717805)    \n345046f023114df709fefdf9ed3e8171859de0afe762597ce3c4fb4dbf01860d <not found>      (4, 1003843)    \n13b660a93fc3d3332d17776fce7257aca9cadf7a6b0cac39f08ce04be5c6f5c6 <not found>      (4, 8)          \n14c8b73bba2c42cddc7f9cdcebfb98e989078e66714a3ee7a13d34933e794f21 <not found>      (4, 1522395)    \nefab4a77155c2f3f89933c441f6594d39c46ed4ca258643fe69377310f6328af <not found>      (4, 3144932)    \n28b1a3e69d22e7d12110e445982bf7df3d8a5777f61c57fd8346bd94eb46f989 <not found>      (4, 1057530)    \nf6d8da58ed51ffcc77f74a9fa12fb7687d8eb1752f0145e275e84e411ff0b03f <not found>      (4, 3400689)    \n7aa73fe24b3145478a80bb3c17b3772008e7a4251bd4f6c5c8335fee1a5bc103 <not found>      (4, 1080821)    \n7dbd4eb812f75d7200fc5958a8d59a046cbee0971fdeaf04095f22b2b9f83116 <not found>      (4, 1316624)    \na38fb0861f3ed036a7e9f7ccc77a10da6521a792c46576d1fe787cab651a053a <not found>      (4, 464428)     \ne78d1b98d83fcddb0b10cdeda5b10b12baa72aacc12e0599380ee51c1951462d <not found>      (4, 997343)     \n1c5e783b62e099c8454a488d31990641dc3431a9cf2f0954e2d6f633a446472b <not found>      (4, 1146808)    \n49ce3de4acf057025c904a0f2132d482b1d2807e3170411bc37ba0252789c6a9 <not found>      (4, 3431277)    \n1f051a0c6b9c7b117137ab4af0ed1ef165f39b176caaa14ecc00fb0e575e1bcb <not found>      (4, 3536091)    \n0a811b4c43439b8e13c9d7839f5a4ca6ca63465bb5fdf7b06e43da81124a2f93 <not found>      (4, 2445536)    \n3a71a8a7330b8999b6eb24bc0ca7c467a08ee487c9b083fc0b9f3b46eb8f719c <not found>      (4, 1498651)    \n811120405b238eb7746fca504f0355a18f2c6ac04f0fb955bc4a8e634f28c320 <not found>      (4, 1053721)    \n29d6021ea444635e189c4904bf2ecac7193aed2c9952446ac6a19c849f303433 <not found>      (4, 3402532)    \nf038cfead9be72dab5077c23a0ebdc567e5cd75c844c0d75c34b92af2f84db21 <not found>      (4, 113629)     \n4325fc3d8d388d06b4791a8520522a0bb2c63aad27f068bb4ec0148a2bff41a4 <not found>      (4, 110733)     \n32705f06c23cf20fe419d0f90f4140474c4b4ba849b879d12f2465eba2b77d03 <not found>      (4, 1085588)    \n6cd100fe9bd1b77cc5e8bbecbe94141c825463c9073623df695fe1f692b3777d <not found>      (4, 1039965)    \n471d61abe915a5c20dfa5e053e49037c4cf6a71be6b439e231b3c4ee2c7e149f <not found>      (4, 1337181)    \n6fc4b1dc2990e952038c9841bf7dfdb82603d446346f69a8a6d88dac1e0681ff <not found>      (4, 1018630)    \n1e00ba8e989ffbdc2cb651aaac8a501777b745f1b70e09ce054e4b175d2d7426 <not found>      (4, 3707280)    \nd7f12f06854e7808b72846708615996c9b1fd80f4e2f50a4490340f504633885 <not found>      (4, 3544419)    \n13981c0a37fce34935251ba7dafdfc3d9d8835b224612309de22869a94a3f112 <not found>      (4, 3692810)    \nb00390823ef45ffa452b499a507e80297d94d0f1a6f8a419ff9cae2007671907 <not found>      (4, 3553966)    \n1962e377a766717db33624c1082ffb22550b0049a5d3f3a3c44a2218367c0c48 <not found>      (4, 1162566)    \n8c55e6b8842da27674ea725881d5a3e1cbf1bb9963a182f9f55e8dddaf46481a <not found>      (4, 3696924)    \nd233c6b6b20b836e5294570c8014abbdc5a8af52dbfd3a322d62887818033daf <not found>      (4, 1910781)    \nfbb96bf4168337d43ff31d69cb05cb9ee5a53cc598c4b71fd4e8e29b33ece6dd <not found>      (4, 1854140)    \n224bc2b43835dc1facb56fcd985e3ba5884ed5d2b9374c7d7a054b18310c6980 <not found>      (4, 83996)      \n96b6b3b945819a64f20272cd0400132878a44def35cb2860351ef0aa6199d528 <not found>      (4, 108075)     \n9f9ed84401f5a0c8ae08a3624627e3a21b74d6fa964c520975b2e273a68ced55 <not found>      (4, 1049337)    \nf38644dc3d47a16bf256b8b97ce7bd5a34cc42ff48149e71bcbc6220a5ade8c4 <not found>      (4, 182376)     \nc676c50b130b02b184d4e14009e780f9cebe182a4bc2543e06ba2a6d55d3c61a <not found>      (4, 1406109)    \ne48be54b957e764f495fe4a011fab3eca63065d918e6031081633b874c09add5 <not found>      (4, 469453)     \n2fe99459a4c38c0cd3cc0687fb72a54bc95acedfaf60bee7e8e8ed4102bb61ae <not found>      (4, 193541)     \nd0304f17f0fc68b02546527bdfa20198632bc81ab95d6bf709f205322c38f1a1 <not found>      (4, 1412319)    \n4dc4259c9f8c55ed0564f0ace955b974b6aef5be9682baefe3fc516db74e20ba <not found>      (4, 1033114)    \n43d1de059905392951a9f58e46bbdc316490305c1e27aa96c33f157694b27443 <not found>      (4, 1087461)    \nb5042b1ed21a8c6c1cbd564f723eca42bd582f40b88af48dda88bb6b2db519aa <not found>      (4, 1482474)    \n86084d6e3542609acf578042721747694d0e09107544e598ea11677cece736e8 <not found>      (4, 286880)     \nc1e5b558372c5ef107b835d14ccfcebc346b06361d920bd2d296f67589759117 <not found>      (4, 1479845)    \nafc4b7ab74c14524b00d2975aaa1aec61b866848b1433170b84f0b854f7c4ba3 <not found>      (4, 1496589)    \nf0188f35743fc61299e4d08a7ee422d2f0c5fb5b607ffae28086cbf86c25ed8e <not found>      (4, 1284809)    \n25ada8cbb76fe3eacf55940b47412b21cdfc2c2c024590359aa9da05010e13dc <not found>      (4, 1149568)    \n25106324cff4d5064455d67e44e773285eaec0d096d8545515f0dce180195b2f <not found>      (4, 996441)     \n782d5a4209d1eb025f0a09a2af9fae3a84a02d460b477e06dea2b324f1173d85 <not found>      (4, 106716)     \ne6643840f546249bd4982e72080c04cf1ba234d116330b855e44dbc404d9ef83 <not found>      (4, 1141266)    \n8ef68d5a9cb979f69f80fd8e5cf2ea5ba8367ad52c6eb396f22770bf5a3b33c4 <not found>      (4, 3547564)    \nd68929ee01e6c974ac9ba81898407ccee33f58b1ce0de947e2f6633170c7bd6b <not found>      (4, 1516492)    \ndc6cc3e16871309e05e97e77262e4350208c4bf4cf559b09ef30e705209b621c <not found>      (4, 102777)     \nae84355e0bfc31411749800dfe8f14a20df92759b47980172cb278ef6a8a1232 <not found>      (4, 1016579)    \ndc6b9f5deea7f4648c69034da57d3e396884615ea4e5c8ea7e43782bb5c148b8 <not found>      (4, 3400936)    \n4b734bfd3009c7f62fafeda4a855927637d58aeaa79fed8a97f5447798810c08 <not found>      (4, 1059673)    \nb32b3952ef250390da75d0836cd9d0d298b7306173dab9feeef34d79c5684767 <not found>      (4, 1038599)    \n908ec843f0c435681959238248ca2997e4276a3a19617a9c7fbc5dbb79f03347 <not found>      (4, 63013)      \nfb98a84584a20fd6cbeaa25f6a7bf93124bb848dcc674a3d9d63b51176d47a65 <not found>      (4, 195221)     \ne7ff1b6ec36421d088d5f82a94574f22fb581470a840c1bedc4d96394f1fd29f <not found>      (4, 105934)     \nd957cfb1ba437cb0fc892800d0ae328cceb660ca3f1ae4ab599d58f3c13fe9d8 <not found>      (4, 1492393)    \ncc9c6588bc4ddcb63071c1f039e8027a07dbfd9bdff45041d8433310d9bd0b4a <not found>      (4, 1361971)    \n8c528d5ff5540f5ededa1dc194e99ad019e27f0adb78c0c0e0fb0d41d421eff1 <not found>      (4, 1418555)    \ne529e4d02de908c5ebb07499cf640f55b3a37671e91d2e59720c70f30e9b9b21 <not found>      (4, 1879689)    \n5d9214e2f8dabae680d891000f60c5bda0312260e293db49113f2e63a62a84e2 <not found>      (4, 1445914)    \n3966080a8bf2090b97cd05c96a2d04f151103bdba3c86d6131d53c97f8843d0e <not found>      (4, 3717266)    \nc726855984e16e878d9810addeee51835113c55a5507153536dfc6ae7975160c <not found>      (4, 1871577)    \nf02250caf5861d5452ec7f5e3901570c54ae1a782651c53216fdceba513a1129 <not found>      (4, 1438019)    \ndae8a4155aab0ddbe98e2e40530b1806cb0934ee9b207ee3dda4ca701532ee5c <not found>      (4, 287237)     \n1c5d70af0c9fcb5d0267a4f7db9195dd0c4c917f4e98df2f55bb948b76c16d6f <not found>      (4, 3600051)    \n5aa4042cafeba7724ab1d49985d4a12341492f620fffa6090c933bb0be162ac3 <not found>      (4, 280616)     \n1e958bff28639832561ffd5fff99c372571a866db1e60deeac45f1e2e4257daf <not found>      (4, 40803)      \nf4ca6b557b171304ba54826f6b007f289368dba3c378846270b108061b3db78a <not found>      (4, 187348)     \n86be3b9bb7da793afef3e2212e9767474ba06e03abebac48fda6d835fe42dc2b <not found>      (4, 1478003)    \n27b786dc79330cbb3eeae9ed6d2f4f1fd00e485e78ffc5cb1d7f023c28a51383 <not found>      (4, 108885)     \ncedc79351e25e913f1c2c58f768ae7de210d97e05e0d6d9b04b8b3b933b8d825 <not found>      (4, 1884794)    \n8b78fdc61b9e3694324d202ac0871f8eecb71b2a1c975fca52f28b0b5dbb3adf <not found>      (4, 3482960)    \n5c54cd5d1986742de1c6d3af7756604db25d05b0f5a853486f83718e066488ce <not found>      (4, 1063370)    \nd04c64157070f79bbd4d7aa98a0466323c6fac2ff458d102dacaf1eb71f0351a <not found>      (4, 1076062)    \n26b216b836940c9b014959dfacd13f60b4a147c4f9114922304d9654edc9fe5d <not found>      (4, 1512333)    \nddb2d4f5ab6a964d5dd97a57cec1dc0c5dfd31fde2c1912b52eaff5fa64fa667 <not found>      (4, 1182953)    \nda70fbe79bfcfeb1fed1beb75772eedb64d768db4e84079be549e5529894b835 <not found>      (4, 3699539)    \nc7b52bd09522659de1b0da746b66cb376efde533a4b7d90f3c9fd136c961d060 <not found>      (4, 1340450)    \n66c78170a106eeff0961e551c66970c491560f4781c886201ca8618f42823591 <not found>      (4, 3481381)    \n4fe55f75f8cbdeaa4d616d22fc19a1a551a661684a58e11d4cda241a58492ec2 <not found>      (4, 1030347)    \n471fab706454aef36fad3dff4c4d77b8bfc060b0eb531885229e2ed59fecb0c5 <not found>      (4, 1079986)    \n0ac2751cae5823585de40c98a0afdd47d1b7de57a83bb6bd4dc985c3eccda650 <not found>      (4, 1015299)    \nc6b267c0f31460257b2de2f836469de3ec5870211e26d141196a0bdf83a06a58 <not found>      (4, 1011462)    \n6aaed8a2f5b6c848cba6e3bdf66213d5daab445822eeacf2878da80704a53995 <not found>      (4, 1414591)    \n684dbdad333da537a5c2600827f1f624e4f1689f47c61da77fdda0d51eda8183 <not found>      (4, 1354095)    \n33b864cb2b312d495c7fac826f2c291a57dad7b777f758db084f0129887f140e <not found>      (4, 1475260)    \n6aa06f7d407acad6bc65b0fbe638dbe4993fd4d9834fb18ea56d0c392c3f54d2 <not found>      (4, 1173672)    \n651e4ec0e6c4b110f73cddf9195e94d19fb275d6f8432f0cb6b07fd32847f8a6 <not found>      (4, 1874048)    \n520e1096b411645aa07f4dc04621ba20904338ffa3fe456f337082dd3e15ac79 <not found>      (4, 1420771)    \n19250790aaea149398a6becc43ebb5875073b5c78311441b69e676e12e3ac8a9 <not found>      (4, 463911)     \nd0803dbbc450af7ed3a61371a3a275b10e7def88424a47b33de54d727ff5b1ba <not found>      (4, 1047146)    \n27df4cdc10bebdd21fdaa34db561804cd8517cb11d2e3fc3e1b7212870fb38a3 <not found>      (4, 1058964)    \ne83e56a1e7a14358d56cd8ed383c5650748aa62f65397707739b11e821fa70dd <not found>      (4, 1426519)    \n0cf2f899b1e26d74215b5e1bceb153e0c434aa7ca430b46d5bc80287daba9831 <not found>      (4, 1301160)    \n0b6b12ead5f42f93a6f4bed3ca876100a4683ab4a99b0490b4096d8061a2e15c <not found>      (4, 1046545)    \n9d550583c5670c07e3fff03377b45461fad266983495d83cca81a2b15fb86fdf <not found>      (4, 195027)     \nf5a0cf724df50d45efa1a795d60cdd86a9819d9b5c4721d963002161f00c9985 <not found>      (4, 2782486)    \n21e020752f1ddc377dc15e16054be8e46e7caa7039082202427d23aec301f4f9 <not found>      (4, 1084100)    \n1c04f416bb5bb77435710a4e29f7b587bd6209086f8cb3bb2c534fa99f4dccd6 <not found>      (4, 149187)     \nb70409beef1e57b05e395043ff863bbb6a2d644cd894fd22b49eaa09cb2b677f <not found>      (4, 1518894)    \n89076990c9d5ec24c6d0cfa64b020be85a1db6cb545af142dee75c0793574784 <not found>      (4, 3591867)    \n04ff36ea1303848a2f3c51a74e01eecaa4f70823f001cfe1d197df3178af1fa0 <not found>      (4, 132)        \n6342a43806a6ada5c7b3e568adf3bbab4e6e8be154e273d0fad3ef45b4c72d61 <not found>      (4, 556584)     \nc5878e6520f6714a157a1acab6185d3c4ce49544a15b11f3f72f379c64bb0c69 <not found>      (4, 3426260)    \nb7484f210535d5d2da0d640329e5ab2d83485c8f3d43f0a6821601a48976c514 <not found>      (4, 3399936)    \na045f466857cbc6d661e9370cdf5e7d681e61a2c6f815748d4af1b47efc1e3ef <not found>      (4, 89526)      \nc93bd5ad56d2ed43f70a78f8ce780208086df9d0bd7a2236d0e23dc380d9de71 <not found>      (4, 468852)     \nd2e0ddaf58566ad355ca4dd01e1d90295988815791c6f28481b2bb7618651a75 <not found>      (4, 1350597)    \nbdc4e2bfce51b077af33fa8e64fe70eb831380e55e1d06ee49b65ca419a0d7a3 <not found>      (4, 1155935)    \n43b24e7b6c5f06c9507b54727f05d201e1143c0322b9d56413259b86c4d71459 <not found>      (4, 1056885)    \nd8a67aa989db13a0c520f59b3536e7da987d5484f7dcb82cb6d853a5c746e609 <not found>      (4, 3229004)    \n8addd383212cce732ab27a6c78c4f0339f6ed4b15bf7c08fa4a9532f84bd7b50 <not found>      (4, 119992)     \n7ae4b4a1e8d2e408ee7fd8ee11d4f3b05a30e2611a0d1134ea088741fb943832 <not found>      (4, 3506226)    \n8670356437aab40620aaedffff02179744328dd0a400ad62a294c9f7c056197d <not found>      (4, 1853904)    \n6b9bf5973f0e0485f17ae9e1e5049f6aa8e97b1472a91a13cc00dacd98f7f153 <not found>      (4, 3545913)    \ne4a8592bd584c4a08425ea6e67ba587334db6aa384a25102f6d77bb4b11f5f90 <not found>      (4, 1002572)    \nc2a12fed23464b145ff7310747a6c8551d6eb33649364043a15000d2ac4c6dcb <not found>      (4, 1368030)    \n60352646630ac8090c926c79b823f31dce20807115de1c404a06b9d3c251352f <not found>      (4, 1307199)    \n5217fe062a21d5487cbbfb3585f694f6ebd0e1ba7da647ba72cd38c3ca397f14 <not found>      (4, 1309683)    \n999770f264f934910f8e0e756fb5b322b097179142ac7b48aab0da7756a47639 <not found>      (4, 1036562)    \nCompleted repository check, errors found.\n'

borg/testsuite/archiver.py:1502: AssertionError
____________________ ArchiverCheckTestCase.test_check_usage ____________________

self = <borg.testsuite.archiver.ArchiverCheckTestCase testMethod=test_check_usage>

    def test_check_usage(self):
        output = self.cmd('check', '-v', '--progress', self.repository_location, exit_code=0)
>       self.assert_in('Starting repository check', output)
E       AssertionError: 'Starting repository check' not found in 'Checking segments 0.0%\rChecking segments 16.7%\rChecking segments 33.3%\rChecking segments 50.0%\rChecking segments 66.7%\rChecking segments 83.3%\r\r'

borg/testsuite/archiver.py:2944: AssertionError
________________ ArchiverCheckTestCase.test_corrupted_manifest _________________

self = <borg.testsuite.archiver.ArchiverCheckTestCase testMethod=test_corrupted_manifest>

    def test_corrupted_manifest(self):
        archive, repository = self.open_archive('archive1')
        with repository:
            manifest = repository.get(Manifest.MANIFEST_ID)
            corrupted_manifest = manifest + b'corrupted!'
            repository.put(Manifest.MANIFEST_ID, corrupted_manifest)
            repository.commit()
        self.cmd('check', self.repository_location, exit_code=1)
        output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0)
>       self.assert_in('archive1', output)
E       AssertionError: 'archive1' not found in "'check --repair' is an experimental feature that might result in data loss.\nType 'YES' if you understand this and want to continue: YES (from BORG_CHECK_I_KNOW_WHAT_I_AM_DOING)\nRepository manifest is corrupted: Data integrity error: Chunk (unknown): Encryption envelope checksum mismatch\nArchive consistency check complete, problems found.\n"

borg/testsuite/archiver.py:3053: AssertionError
_________ ArchiverCheckTestCase.test_manifest_rebuild_corrupted_chunk __________

self = <borg.testsuite.archiver.ArchiverCheckTestCase testMethod=test_manifest_rebuild_corrupted_chunk>

    def test_manifest_rebuild_corrupted_chunk(self):
        archive, repository = self.open_archive('archive1')
        with repository:
            manifest = repository.get(Manifest.MANIFEST_ID)
            corrupted_manifest = manifest + b'corrupted!'
            repository.put(Manifest.MANIFEST_ID, corrupted_manifest)
    
            chunk = repository.get(archive.id)
            corrupted_chunk = chunk + b'corrupted!'
            repository.put(archive.id, corrupted_chunk)
            repository.commit()
        self.cmd('check', self.repository_location, exit_code=1)
        output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0)
>       self.assert_in('archive2', output)
E       AssertionError: 'archive2' not found in "'check --repair' is an experimental feature that might result in data loss.\nType 'YES' if you understand this and want to continue: YES (from BORG_CHECK_I_KNOW_WHAT_I_AM_DOING)\nRepository manifest is corrupted: Data integrity error: Chunk (unknown): Encryption envelope checksum mismatch\nSkipping corrupted chunk: Data integrity error: Chunk 25da34b42c0017747af714518ed27845ffccd90826bf8e6d7494da6d2e089b9a: Encryption envelope checksum mismatch\n1 orphaned objects found!\nArchive consistency check complete, problems found.\n"

borg/testsuite/archiver.py:3070: AssertionError
________________ ArchiverCheckTestCase.test_missing_file_chunk _________________

self = <borg.testsuite.archiver.ArchiverCheckTestCase testMethod=test_missing_file_chunk>

    def test_missing_file_chunk(self):
        archive, repository = self.open_archive('archive1')
        with repository:
            for item in archive.iter_items():
                if item.path.endswith('testsuite/archiver.py'):
                    valid_chunks = item.chunks
                    killed_chunk = valid_chunks[-1]
                    repository.delete(killed_chunk.id)
                    break
            else:
                self.fail('should not happen')
            repository.commit()
        self.cmd('check', self.repository_location, exit_code=1)
        output = self.cmd('check', '--repair', self.repository_location, exit_code=0)
        self.assert_in('New missing file chunk detected', output)
        self.cmd('check', self.repository_location, exit_code=0)
        output = self.cmd('list', '--format={health}#{path}{LF}', self.repository_location + '::archive1', exit_code=0)
        self.assert_in('broken#', output)
        # check that the file in the old archives has now a different chunk list without the killed chunk
        for archive_name in ('archive1', 'archive2'):
            archive, repository = self.open_archive(archive_name)
            with repository:
                for item in archive.iter_items():
                    if item.path.endswith('testsuite/archiver.py'):
                        self.assert_not_equal(valid_chunks, item.chunks)
                        self.assert_not_in(killed_chunk, item.chunks)
                        break
                else:
                    self.fail('should not happen')
        # do a fresh backup (that will include the killed chunk)
        with patch.object(ChunkBuffer, 'BUFFER_SIZE', 10):
            self.create_src_archive('archive3')
        # check should be able to heal the file now:
        output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0)
>       self.assert_in('Healed previously missing file chunk', output)
E       AssertionError: 'Healed previously missing file chunk' not found in "'check --repair' is an experimental feature that might result in data loss.\nType 'YES' if you understand this and want to continue: YES (from BORG_CHECK_I_KNOW_WHAT_I_AM_DOING)\n"

borg/testsuite/archiver.py:2999: AssertionError
_________________ ArchiverCheckTestCase.test_missing_manifest __________________

self = <borg.testsuite.archiver.ArchiverCheckTestCase testMethod=test_missing_manifest>

    def test_missing_manifest(self):
        archive, repository = self.open_archive('archive1')
        with repository:
            repository.delete(Manifest.MANIFEST_ID)
            repository.commit()
        self.cmd('check', self.repository_location, exit_code=1)
        output = self.cmd('check', '-v', '--repair', self.repository_location, exit_code=0)
>       self.assert_in('archive1', output)
E       AssertionError: 'archive1' not found in "'check --repair' is an experimental feature that might result in data loss.\nType 'YES' if you understand this and want to continue: YES (from BORG_CHECK_I_KNOW_WHAT_I_AM_DOING)\nRepository manifest not found!\nArchive consistency check complete, problems found.\n"

borg/testsuite/archiver.py:3040: AssertionError
_________________ ManifestAuthenticationTest.test_not_required _________________

self = <borg.testsuite.archiver.ManifestAuthenticationTest testMethod=test_not_required>

    def test_not_required(self):
        self.cmd('init', '--encryption=repokey', self.repository_location)
        self.create_src_archive('archive1234')
        repository = Repository(self.repository_path, exclusive=True)
        with repository:
            shutil.rmtree(get_security_dir(bin_to_hex(repository.id)))
            _, key = Manifest.load(repository, Manifest.NO_OPERATION_CHECK)
            key.tam_required = False
            key.change_passphrase(key._passphrase)
    
            manifest = msgpack.unpackb(key.decrypt(None, repository.get(Manifest.MANIFEST_ID)))
            del manifest[b'tam']
            repository.put(Manifest.MANIFEST_ID, key.encrypt(msgpack.packb(manifest)))
            repository.commit()
        output = self.cmd('list', '--debug', self.repository_location)
        assert 'archive1234' in output
>       assert 'TAM not found and not required' in output
E       AssertionError: assert 'TAM not found and not required' in 'using builtin fallback logging configuration\narchive1234                          Wed, 2018-07-04 14:01:40 [111123cc3edf41dc9041e76cf6c5ffa53d2253ab6a65b7553ff0cf147532956a]\n'

borg/testsuite/archiver.py:3218: AssertionError
__________________ RemoteArchiverTestCase.test_common_options __________________

self = <borg.testsuite.archiver.RemoteArchiverTestCase testMethod=test_common_options>

    def test_common_options(self):
        self.create_test_files()
        self.cmd('init', '--encryption=repokey', self.repository_location)
        log = self.cmd('--debug', 'create', self.repository_location + '::test', 'input')
>       assert 'security: read previous location' in log
E       AssertionError: assert 'security: read previous location' in 'using builtin fallback logging configuration\nRemote: using builtin fallback logging configuration\nRemote: 35 self t...lated working space for compact as -33 bytes\nRemote: check_free_space: required bytes 169135, free bytes 8312791040\n'

borg/testsuite/archiver.py:2025: AssertionError
_________________ ArchiverCorruptionTestCase.test_cache_files __________________

self = <borg.testsuite.archiver.ArchiverCorruptionTestCase testMethod=test_cache_files>

    def test_cache_files(self):
        self.cmd('create', self.repository_location + '::test', 'input')
        self.corrupt(os.path.join(self.cache_path, 'files'))
        out = self.cmd('create', self.repository_location + '::test1', 'input')
        # borg warns about the corrupt files cache, but then continues without files cache.
>       assert 'files cache is corrupted' in out
E       AssertionError: assert 'files cache is corrupted' in ''

borg/testsuite/archiver.py:3345: AssertionError
________________ ArchiverCorruptionTestCase.test_chunks_archive ________________

self = <borg.testsuite.archiver.ArchiverCorruptionTestCase testMethod=test_chunks_archive>

    def test_chunks_archive(self):
        self.cmd('create', self.repository_location + '::test1', 'input')
        # Find ID of test1 so we can corrupt it later :)
        target_id = self.cmd('list', self.repository_location, '--format={id}{LF}').strip()
        self.cmd('create', self.repository_location + '::test2', 'input')
    
        # Force cache sync, creating archive chunks of test1 and test2 in chunks.archive.d
        self.cmd('delete', '--cache-only', self.repository_location)
        self.cmd('info', self.repository_location, '--json')
    
        chunks_archive = os.path.join(self.cache_path, 'chunks.archive.d')
        assert len(os.listdir(chunks_archive)) == 4  # two archives, one chunks cache and one .integrity file each
    
        self.corrupt(os.path.join(chunks_archive, target_id + '.compact'))
    
        # Trigger cache sync by changing the manifest ID in the cache config
        config_path = os.path.join(self.cache_path, 'config')
        config = ConfigParser(interpolation=None)
        config.read(config_path)
        config.set('cache', 'manifest', bin_to_hex(bytes(32)))
        with open(config_path, 'w') as fd:
            config.write(fd)
    
        # Cache sync notices corrupted archive chunks, but automatically recovers.
        out = self.cmd('create', '-v', self.repository_location + '::test3', 'input', exit_code=1)
>       assert 'Reading cached archive chunk index for test1' in out
E       AssertionError: assert 'Reading cached archive chunk index for test1' in 'Cached archive chunk index of test1 is corrupted: File failed integrity check: /tmp/tmp6pnmxa1f/cache/30a6c025b272935...c98ca2517321ebaced30dbb6fd/chunks.archive.d/65f7a4569f6f41db53008e0af6c787fabd39c744cfc9522c53675ddedd7be277.compact\n'

borg/testsuite/archiver.py:3372: AssertionError
____________ ArchiverCorruptionTestCase.test_old_version_interfered ____________

self = <borg.testsuite.archiver.ArchiverCorruptionTestCase testMethod=test_old_version_interfered>

    def test_old_version_interfered(self):
        # Modify the main manifest ID without touching the manifest ID in the integrity section.
        # This happens if a version without integrity checking modifies the cache.
        config_path = os.path.join(self.cache_path, 'config')
        config = ConfigParser(interpolation=None)
        config.read(config_path)
        config.set('cache', 'manifest', bin_to_hex(bytes(32)))
        with open(config_path, 'w') as fd:
            config.write(fd)
    
        out = self.cmd('info', self.repository_location)
>       assert 'Cache integrity data not available: old Borg version modified the cache.' in out
E       AssertionError: assert 'Cache integrity data not available: old Borg version modified the cache.' in 'Repository ID: 8cf3479a67f4dd5ab4f8cbd675c1db32da964d4a74ad5a227a20fb2a20bbd061\nLocation: /tmp/tmpju4hp9l1/repositor...                       Unique chunks         Total chunks\nChunk index:                       0                    0\n'

borg/testsuite/archiver.py:3387: AssertionError

----------- coverage: platform linux, python 3.7.0-final-0 -----------
Name                               Stmts   Miss  Cover
------------------------------------------------------
borg/__init__.py                       4      0   100%
borg/__main__.py                       2      2     0%
borg/_version.py                       1      0   100%
borg/algorithms/__init__.py            0      0   100%
borg/archive.py                     1296    191    85%
borg/archiver.py                    1924    245    87%
borg/cache.py                        765     73    90%
borg/constants.py                     33      0   100%
borg/crypto/__init__.py                0      0   100%
borg/crypto/file_integrity.py        145      2    99%
borg/crypto/key.py                   560     52    91%
borg/crypto/keymanager.py            170      9    95%
borg/crypto/nonces.py                 56      1    98%
borg/fuse.py                         418    418     0%
borg/helpers.py                     1393    163    88%
borg/locking.py                      268     23    91%
borg/logger.py                       118     27    77%
borg/lrucache.py                      39      0   100%
borg/nanorst.py                      159     13    92%
borg/patterns.py                     197     12    94%
borg/platform/__init__.py             23      8    65%
borg/platform/base.py                106     22    79%
borg/remote.py                       606     78    87%
borg/repository.py                   978     83    92%
borg/selftest.py                      42      7    83%
borg/shellpattern.py                  36      0   100%
borg/testsuite/__init__.py           215     54    75%
borg/testsuite/archive.py            171      2    99%
borg/testsuite/archiver.py          2859    414    86%
borg/testsuite/benchmark.py           61     32    48%
borg/testsuite/cache.py              146      0   100%
borg/testsuite/checksums.py           26      0   100%
borg/testsuite/chunker.py             33      0   100%
borg/testsuite/compress.py           131      3    98%
borg/testsuite/crypto.py             103      0   100%
borg/testsuite/file_integrity.py     100      2    98%
borg/testsuite/hashindex.py          390      0   100%
borg/testsuite/helpers.py            659     10    98%
borg/testsuite/item.py               116      0   100%
borg/testsuite/key.py                297      0   100%
borg/testsuite/locking.py            199      0   100%
borg/testsuite/logger.py              40      0   100%
borg/testsuite/lrucache.py            51      0   100%
borg/testsuite/nanorst.py             21      0   100%
borg/testsuite/nonces.py             166      2    99%
borg/testsuite/patterns.py           134      0   100%
borg/testsuite/platform.py           146     43    71%
borg/testsuite/remote.py             137      0   100%
borg/testsuite/repository.py         766      0   100%
borg/testsuite/shellpattern.py        20      0   100%
borg/testsuite/upgrader.py            85      0   100%
borg/testsuite/version.py             15      0   100%
borg/testsuite/xattr.py               50     34    32%
borg/upgrader.py                     169     54    68%
borg/version.py                       26      0   100%
borg/xattr.py                        252    150    40%
------------------------------------------------------
TOTAL                              16923   2229    87%

 12 failed, 1104 passed, 225 skipped, 1 deselected, 2 xfailed in 578.25 seconds 
```
https://github.com/python/cpython/commit/78c18a9b9a1445f7c755929917a790ba02b4a5e0#diff-c11fc567cf5b6e518eb855c23b5599bc

it seems to be a bug in python 3.7 due to caching in the logging module.

i reviewed the code, but could not find an obvious bug in caching or invalidation of the cache, but some must be there. i reverted the caching change locally and without caching our tests work as they used to work in 3.6.
Hmm, maybe part of the problem is also our test teardown method is clearing the internal `loggerDict` of `logging` before setting up logging from scratch.

I am currently testing whether this is needed at all, because without that, it seems to work on 3.6 and on 3.7.


Resolving patch:
diff --git a/src/borg/testsuite/archiver.py b/src/borg/testsuite/archiver.py
--- a/src/borg/testsuite/archiver.py
+++ b/src/borg/testsuite/archiver.py
@@ -279,8 +279,6 @@ def tearDown(self):
         os.chdir(self._old_wd)
         # note: ignore_errors=True as workaround for issue #862
         shutil.rmtree(self.tmpdir, ignore_errors=True)
-        # destroy logging configuration
-        logging.Logger.manager.loggerDict.clear()
         setup_logging()
 
     def cmd(self, *args, **kw):


Test patch: