Repository: cfv-project/cfv (language: python)

Issue:
Test suite expects `/data1` to not exists
As reported by @Terry-Kennedy, a test case fails, when `/data1` exists: https://github.com/cfv-project/cfv/issues/8#issuecomment-889520893

```
>>> failed test: /sysprog/terry/cfv-python3/test/cfv -ZNVRMUI --unquote=no --fixpaths="" --strippaths=0 --showpaths=autorelative --progress=no --announceurl=url --noprivate_torrent --strippaths=none -T -f teststrip-none.csv4 
/data1 : file size does not match (13!=33)
teststrip-none.csv4: 1 files, 0 OK, 1 badsize.  0.000 seconds, 0.0K/s
FAILED (4)
  File "test/test.py", line 1876, in <module>
  File "test/test.py", line 1719, in all_tests
  File "test/test.py", line 269, in test_generic
  File "test/test.py", line 254, in test_log_results
  File "test/test.py", line 237, in test_log_finish
```

It is bad practice that the test suite makes assumptions about things outside it's own scope (i.e. temp directory). However, the test case cannot easily be adjusted without modifying it's intention. Therefore, we just change the expected test outcome, depending on if the `/data1` exists or not. Leaving only an unexpected error, if someone has a file `/data1` with exactly the same content as https://github.com/cfv-project/cfv/blob/python3/test/testdata/data1.
Issue warnings when external verification tools are not installed
The test suite can verify created files using external tools like cksfv and *sum. However, if they are not installed the verification is just silently skipped. We should at least issue a warning, if the tools are not installed.
Issue warnings when external verification tools are not installed
The test suite can verify created files using external tools like cksfv and *sum. However, if they are not installed the verification is just silently skipped. We should at least issue a warning, if the tools are not installed.


Discussion / hints:

Fix patch:
diff --git a/.github/workflows/ci-python3.yml b/.github/workflows/ci-python3.yml
index a9979ef..5bc14ab 100644
--- a/.github/workflows/ci-python3.yml
+++ b/.github/workflows/ci-python3.yml
@@ -16,11 +16,11 @@ jobs:
   test:
     # The type of runner that the job will run on
     runs-on: ${{ matrix.os }}
-    
+
     strategy:
       matrix:
-        os: [ubuntu-20.04, ubuntu-18.04]
-        python-version: ['3.5', '3.6', '3.7', '3.8', '3.9']
+        os: [ ubuntu-20.04, ubuntu-18.04 ]
+        python-version: [ '3.5', '3.6', '3.7', '3.8', '3.9' ]
 
     # Steps represent a sequence of tasks that will be executed as part of the job
     steps:
@@ -31,44 +31,45 @@ jobs:
         uses: actions/setup-python@v2
         with:
           python-version: ${{ matrix.python-version }}
-      
+
       - name: Install test dependencies
         run: |
-         pip install pyroma
-         pip install check-manifest
-         pip install twine
-         pip install wheel
-         pip install flake8
-         pip list
-         
+          sudo apt install coreutils cksfv
+          pip install pyroma
+          pip install check-manifest
+          pip install twine
+          pip install wheel
+          pip install flake8
+          pip list
+
       - name: Check syntax by compiling code
         run: python -m compileall -f .
-          
+
       - name: Run unit tests
         run: python -bb test/test.py --unit --exit-early
-        
+
       - name: Run integration tests (internal)
         run: |
           ulimit -n
           ulimit -n 4096
           python -bb test/test.py -i --exit-early
-        
+
       - name: Run integration tests (external process)
         run: |
           ulimit -n
           ulimit -n 4096
           test/test.py -e --exit-early
-        
+
       - name: Check package quality
         run: pyroma -n 9 .
-        
+
       - name: Check the completeness of MANIFEST.in
         run: check-manifest --ignore=Release.md .
-      
+
       - name: Run flake
         continue-on-error: true
         run: flake8 --exclude=build,venv --ignore= --max-line-length=200 --max-complexity=75 --show-source --statistics .
-          
+
       - name: Check distribution
         run: |
           python setup.py sdist bdist_wheel


Test patch:
diff --git a/test/test.py b/test/test.py
index ccd64d1..2d230ae 100755
--- a/test/test.py
+++ b/test/test.py
@@ -1712,11 +1712,17 @@ def cfv_torrentcommentencoding_test(s, o):
     test_generic(cfvcmd + ' -r -u -p a ' + os.path.join('C', 'foo.bar'), cfv_test)
     test_generic(cfvcmd + ' -ri -u -p a ' + os.path.join('c', 'fOo.BaR'), cfv_test)
 
+    def cfv_notfound_or_bad_test(path):
+        if os.path.exists(path):
+            return cfv_bad_test
+        else:
+            return cfv_notfound_test
+
     test_generic(cfvcmd + ' --strippaths=0 -T -f teststrip0.csv4', cfv_test)
     test_generic(cfvcmd + ' --strippaths=1 -T -f teststrip1.csv4', cfv_test)
     test_generic(cfvcmd + ' --strippaths=2 -T -f teststrip2.csv4', cfv_test)
     test_generic(cfvcmd + ' --strippaths=all -T -f teststrip-1.csv4', cfv_test)
-    test_generic(cfvcmd + ' --strippaths=none -T -f teststrip-none.csv4', cfv_notfound_test)
+    test_generic(cfvcmd + ' --strippaths=none -T -f teststrip-none.csv4', cfv_notfound_or_bad_test('/data1'))
     test_generic(cfvcmd + r' --strippaths=0 --fixpaths \\/ -T -f testdrivestrip.md5', rcurry(cfv_all_test, ok=4))
     test_generic(cfvcmd + r' --strippaths=0 --unquote=yes --fixpaths \\/ -T -f testdrivestripquoted.md5', rcurry(cfv_all_test, ok=4))
     test_generic(cfvcmd + r' --strippaths=0 --unquote=yes --fixpaths \\/ -T -f testdrivestripquoted.md5 data1 data3 data4', rcurry(cfv_all_test, ok=3))
@@ -1736,6 +1742,7 @@ def cfv_torrentcommentencoding_test(s, o):
             def coreutils_verify(f):
                 test_external(fmt + 'sum -c ' + f, status_test)
         else:
+            print('skipping %s verify using external tool %ssum, as it is not installed.' % (fmt, fmt))
             coreutils_verify = None
         C_test(fmt, verify=coreutils_verify)
     C_test('csv')
@@ -1743,6 +1750,7 @@ def coreutils_verify(f):
         def sfvverify(f):
             test_external('cksfv -f ' + f, status_test)
     else:
+        print('skipping sfv verify using external tool cksfv, as it is not installed.')
         sfvverify = None
     C_test('sfv', verify=sfvverify)
     C_test('sfvmd5', '-t sfvmd5')