Repository: tarantool/tarantool-python (language: python)

Issue:
Ping test fails because of Windows time precision
See https://github.com/tarantool/tarantool-python/pull/213 PR. The issue is as follows: [ping test](https://github.com/tarantool/tarantool-python/blob/f1b3cf2262e88531487202a039106203b7511a40/test/suites/test_dml.py#L145-L150) started to fail (in ~70% cases) after I migrated Windows CI from Appveyor to GitHub Actions. As a part of this migration, the test Tarantool instance was changed from an instance started on a remote Linux server to an instance started on the same Windows server under WSL. Thus, request time has shortened and it supposedly caused the ping test to fail.

Python [documentation](https://docs.python.org/3/library/time.html#time.time) declares that

> though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second

Particularly, this [StackOverflow answer](https://stackoverflow.com/a/1938096/11646599) argues that

> For Linux and Mac precision is +- 1 microsecond or 0.001 milliseconds. Python on Windows uses +- 16 milliseconds precision due to clock implementation problems due to process interrupts.

based on Windows [documentation](https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/high-resolution-timers).

Assuming that ping requests between the same server services can easily be under 1 ms, it caused the test to fail.

I see two ways to "solve" this issue:
- Rework the test, document ping return value precision on Windows.
- Rework time measurement to be more precise on Windows.

The second approach is more time-consuming and I'm not sure there are any users that really need this to be fixed. So I think it should be the first one until someone interested in the second one appears.
Setup GitHub Actions
Instead of Travis CI (which is going to be paid service) and AppVeyor (because GitHub Actions has Windows runners).
Ping test fails because of Windows time precision
See https://github.com/tarantool/tarantool-python/pull/213 PR. The issue is as follows: [ping test](https://github.com/tarantool/tarantool-python/blob/f1b3cf2262e88531487202a039106203b7511a40/test/suites/test_dml.py#L145-L150) started to fail (in ~70% cases) after I migrated Windows CI from Appveyor to GitHub Actions. As a part of this migration, the test Tarantool instance was changed from an instance started on a remote Linux server to an instance started on the same Windows server under WSL. Thus, request time has shortened and it supposedly caused the ping test to fail.

Python [documentation](https://docs.python.org/3/library/time.html#time.time) declares that

> though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second

Particularly, this [StackOverflow answer](https://stackoverflow.com/a/1938096/11646599) argues that

> For Linux and Mac precision is +- 1 microsecond or 0.001 milliseconds. Python on Windows uses +- 16 milliseconds precision due to clock implementation problems due to process interrupts.

based on Windows [documentation](https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/high-resolution-timers).

Assuming that ping requests between the same server services can easily be under 1 ms, it caused the test to fail.

I see two ways to "solve" this issue:
- Rework the test, document ping return value precision on Windows.
- Rework time measurement to be more precise on Windows.

The second approach is more time-consuming and I'm not sure there are any users that really need this to be fixed. So I think it should be the first one until someone interested in the second one appears.
Setup GitHub Actions
Instead of Travis CI (which is going to be paid service) and AppVeyor (because GitHub Actions has Windows runners).


Discussion / hints:

Fix patch:
diff --git a/README.rst b/README.rst
index 3b5af060..aa0167d5 100644
--- a/README.rst
+++ b/README.rst
@@ -11,11 +11,8 @@ This package is a pure-python client library for `Tarantool`_.
 .. _`GitHub`: https://github.com/tarantool/tarantool-python
 .. _`Issue tracker`: https://github.com/tarantool/tarantool-python/issues
 
-.. image:: https://travis-ci.org/tarantool/tarantool-python.svg?branch=master
-    :target: https://travis-ci.org/tarantool/tarantool-python
-
-.. image:: https://ci.appveyor.com/api/projects/status/github/tarantool/tarantool-python?branch=master
-    :target: https://ci.appveyor.com/project/tarantool/tarantool-python
+.. image:: https://github.com/tarantool/tarantool-python/actions/workflows/testing.yml/badge.svg?branch=master
+    :target: https://github.com/tarantool/tarantool-python/actions/workflows/testing.yml
 
 Download and Install
 --------------------


Test patch:
diff --git a/.github/workflows/testing.yml b/.github/workflows/testing.yml
new file mode 100644
index 00000000..c6d6cf08
--- /dev/null
+++ b/.github/workflows/testing.yml
@@ -0,0 +1,167 @@
+name: testing
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  run_tests_linux:
+    # We want to run on external PRs, but not on our own internal
+    # PRs as they'll be run by the push to the branch.
+    #
+    # The main trick is described here:
+    # https://github.com/Dart-Code/Dart-Code/pull/2375
+    if: github.event_name == 'push' ||
+      github.event.pull_request.head.repo.full_name != github.repository
+
+    runs-on: ubuntu-20.04
+
+    strategy:
+      fail-fast: false
+      matrix:
+        tarantool:
+          - '1.10'
+          - '2.8'
+          - '2.x-latest'
+        python:
+          - '2.7'
+          - '3.5'
+          - '3.6'
+          - '3.7'
+          - '3.8'
+          - '3.9'
+          - '3.10'
+        msgpack-deps:
+          # latest msgpack will be installed as a part of requirements.txt
+          - ''
+
+        # Adding too many elements to three-dimentional matrix results in
+        # too many test cases. It causes GitHub webpages to fail with 
+        # "This page is taking too long to load." error. Thus we use
+        # pairwise testing.
+        include:
+          - tarantool: '2.8'
+            python: '3.10'
+            msgpack-deps: 'msgpack-python==0.4.0'
+          - tarantool: '2.8'
+            python: '3.10'
+            msgpack-deps: 'msgpack==0.5.0'
+          - tarantool: '2.8'
+            python: '3.10'
+            msgpack-deps: 'msgpack==0.6.2'
+          - tarantool: '2.8'
+            python: '3.10'
+            msgpack-deps: 'msgpack==1.0.0'
+
+    steps:
+      - name: Clone the connector
+        uses: actions/checkout@v2
+
+      - name: Install tarantool ${{ matrix.tarantool }}
+        if: matrix.tarantool != '2.x-latest'
+        uses: tarantool/setup-tarantool@v1
+        with:
+          tarantool-version: ${{ matrix.tarantool }}
+
+      - name: Install latest tarantool 2.x
+        if: matrix.tarantool == '2.x-latest'
+        run: |
+          curl -L https://tarantool.io/pre-release/2/installer.sh | sudo bash
+          sudo apt install -y tarantool tarantool-dev
+
+      - name: Setup Python for tests
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python }}
+
+      - name: Install specific version of msgpack package
+        if: startsWith(matrix.msgpack-deps, 'msgpack==') == true
+        run: |
+          pip install ${{ matrix.msgpack-deps }}
+
+      - name: Install specific version of msgpack-python package
+        # msgpack package is a replacement for deprecated msgpack-python.
+        # To test compatibility with msgpack-python we must ignore
+        # requirements.txt install of msgpack package by overwriting it
+        # with sed.
+        if: startsWith(matrix.msgpack-deps, 'msgpack-python==') == true
+        run: |
+          pip install ${{ matrix.msgpack-deps }}
+          sed -i -e "s/^msgpack.*$/${{ matrix.msgpack-deps }}/" requirements.txt
+
+      - name: Install package requirements
+        run: pip install -r requirements.txt
+
+      - name: Install test requirements
+        run: pip install -r requirements-test.txt
+
+      - name: Run tests
+        run: make test
+
+  run_tests_windows:
+    # We want to run on external PRs, but not on our own internal
+    # PRs as they'll be run by the push to the branch.
+    #
+    # The main trick is described here:
+    # https://github.com/Dart-Code/Dart-Code/pull/2375
+    if: github.event_name == 'push' ||
+      github.event.pull_request.head.repo.full_name != github.repository
+
+    runs-on: windows-2022
+
+    strategy:
+      fail-fast: false
+      matrix:
+        tarantool:
+          - '1.10'
+          - '2.8'
+        python:
+          - '2.7'
+          - '3.10'
+
+    steps:
+      - name: Clone the connector
+        uses: actions/checkout@v2
+
+      - name: Setup Python for tests
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python }}
+
+      - name: Install connector requirements
+        run: pip install -r requirements.txt
+
+      - name: Install test requirements
+        run: pip install -r requirements-test.txt
+
+      - name: Setup WSL for tarantool
+        uses: Vampire/setup-wsl@v1
+        with:
+          distribution: Ubuntu-20.04
+
+      - name: Install tarantool ${{ matrix.tarantool }} for WSL
+        shell: wsl-bash_Ubuntu-20.04 {0}
+        run: |
+          curl -L https://tarantool.io/installer.sh | VER=${{ matrix.tarantool }} bash -s -- --type "release"
+          sudo apt install -y tarantool tarantool-dev
+
+      - name: Setup test tarantool instance
+        shell: wsl-bash_Ubuntu-20.04 {0}
+        run: |
+          rm -f ./tarantool.pid ./tarantool.log
+          TNT_PID=$(tarantool ./test/suites/lib/tarantool_python_ci.lua > tarantool.log 2>&1 & echo $!)
+          touch tarantool.pid
+          echo $TNT_PID > ./tarantool.pid
+
+      - name: Run tests
+        env:
+          REMOTE_TARANTOOL_HOST: localhost
+          REMOTE_TARANTOOL_CONSOLE_PORT: 3302
+        run: make test
+
+      - name: Stop test tarantool instance
+        if: ${{ always() }}
+        shell: wsl-bash_Ubuntu-20.04 {0}
+        run: |
+          cat tarantool.log || true
+          kill $(cat tarantool.pid) || true
diff --git a/test/suites/lib/tarantool_python_ci.lua b/test/suites/lib/tarantool_python_ci.lua
index 2305c0d7..471b686f 100644
--- a/test/suites/lib/tarantool_python_ci.lua
+++ b/test/suites/lib/tarantool_python_ci.lua
@@ -179,18 +179,33 @@ clean = function()
     end)
 
     local _FUNC_NAME = 3
+    local _FUNC_LANGUAGE = 5
     local allowed_funcs = {
         ['box.schema.user.info'] = true,
     }
+    local allowed_langs = {
+        ['SQL_BUILTIN'] = true,
+    }
     box.space._func:pairs():map(function(tuple)
         local name = tuple[_FUNC_NAME]
-        return name
-    end):filter(function(name)
-        return not allowed_funcs[name]
-    end):each(function(name)
-        box.schema.func.drop(name)
+        local lang = tuple[_FUNC_LANGUAGE]
+        return { name = name, lang = lang }
+    end):filter(function(prop)
+        return not allowed_funcs[prop.name]
+    end):filter(function(prop)
+        return not allowed_langs[prop.lang]
+    end):each(function(prop)
+        box.schema.func.drop(prop.name)
     end)
 
+    local sql_builtin_func_count = box.space._func:pairs():map(function(tuple)
+        local lang = tuple[_FUNC_LANGUAGE]
+        if lang == 'SQL_BUILTIN' then
+            return 1
+        end
+        return 0
+    end):sum()
+
     cleanup_cluster()
 
     local cleanup_list = function(list, allowed)
@@ -333,8 +348,9 @@ clean = function()
     local user_count = box.space._user:count()
     assert(user_count == 4 or user_count == 5,
         'box.space._user:count() should be 4 (1.10) or 5 (2.0)')
-    assert(box.space._func:count() == 1,
-        'box.space._func:count() should be only one')
+    assert(box.space._func:count() == 1 + sql_builtin_func_count,
+        'box.space._func:count() should be 1 (1.10 and >= 2.10)' ..
+        ' or 1 + count of SQL_BUILTIN functions (>= 2.2.1, < 2.10)')
     assert(box.space._cluster:count() == 1,
         'box.space._cluster:count() should be only one')
 
diff --git a/test/suites/test_dml.py b/test/suites/test_dml.py
index e3922a10..6463778f 100644
--- a/test/suites/test_dml.py
+++ b/test/suites/test_dml.py
@@ -146,6 +146,9 @@ def test_05_ping(self):
         # Simple ping test
         # * No exceptions are raised
         # * Ping time > 0
+        if sys.platform.startswith("win"):
+            self.skipTest("Windows clock precision causes test to fail sometimes, see #214")
+
         self.assertTrue(self.con.ping() > 0)
         self.assertEqual(self.con.ping(notime=True), "Success")