Repository: jina-ai/serve
Pull request: refactor(logs): remove legacy SSE logserver from core
## Breaking Change #1606 

`jinad` and `fluentd` logger have been there for quite a while, yet there is no clue they work with dashboard. 

I'm leaving this PR here to push the progress a bit and clarify the scope. This PR removes the legacy SSE logger completely from the core, **breaking** the connection between the core and dashboard.


Problem statement:
use jinad for logging and communicating with dashboard
**Describe your proposal/problem**

I have never really see how fluentd works in practice. This makes me doubt the completeness of the old ticket on logging.

High priority. Must be done before 1.0, the hello-world with dashboard example must use fluentd as the logging source.

To make this first step, removing `sse_logger` in the next PR to kick off this feature. I'm afraid if we don't do this step, then this ticket is forgot again and we never really use `fluentd` anywhere.


---

<!-- Optional, but really help us locate the problem faster -->

**Environment**
<!-- Run `jina --version-full` and copy paste the output here -->

**Screenshots**
<!-- If applicable, add screenshots to help explain your problem. -->

Hints from issue discussion:

Commit message: refactor: remove sse logger

Patch to review:
diff --git a/.github/workflows/cd.yml b/.github/workflows/cd.yml
index 25dcfc32615ac..d36f73f03fd9a 100644
--- a/.github/workflows/cd.yml
+++ b/.github/workflows/cd.yml
@@ -22,6 +22,7 @@ jobs:
       - uses: actions/checkout@v2
         with:
           submodules: true
+          fetch-depth: 100
       - run: |
           truncate --size=24KB README.md > README-trunc.md
       - uses: peter-evans/dockerhub-description@v2.3.0
@@ -166,7 +167,7 @@ jobs:
       - id: set-matrix
         run: |
           sudo apt-get install jq
-          echo "::set-output name=matrix::$(bash scripts/get-test-paths.sh)"
+          echo "::set-output name=matrix::$(bash scripts/get-all-test-paths.sh tests)"
     outputs:
       matrix: ${{ steps.set-matrix.outputs.matrix }}
 
@@ -192,14 +193,14 @@ jobs:
           docker pull docker.pkg.github.com/jina-ai/jina/jina:test-pip
           docker tag docker.pkg.github.com/jina-ai/jina/jina:test-pip jinaai/jina:test-pip
           python -m pip install --upgrade pip
-          pip install ".[cicd,test]" --no-cache-dir
+          pip install ".[cicd,test,daemon]" --no-cache-dir
           jina check
           export JINA_LOG_VERBOSITY="ERROR"
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       - name: Test
         run: |
-          pytest --suppress-no-test-exit-code --force-flaky --min-passes 1 --max-runs 5 --cov=jina --cov-report=xml -n 1 --timeout=120 -v --ignore-glob='tests/integration/hub_usage/dummyhub*' tests/${{ matrix.test-path }}
+          pytest --suppress-no-test-exit-code --force-flaky --min-passes 1 --max-runs 5 --cov=jina --cov-report=xml -n 1 --timeout=120 -v --ignore-glob='tests/integration/hub_usage/dummyhub*' ${{ matrix.test-path }}
         timeout-minutes: 20
         env:
           JINAHUB_USERNAME: ${{ secrets.JINAHUB_USERNAME }}
@@ -216,50 +217,4 @@ jobs:
         with:
           file: coverage.xml
           name: ${{ matrix.test-path }}-codecov
-          flags: core, cd
-          fail_ci_if_error: false
-
-  daemon-test:
-    needs: prep-testbed
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version: [3.7]
-        test-path: ['unit', 'integration']
-
-    steps:
-      - uses: actions/checkout@v2
-        with:
-          submodules: true
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v1
-        with:
-          python-version: ${{ matrix.python-version }}
-      - name: Prepare enviroment
-        run: |
-          docker login docker.pkg.github.com -u $GITHUB_ACTOR -p $GITHUB_TOKEN
-          docker pull docker.pkg.github.com/jina-ai/jina/jina:test-pip
-          docker tag docker.pkg.github.com/jina-ai/jina/jina:test-pip jinaai/jina:test-pip
-          python -m pip install --upgrade pip
-          pip install ".[cicd,test,daemon]" --no-cache-dir
-          jina check
-          export JINA_LOG_VERBOSITY="ERROR"
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      - name: Test
-        run: |
-          pytest --suppress-no-test-exit-code --force-flaky --min-passes 1 --max-runs 5 --cov=jina --cov-report=xml -n 1 --timeout=120 -v tests/jinad/${{ matrix.test-path }}
-        timeout-minutes: 20
-        env:
-          JINAHUB_USERNAME: ${{ secrets.JINAHUB_USERNAME }}
-          JINAHUB_PASSWORD: ${{ secrets.JINAHUB_PASSWORD }}
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      - name: Upload coverage from test to Codecov
-        uses: codecov/codecov-action@v1
-        if: ${{ matrix.python-version }} == 3.7
-        with:
-          file: coverage.xml
-          name: jinad-${{ matrix.python-version }}-${{ matrix.test-path }}-codecov
-          flags: ${{ matrix.test-path }}, jinad, ci
           fail_ci_if_error: false
\ No newline at end of file
diff --git a/.github/workflows/ci-hub.yml b/.github/workflows/ci-hub.yml
index d1ab0548ceeb6..4cf03bdb00b04 100644
--- a/.github/workflows/ci-hub.yml
+++ b/.github/workflows/ci-hub.yml
@@ -6,18 +6,14 @@ on:
 jobs:
   hub-test:
     runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        python-version: [3.7]
-
     steps:
       - uses: actions/checkout@v2
         with:
           submodules: true
-      - name: Set up Python ${{ matrix.python-version }}
+      - name: Set up Python 3.7
         uses: actions/setup-python@v1
         with:
-          python-version: ${{ matrix.python-version }}
+          python-version: 3.7
       - name: Test hubapp with hubpods
         run: |
           ./tests/jinahub/test_integration.sh
diff --git a/.github/workflows/ci-jinad.yml b/.github/workflows/ci-jinad.yml
deleted file mode 100644
index 11739e93e1cb6..0000000000000
--- a/.github/workflows/ci-jinad.yml
+++ /dev/null
@@ -1,46 +0,0 @@
-name: CI
-
-on:
-  pull_request:
-#    paths-ignore:
-#      - 'README.md'
-#      - '.all-contributorsrc'
-#      - 'docs/*'
-
-jobs:
-  daemon-test:
-    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        python-version: [3.7]
-        test-path: ['unit', 'integration']
-
-    steps:
-      - uses: actions/checkout@v2
-        with:
-          submodules: true
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v1
-        with:
-          python-version: ${{ matrix.python-version }}
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          pip install ".[cicd,daemon,test]" --no-cache-dir
-          docker build --build-arg PIP_TAG="[devel]" -f Dockerfiles/pip.Dockerfile -t jinaai/jina:test-pip .
-      - name: Run tests
-        run: |
-          export JINA_LOG_VERBOSITY="ERROR"
-          pytest --force-flaky --min-passes 1 --max-runs 5 --cov=jina --cov-report=xml -n 1 --timeout=120 -v tests/jinad/${{ matrix.test-path }}
-        timeout-minutes: 40
-        env:
-          JINAHUB_USERNAME: ${{ secrets.JINAHUB_USERNAME }}
-          JINAHUB_PASSWORD: ${{ secrets.JINAHUB_PASSWORD }}
-      - name: Upload coverage from test to Codecov
-        uses: codecov/codecov-action@v1
-        if: ${{ matrix.python-version }} == 3.7
-        with:
-          file: coverage.xml
-          name: jinad-${{ matrix.python-version }}-${{ matrix.test-path }}-codecov
-          flags: ${{ matrix.test-path }}, jinad, ci
-          fail_ci_if_error: false
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 47533b6ded95f..4fc57f91fefe0 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -71,7 +71,6 @@ jobs:
           # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
           flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics --exclude .git,__pycache__,docs/source/conf.py,old,build,dist,tests/,jina/hub/
 
-
   prep-testbed:
     needs: lint-flake-8
     runs-on: ubuntu-latest
@@ -95,7 +94,7 @@ jobs:
       - id: set-matrix
         run: |
           sudo apt-get install jq
-          echo "::set-output name=matrix::$(bash scripts/get-test-paths.sh)"
+          echo "::set-output name=matrix::$(bash scripts/get-all-test-paths.sh)"
     outputs:
       matrix: ${{ steps.set-matrix.outputs.matrix }}
 
@@ -131,14 +130,14 @@ jobs:
           docker pull docker.pkg.github.com/jina-ai/jina/jina:test-pip
           docker tag docker.pkg.github.com/jina-ai/jina/jina:test-pip jinaai/jina:test-pip
           python -m pip install --upgrade pip
-          pip install ".[cicd,test]" --no-cache-dir
+          pip install ".[cicd,test,daemon]" --no-cache-dir
           jina check
           export JINA_LOG_VERBOSITY="ERROR"
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       - name: Test
         run: |
-          pytest --suppress-no-test-exit-code --force-flaky --min-passes 1 --max-runs 5 --cov=jina --cov-report=xml -n 1 --timeout=120 -v --ignore-glob='tests/integration/hub_usage/dummyhub*' tests/${{ matrix.test-path }}
+          pytest --suppress-no-test-exit-code --force-flaky --min-passes 1 --max-runs 5 --cov=jina --cov-report=xml -n 1 --timeout=120 -v --ignore-glob='tests/integration/hub_usage/dummyhub*' ${{ matrix.test-path }}
         timeout-minutes: 20
         env:
           JINAHUB_USERNAME: ${{ secrets.JINAHUB_USERNAME }}
@@ -155,5 +154,11 @@ jobs:
         with:
           file: coverage.xml
           name: ${{ matrix.test-path }}-codecov
-          flags: core, ci
           fail_ci_if_error: false
+
+  # just for blocking the merge until all parallel core-test are successful
+  success-all-test:
+    needs: [core-test, docker-image-test]
+    runs-on: ubuntu-latest
+    steps:
+      - run: echo "All Done"
diff --git a/cli/autocomplete.py b/cli/autocomplete.py
index 9509e3c9257a9..d8fba34735269 100644
--- a/cli/autocomplete.py
+++ b/cli/autocomplete.py
@@ -34,10 +34,9 @@ def _gaa(key, parser):
 ac_table = {
     'commands': ['--help', '--version', '--version-full', 'hello-world', 'pod', 'flow', 'gateway', 'ping', 'check',
                  'hub', 'pea', 'log', 'client', 'export-api'], 'completions': {
-        'hello-world': ['--help', '--workdir', '--logserver', '--logserver-config', '--download-proxy', '--shards',
-                        '--parallel', '--uses-index', '--index-data-url', '--index-labels-url', '--index-batch-size',
-                        '--uses-query', '--query-data-url', '--query-labels-url', '--query-batch-size', '--num-query',
-                        '--top-k'],
+        'hello-world': ['--help', '--workdir', '--download-proxy', '--shards', '--parallel', '--uses-index',
+                        '--index-data-url', '--index-labels-url', '--index-batch-size', '--uses-query',
+                        '--query-data-url', '--query-labels-url', '--query-batch-size', '--num-query', '--top-k'],
         'pod': ['--help', '--name', '--log-config', '--identity', '--log-id', '--show-exc-info', '--port-ctrl',
                 '--ctrl-with-ipc', '--timeout-ctrl', '--ssh-server', '--ssh-keyfile', '--ssh-password', '--uses',
                 '--py-modules', '--port-in', '--port-out', '--host-in', '--host-out', '--socket-in', '--socket-out',
@@ -47,8 +46,7 @@ def _gaa(key, parser):
                 '--timeout-ready', '--env', '--expose-public', '--pea-id', '--pea-role', '--uses-before',
                 '--uses-after', '--parallel', '--shards', '--polling', '--scheduling', '--remote-access',
                 '--remote-type', '--pod-role'],
-        'flow': ['--help', '--uses', '--logserver', '--logserver-config', '--log-id', '--optimize-level',
-                 '--output-type', '--output-path', '--inspect'],
+        'flow': ['--help', '--uses', '--log-id', '--optimize-level', '--output-type', '--output-path', '--inspect'],
         'gateway': ['--help', '--name', '--log-config', '--identity', '--log-id', '--show-exc-info', '--port-ctrl',
                     '--ctrl-with-ipc', '--timeout-ctrl', '--ssh-server', '--ssh-keyfile', '--ssh-password', '--uses',
                     '--py-modules', '--port-in', '--port-out', '--host-in', '--host-out', '--socket-in', '--socket-out',
diff --git a/extra-requirements.txt b/extra-requirements.txt
index 6af809ce80739..16c8e2a8b3ca0 100644
--- a/extra-requirements.txt
+++ b/extra-requirements.txt
@@ -17,8 +17,6 @@
 # 4. Tag `all` is reserved for representing all packages
 
 scipy>=1.4.1:               index, numeric, cicd
-flask:                      devel, cicd, http, sse
-flask_cors:                 devel, cicd, http, sse
 fastapi:                    devel, cicd, http, test, daemon
 uvicorn>=0.12.1:            devel, cicd, http, test, daemon
 fluent-logger:              logging, http, sse, dashboard, devel, cicd, test, daemon
diff --git a/jina/__init__.py b/jina/__init__.py
index 34957d4759608..990e620390b14 100644
--- a/jina/__init__.py
+++ b/jina/__init__.py
@@ -100,7 +100,6 @@
 # ADD GLOBAL NAMESPACE VARIABLES
 
 JINA_GLOBAL = _types.SimpleNamespace()
-JINA_GLOBAL.logserver = _types.SimpleNamespace()
 
 import jina.importer as _ji
 
diff --git a/jina/enums.py b/jina/enums.py
index abfa79b16dc02..8e946c226d4fb 100644
--- a/jina/enums.py
+++ b/jina/enums.py
@@ -9,7 +9,6 @@
 
     !Flow
     with:
-      logserver_config: yaml/test-server-config.yml
       optimize_level: !FlowOptimizeLevel IGNORE_GATEWAY
       # or
       optimize_level: IGNORE_GATEWAY
diff --git a/jina/flow/base.py b/jina/flow/base.py
index 7a67b869a7b46..2938d839efb34 100644
--- a/jina/flow/base.py
+++ b/jina/flow/base.py
@@ -6,14 +6,12 @@
 import copy
 import os
 import threading
-import time
 from collections import OrderedDict, defaultdict
 from contextlib import ExitStack
 from typing import Optional, Union, Tuple, List, Set, Dict, TextIO, TypeVar
 from urllib.request import Request, urlopen
 
 from .builder import build_required, _build_flow, _optimize_flow, _hanging_pods
-from .. import JINA_GLOBAL
 from ..clients import Client, WebSocketClient
 from ..enums import FlowBuildLevel, PodRoleType, FlowInspectType
 from ..excepts import FlowTopologyError, FlowMissingPodError
@@ -21,7 +19,6 @@
     get_public_ip, get_internal_ip, typename, ArgNamespace
 from ..jaml import JAML, JAMLCompatible
 from ..logging import JinaLogger
-from ..logging.sse import start_sse_logger
 from ..parsers import set_client_cli_parser, set_gateway_parser, set_pod_parser
 
 __all__ = ['BaseFlow', 'FlowLike']
@@ -390,8 +387,6 @@ def __enter__(self):
 
     def __exit__(self, exc_type, exc_val, exc_tb):
         super().__exit__(exc_type, exc_val, exc_tb)
-        if self.args.logserver:
-            self._stop_log_server()
 
         # unset all envs to avoid any side-effect
         if self._env:
@@ -403,40 +398,6 @@ def __exit__(self, exc_type, exc_val, exc_tb):
             f'flow is closed and all resources should be released already, current build level is {self._build_level}')
         self.logger.close()
 
-    def _stop_log_server(self):
-        import urllib.request
-        try:
-            # it may have been shutdown from the outside
-            urllib.request.urlopen(JINA_GLOBAL.logserver.shutdown, timeout=5)
-        except Exception as ex:
-            self.logger.info(f'Failed to connect to shutdown log sse server: {ex!r}')
-
-    def _start_log_server(self):
-        try:
-            import urllib.request
-            import flask, flask_cors
-            try:
-                with open(self.args.logserver_config) as fp:
-                    log_config = JAML.load(fp)
-                self._sse_logger = threading.Thread(name='sentinel-sse-logger',
-                                                    target=start_sse_logger, daemon=True,
-                                                    args=(log_config,
-                                                          self.args.log_id,
-                                                          self.yaml_spec))
-                self._sse_logger.start()
-                time.sleep(1)
-                response = urllib.request.urlopen(JINA_GLOBAL.logserver.ready, timeout=5)
-                if response.status == 200:
-                    self.logger.success(f'logserver is started and available at {JINA_GLOBAL.logserver.address}')
-            except Exception as ex:
-                self.logger.error(f'Could not start logserver because of {ex!r}')
-        except ModuleNotFoundError:
-            self.logger.error(
-                f'sse logserver can not start because of "flask" and "flask_cors" are missing, '
-                f'use pip install "jina[http]" (with double quotes) to install the dependencies')
-        except Exception as ex:
-            self.logger.error(f'logserver fails to start: {ex!r}')
-
     def start(self):
         """Start to run all Pods in this Flow.
 
@@ -449,10 +410,6 @@ def start(self):
         if self._build_level.value < FlowBuildLevel.GRAPH.value:
             self.build(copy_flow=False)
 
-        if self.args.logserver:
-            self.logger.info('starting logserver...')
-            self._start_log_server()
-
         # set env only before the pod get started
         if self._env:
             for k, v in self._env.items():
diff --git a/jina/helloworld/__init__.py b/jina/helloworld/__init__.py
index 3279d0dcb8ef0..ee35bbdbbebb5 100644
--- a/jina/helloworld/__init__.py
+++ b/jina/helloworld/__init__.py
@@ -51,7 +51,6 @@ def hello_world(args):
     os.environ['SHARDS'] = str(args.shards)
     os.environ['PARALLEL'] = str(args.parallel)
     os.environ['HW_WORKDIR'] = args.workdir
-    os.environ['WITH_LOGSERVER'] = str(args.logserver)
 
     # reduce the network load by using `fp16`, or even `uint8`
     os.environ['JINA_ARRAY_QUANT'] = 'fp16'
diff --git a/jina/logging/sse.py b/jina/logging/sse.py
deleted file mode 100644
index 923a5febf9991..0000000000000
--- a/jina/logging/sse.py
+++ /dev/null
@@ -1,122 +0,0 @@
-__copyright__ = "Copyright (c) 2020 Jina AI Limited. All rights reserved."
-__license__ = "Apache-2.0"
-
-import time
-import warnings
-from typing import Optional, Dict
-
-from . import default_logger
-from .. import JINA_GLOBAL, __version__
-from ..importer import ImportExtensions
-
-
-def start_sse_logger(log_config: Dict,
-                     log_id: str,
-                     flow_yaml: Optional[str] = None):
-    """Start a logger that emits server-side event from the log queue, so that one can use a browser to monitor the logs
-
-    :param log_config: configuration of the sse server
-    :param log_id: log-id of `Flow`
-    :param flow_yaml: Flow yaml description
-
-    Example:
-
-    .. highlight:: javascript
-    .. code-block:: javascript
-
-        var stream = new EventSource('http://localhost:5000/log/stream');
-        stream.onmessage = function (e) {
-            console.info(e.data);
-        };
-        stream.onerror = function (err) {
-            console.error("EventSource failed:", err);
-            stream.close()
-        };
-
-    """
-
-    warnings.warn('this function will be removed soon. dashboard will read from "jinad"', DeprecationWarning)
-
-    with ImportExtensions(required=True):
-        from flask import Flask, Response, jsonify
-        from flask_cors import CORS
-        from gevent.pywsgi import WSGIServer
-        import gevent
-
-    JINA_GLOBAL.logserver.address = f'http://{log_config["host"]}:{log_config["port"]}'
-
-    JINA_GLOBAL.logserver.ready = JINA_GLOBAL.logserver.address + \
-                                  log_config['endpoints']['ready']
-    JINA_GLOBAL.logserver.shutdown = JINA_GLOBAL.logserver.address + \
-                                     log_config['endpoints']['shutdown']
-
-    app = Flask(__name__)
-    CORS(app)
-    server = WSGIServer((log_config['host'], log_config['port']), app, log=None)
-
-    def _log_stream(path):
-        import glob
-        # fluentd creates files under this path with some tag based on day, so as temp solution,
-        # just get the first file matching this patter once it appears
-        file = f'{path}/{log_id}/log.log'
-        with open(file) as fp:
-            fp.seek(0, 2)
-            while True:
-                readline = fp.readline()
-                line = readline.strip()
-                if line:
-                    yield f'data: {line}\n\n'
-                else:
-                    time.sleep(0.1)
-
-    @app.route(log_config['endpoints']['log'])
-    def get_log():
-        """Get the logs, endpoint `/log/stream`  """
-        return Response(_log_stream(log_config['files']['log']), mimetype="text/event-stream")
-
-    @app.route(log_config['endpoints']['yaml'])
-    def get_yaml():
-        """Get the yaml of the flow  """
-        return flow_yaml
-
-    @app.route(log_config['endpoints']['profile'])
-    def get_profile():
-        """Get the profile logs, endpoint `/profile/stream`  """
-        return Response(_log_stream(log_config['files']['profile']), mimetype='text/event-stream')
-
-    @app.route(log_config['endpoints']['podapi'])
-    def get_podargs():
-        """Get the default args of a pod"""
-
-        from ..parsers import set_pod_parser
-        from argparse import _StoreAction, _StoreTrueAction
-        port_attr = ('help', 'choices', 'default')
-        d = {}
-        parser = set_pod_parser()
-        for a in parser._actions:
-            if isinstance(a, _StoreAction) or isinstance(a, _StoreTrueAction):
-                d[a.dest] = {p: getattr(a, p) for p in port_attr}
-                if a.type:
-                    d[a.dest]['type'] = a.type.__name__
-                elif isinstance(a, _StoreTrueAction):
-                    d[a.dest]['type'] = 'bool'
-                else:
-                    d[a.dest]['type'] = a.type
-
-        d = {'pod': d, 'version': __version__, 'usage': parser.format_help()}
-        return jsonify(d)
-
-    @app.route(log_config['endpoints']['shutdown'])
-    def shutdown():
-        server.stop()
-        return 'Server shutting down...'
-
-    @app.route(log_config['endpoints']['ready'])
-    def is_ready():
-        return Response(status=200)
-
-    try:
-        server.serve_forever()
-        gevent.get_hub().join()
-    except Exception as ex:
-        default_logger.error(ex)
diff --git a/jina/parsers/flow.py b/jina/parsers/flow.py
index bf2d9602a2428..4c011a20470fc 100644
--- a/jina/parsers/flow.py
+++ b/jina/parsers/flow.py
@@ -13,13 +13,6 @@ def set_flow_parser(parser=None):
         parser = set_base_parser()
 
     parser.add_argument('--uses', type=str, help='a yaml file represents a flow')
-
-    parser.add_argument('--logserver', action='store_true', default=False,
-                        help='start a log server for the dashboard')
-    parser.add_argument('--logserver-config', type=str,
-                        default=resource_filename('jina',
-                                                  '/'.join(('resources', 'logserver.default.yml'))),
-                        help='the yaml config of the log server')
     parser.add_argument('--log-id', type=str, default=get_random_identity(),
                         help='the log id used to aggregate logs by fluentd' if _SHOW_ALL_ARGS else argparse.SUPPRESS)
     parser.add_argument('--optimize-level', type=FlowOptimizeLevel.from_string, default=FlowOptimizeLevel.NONE,
diff --git a/jina/parsers/helloworld.py b/jina/parsers/helloworld.py
index f70b3e78a726f..192a3a2640b35 100644
--- a/jina/parsers/helloworld.py
+++ b/jina/parsers/helloworld.py
@@ -13,12 +13,6 @@ def set_hw_parser(parser=None):
     gp.add_argument('--workdir', type=str, default=get_random_identity(),
                     help='the workdir for hello-world demo, '
                          'all data, indices, shards and outputs will be saved there')
-    gp.add_argument('--logserver', action='store_true', default=False,
-                    help='start a log server for the dashboard')
-    gp.add_argument('--logserver-config', type=str,
-                    default=resource_filename('jina',
-                                              '/'.join(('resources', 'logserver.default.yml'))),
-                    help='the yaml config of the log server')
     gp.add_argument('--download-proxy', type=str,
                     help='specify the proxy when downloading sample data')
 
diff --git a/jina/resources/logserver.default.yml b/jina/resources/logserver.default.yml
deleted file mode 100644
index 2349e5fc49a21..0000000000000
--- a/jina/resources/logserver.default.yml
+++ /dev/null
@@ -1,12 +0,0 @@
-host: 0.0.0.0
-port: 5000
-endpoints:
-  log: /stream/log  # fetching log in SSE stream
-  profile: /stream/profile  # fetching profiling log in SSE stream
-  yaml: /data/yaml  # get the YAML spec of a flow
-  shutdown: /action/shutdown  # shutdown the log server
-  ready: /status/ready  # tell if the log server is ready, return 200 if yes
-  podapi: /data/api/pod  # get the api spec of the pod
-files:
-  log: /tmp/jina-log  # the file path of the log
-  profile: /tmp/jina-profile  # the file path of the profile log
diff --git a/scripts/get-all-test-paths.sh b/scripts/get-all-test-paths.sh
new file mode 100644
index 0000000000000..ad7aebaf0c277
--- /dev/null
+++ b/scripts/get-all-test-paths.sh
@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+
+set -ex
+
+declare -a array1=( "tests/unit/*.py" "tests/integration/*.py" )
+declare -a array2=( $(ls -d tests/{unit,integration}/*/ | grep -v '__pycache__' ))
+dest1=( "${array1[@]}" "${array2[@]}" )
+
+declare -a array1=( "tests/jinad/unit/*.py" "tests/jinad/integration/*.py" )
+declare -a array2=( $(ls -d tests/jinad/{unit,integration}/*/ | grep -v '__pycache__' ))
+dest2=( "${array1[@]}" "${array2[@]}" )
+
+dest=( "${dest1[@]}" "${dest2[@]}" )
+
+printf '%s\n' "${dest[@]}" | jq -R . | jq -cs .
\ No newline at end of file
diff --git a/scripts/get-test-paths.sh b/scripts/get-test-paths.sh
index e896ec34cade1..15ff68c6250db 100644
--- a/scripts/get-test-paths.sh
+++ b/scripts/get-test-paths.sh
@@ -2,7 +2,7 @@
 
 set -ex
 
-declare -a array1=( "unit/*.py" "integration/*.py" )
-declare -a array2=( $(ls -d tests/{unit,integration}/*/ | cut -f2,3 -d'/' | grep -v '__pycache__' ))
+declare -a array1=( "${1}/unit/*.py" "${1}/integration/*.py" )
+declare -a array2=( $(ls -d ${1}/{unit,integration}/*/ | grep -v '__pycache__' ))
 dest=( "${array1[@]}" "${array2[@]}" )
 printf '%s\n' "${dest[@]}" | jq -R . | jq -cs .
\ No newline at end of file
diff --git a/tests/integration/encode_driver_batching/test_encode_driver_batching.py b/tests/integration/encode_driver_batching/test_encode_driver_batching.py
index b3042e0643fbc..391f9c86cee51 100644
--- a/tests/integration/encode_driver_batching/test_encode_driver_batching.py
+++ b/tests/integration/encode_driver_batching/test_encode_driver_batching.py
@@ -82,10 +82,10 @@ def document_generator(num_docs, num_chunks, num_chunks_chunks):
         yield doc
 
 
-@pytest.mark.parametrize('request_batch_size', [5, 100, 500])
+@pytest.mark.parametrize('request_batch_size', [8, 16, 32])
 @pytest.mark.parametrize('driver_batch_size', [8, 16, 64])
 def test_encode_driver_batching(request_batch_size, driver_batch_size, tmpdir):
-    num_docs = 1315
+    num_docs = 128
     num_chunks = 0
     num_chunks_chunks = 0
 
@@ -122,13 +122,13 @@ def fail_if_error(resp):
                  on_error=fail_if_error)
 
 
-@pytest.mark.parametrize('request_batch_size', [5, 100, 500])
-@pytest.mark.parametrize('driver_batch_size', [8, 64, 128])
+@pytest.mark.parametrize('request_batch_size', [8, 16, 32])
+@pytest.mark.parametrize('driver_batch_size', [8, 32, 64])
 @pytest.mark.parametrize('num_chunks', [2, 8])
 @pytest.mark.parametrize('num_chunks_chunks', [2, 8])
 def test_encode_driver_batching_with_chunks(request_batch_size, driver_batch_size, num_chunks, num_chunks_chunks,
                                             tmpdir):
-    num_docs = 1315
+    num_docs = 128
     num_requests = int(num_docs / request_batch_size)
     num_docs_last_req_batch = num_docs % (num_requests * request_batch_size)
 
diff --git a/tests/integration/issues/github_936/test_flow_with_sse_no_deadlock.py b/tests/integration/issues/github_936/test_flow_with_sse_no_deadlock.py
deleted file mode 100644
index e55353cde9443..0000000000000
--- a/tests/integration/issues/github_936/test_flow_with_sse_no_deadlock.py
+++ /dev/null
@@ -1,94 +0,0 @@
-import os
-
-import pytest
-
-from jina.flow import Flow
-from jina.parsers import set_pod_parser
-from jina.peapods import Pod
-
-"""
-Github issue: https://github.com/jina-ai/jina/issues/936
-PR solves: https://github.com/jina-ai/jina/issues/935
-
-With a complex Flow structure as in this test, stopping the log server was not properly handled and
-the Flow could not properly finish. It resulted in a deadlock.
-"""
-
-
-@pytest.mark.timeout(60)
-@pytest.mark.repeat(10)
-def test_flow_with_sse_no_deadlock():
-    f = Flow(logserver=True). \
-        add(uses='BaseExecutor', parallel=2, name='crafter').add(uses='BaseExecutor', parallel=2, name='encoder'). \
-        add(uses='BaseExecutor', parallel=2, name='vec_idx').add(uses='BaseExecutor', parallel=2, needs=['gateway'],
-                                                                 name='doc_idx'). \
-        add(needs=['vec_idx', 'doc_idx'])
-    with f:
-        assert hasattr(f, '_sse_logger')
-        pass
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_flow_with_sse_no_deadlock_one_pod():
-    f = Flow(logserver=True). \
-        add(uses='BaseExecutor', parallel=1, name='crafter')
-    with f:
-        assert hasattr(f, '_sse_logger')
-        pass
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_pod_without_sse_no_parallelism_no_deadlock():
-    args = set_pod_parser().parse_args(['--parallel', '1'])
-    p = Pod(args)
-    with p:
-        pass
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_pod_with_sse_no_parallelism_no_deadlock():
-    args = set_pod_parser().parse_args(['--parallel', '1'])
-    p = Pod(args)
-    with p:
-        pass
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_pod_with_sse_no_deadlock_parallelism():
-    args = set_pod_parser().parse_args(['--parallel', '2'])
-    p = Pod(args)
-    with p:
-        pass
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_pod_with_sse_no_deadlock_log_remote():
-    args = set_pod_parser().parse_args(['--parallel', '2'])
-    p = Pod(args)
-    with p:
-        pass
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_pod_with_sse_no_deadlock_log_file():
-    os.environ['JINA_LOG_FILE'] = 'TXT'
-    args = set_pod_parser().parse_args(['--parallel', '2'])
-    p = Pod(args)
-    with p:
-        pass
-    del os.environ['JINA_LOG_FILE']
-
-
-@pytest.mark.timeout(10)
-@pytest.mark.repeat(10)
-def test_pod_with_sse_no_deadlock_thread():
-    args = set_pod_parser().parse_args(['--parallel', '2', '--runtime', 'thread'])
-    p = Pod(args)
-    with p:
-        pass
diff --git a/tests/integration/logging/test_sse.py b/tests/integration/logging/test_sse.py
deleted file mode 100644
index 600db319d294d..0000000000000
--- a/tests/integration/logging/test_sse.py
+++ /dev/null
@@ -1,91 +0,0 @@
-import os
-import time
-import threading
-
-from jina.helper import random_port
-from jina.jaml import JAML
-from jina.logging.sse import start_sse_logger
-
-cur_dir = os.path.dirname(os.path.abspath(__file__))
-GROUP_ID = 'flow_id'
-LOG_MESSAGE = 'log message'
-
-RANDOM_PORT = random_port()
-
-def feed_path_logs(path, threading_event):
-    while True:
-        os.makedirs(path, exist_ok=True)
-        file = os.path.join(path, 'log.log')
-        if not threading_event.is_set():
-            with open(file, 'a') as fp:
-                fp.write(LOG_MESSAGE)
-                fp.flush()
-                time.sleep(0.1)
-        else:
-            break
-
-
-def sse_client(wrap):
-    from sseclient import SSEClient
-    def with_requests(url):
-        """Get a streaming response for the given event feed using requests."""
-        import requests
-        return requests.get(url, stream=True)
-
-    url = f'http://0.0.0.0:{RANDOM_PORT}/stream/log'
-    response = with_requests(url)
-    client = SSEClient(response)
-    events = client.events()
-    event = next(events)
-    if LOG_MESSAGE in event.data:
-        wrap.count += 1
-
-
-def stop_log_server():
-    import urllib.request
-    urllib.request.urlopen(f'http://0.0.0.0:{RANDOM_PORT}/action/shutdown', timeout=5)
-
-
-def test_sse_client(tmpdir):
-    class WrapAssert:
-        def __init__(self):
-            self.count = 0
-
-    conf_path = os.path.join(tmpdir, 'log')
-    path = os.path.join(conf_path, GROUP_ID)
-    event = threading.Event()
-
-    feed_thread = threading.Thread(name='feed_path_logs',
-                                   target=feed_path_logs, daemon=False,
-                                   kwargs={'path': path, 'threading_event': event})
-
-    with open(os.path.join(cur_dir, 'logserver_config.yml')) as fp:
-        log_config = JAML.load(fp)
-        log_config['files']['log'] = conf_path
-        log_config['port'] = RANDOM_PORT
-
-    sse_server_thread = threading.Thread(name='sentinel-sse-logger',
-                                         target=start_sse_logger, daemon=False,
-                                         args=(log_config,
-                                               GROUP_ID,
-                                               None))
-
-    wrap = WrapAssert()
-    sse_client_thread = threading.Thread(name='sse-client',
-                                         target=sse_client,
-                                         daemon=False,
-                                         kwargs=({'wrap': wrap}))
-
-    feed_thread.start()
-    time.sleep(0.5)
-    sse_server_thread.start()
-    time.sleep(0.5)
-    sse_client_thread.start()
-    time.sleep(0.5)
-    event.set()
-    stop_log_server()
-    feed_thread.join()
-    sse_server_thread.join()
-    sse_client_thread.join()
-
-    assert wrap.count > 0
diff --git a/tests/integration/test_helloworld.py b/tests/integration/test_helloworld.py
index 4d56e63831f86..3c47470a9bec4 100644
--- a/tests/integration/test_helloworld.py
+++ b/tests/integration/test_helloworld.py
@@ -43,7 +43,6 @@ def test_helloworld_flow(tmpdir):
     os.environ['SHARDS'] = str(args.shards)
     os.environ['PARALLEL'] = str(args.parallel)
     os.environ['HW_WORKDIR'] = str(tmpdir)
-    os.environ['WITH_LOGSERVER'] = str(args.logserver)
 
     f = Flow.load_config(resource_filename('jina', '/'.join(('resources', 'helloworld.flow.index.yml'))))
 
@@ -74,7 +73,6 @@ def test_helloworld_flow_dry_run(tmpdir):
     os.environ['SHARDS'] = str(args.shards)
     os.environ['PARALLEL'] = str(args.parallel)
     os.environ['HW_WORKDIR'] = str(tmpdir)
-    os.environ['WITH_LOGSERVER'] = str(args.logserver)
 
     # run it!
     with Flow.load_config(resource_filename('jina', '/'.join(('resources', 'helloworld.flow.index.yml')))):
diff --git a/tests/jinad/integration/distributed/test_simple_hub_pods/__init__.py b/tests/jinad/integration/distributed/test_simple_hub_pods/__init__.py
deleted file mode 100644
index e69de29bb2d1d..0000000000000
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/__init__.py b/tests/jinad/integration/distributed/test_simple_local_remote/__init__.py
deleted file mode 100644
index e69de29bb2d1d..0000000000000
diff --git a/tests/jinad/integration/distributed/helpers.py b/tests/jinad/integration/helpers.py
similarity index 100%
rename from tests/jinad/integration/distributed/helpers.py
rename to tests/jinad/integration/helpers.py
diff --git a/tests/jinad/integration/distributed/hub_mwu/Dockerfile b/tests/jinad/integration/hub_mwu/Dockerfile
similarity index 100%
rename from tests/jinad/integration/distributed/hub_mwu/Dockerfile
rename to tests/jinad/integration/hub_mwu/Dockerfile
diff --git a/tests/jinad/integration/distributed/hub_mwu/README.md b/tests/jinad/integration/hub_mwu/README.md
similarity index 100%
rename from tests/jinad/integration/distributed/hub_mwu/README.md
rename to tests/jinad/integration/hub_mwu/README.md
diff --git a/tests/integration/issues/github_936/__init__.py b/tests/jinad/integration/hub_mwu/__init__.py
similarity index 100%
rename from tests/integration/issues/github_936/__init__.py
rename to tests/jinad/integration/hub_mwu/__init__.py
diff --git a/tests/jinad/integration/distributed/hub_mwu/manifest.yml b/tests/jinad/integration/hub_mwu/manifest.yml
similarity index 100%
rename from tests/jinad/integration/distributed/hub_mwu/manifest.yml
rename to tests/jinad/integration/hub_mwu/manifest.yml
diff --git a/tests/jinad/integration/distributed/hub_mwu/mwu_encoder.py b/tests/jinad/integration/hub_mwu/mwu_encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/hub_mwu/mwu_encoder.py
rename to tests/jinad/integration/hub_mwu/mwu_encoder.py
diff --git a/tests/jinad/integration/distributed/hub_mwu/mwu_encoder.yml b/tests/jinad/integration/hub_mwu/mwu_encoder.yml
similarity index 100%
rename from tests/jinad/integration/distributed/hub_mwu/mwu_encoder.yml
rename to tests/jinad/integration/hub_mwu/mwu_encoder.yml
diff --git a/tests/jinad/integration/distributed/test_index_query/.env b/tests/jinad/integration/test_index_query/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/.env
rename to tests/jinad/integration/test_index_query/.env
diff --git a/tests/jinad/integration/distributed/__init__.py b/tests/jinad/integration/test_index_query/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/__init__.py
rename to tests/jinad/integration/test_index_query/__init__.py
diff --git a/tests/jinad/integration/distributed/test_index_query/docker-compose.yml b/tests/jinad/integration/test_index_query/docker-compose.yml
similarity index 68%
rename from tests/jinad/integration/distributed/test_index_query/docker-compose.yml
rename to tests/jinad/integration/test_index_query/docker-compose.yml
index aa9d40b1715ac..ee7e89cba23b5 100644
--- a/tests/jinad/integration/distributed/test_index_query/docker-compose.yml
+++ b/tests/jinad/integration/test_index_query/docker-compose.yml
@@ -9,7 +9,7 @@ services:
       - "8000:8000"
       - "45678:45678"
     env_file:
-      - tests/jinad/integration/distributed/test_index_query/.env
+      - tests/jinad/integration/test_index_query/.env
     expose:
       - 10000-60000
   encoder:
@@ -18,11 +18,11 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_index_query/.env
+      - tests/jinad/integration/test_index_query/.env
   indexer:
     image: test_index_query
     expose:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_index_query/.env
+      - tests/jinad/integration/test_index_query/.env
diff --git a/tests/jinad/integration/distributed/test_index_query/flow.yml b/tests/jinad/integration/test_index_query/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/flow.yml
rename to tests/jinad/integration/test_index_query/flow.yml
diff --git a/tests/jinad/integration/distributed/hub_mwu/__init__.py b/tests/jinad/integration/test_index_query/pods/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/hub_mwu/__init__.py
rename to tests/jinad/integration/test_index_query/pods/__init__.py
diff --git a/tests/jinad/integration/distributed/test_index_query/pods/dummy-encoder.py b/tests/jinad/integration/test_index_query/pods/dummy-encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/pods/dummy-encoder.py
rename to tests/jinad/integration/test_index_query/pods/dummy-encoder.py
diff --git a/tests/jinad/integration/distributed/test_index_query/pods/encode.yml b/tests/jinad/integration/test_index_query/pods/encode.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/pods/encode.yml
rename to tests/jinad/integration/test_index_query/pods/encode.yml
diff --git a/tests/jinad/integration/distributed/test_index_query/pods/index.yml b/tests/jinad/integration/test_index_query/pods/index.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/pods/index.yml
rename to tests/jinad/integration/test_index_query/pods/index.yml
diff --git a/tests/jinad/integration/distributed/test_index_query/test_integration.py b/tests/jinad/integration/test_index_query/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/test_integration.py
rename to tests/jinad/integration/test_index_query/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/.env b/tests/jinad/integration/test_index_query_with_shards/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/.env
rename to tests/jinad/integration/test_index_query_with_shards/.env
diff --git a/tests/jinad/integration/distributed/test_index_query/__init__.py b/tests/jinad/integration/test_index_query_with_shards/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/__init__.py
rename to tests/jinad/integration/test_index_query_with_shards/__init__.py
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/docker-compose.yml b/tests/jinad/integration/test_index_query_with_shards/docker-compose.yml
similarity index 66%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/docker-compose.yml
rename to tests/jinad/integration/test_index_query_with_shards/docker-compose.yml
index 1de306162d9f6..1ae12b22b6941 100644
--- a/tests/jinad/integration/distributed/test_index_query_with_shards/docker-compose.yml
+++ b/tests/jinad/integration/test_index_query_with_shards/docker-compose.yml
@@ -9,7 +9,7 @@ services:
       - "8000:8000"
       - "45678:45678"
     env_file:
-      - tests/jinad/integration/distributed/test_index_query_with_shards/.env
+      - tests/jinad/integration/test_index_query_with_shards/.env
     expose:
       - 10000-60000
   encoder:
@@ -18,11 +18,11 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_index_query_with_shards/.env
+      - tests/jinad/integration/test_index_query_with_shards/.env
   indexer:
     image: test_index_query_with_shards
     expose:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_index_query_with_shards/.env
+      - tests/jinad/integration/test_index_query_with_shards/.env
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/flow.yml b/tests/jinad/integration/test_index_query_with_shards/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/flow.yml
rename to tests/jinad/integration/test_index_query_with_shards/flow.yml
diff --git a/tests/jinad/integration/distributed/test_index_query/pods/__init__.py b/tests/jinad/integration/test_index_query_with_shards/pods/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query/pods/__init__.py
rename to tests/jinad/integration/test_index_query_with_shards/pods/__init__.py
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/pods/dummy-encoder.py b/tests/jinad/integration/test_index_query_with_shards/pods/dummy-encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/pods/dummy-encoder.py
rename to tests/jinad/integration/test_index_query_with_shards/pods/dummy-encoder.py
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/pods/encode.yml b/tests/jinad/integration/test_index_query_with_shards/pods/encode.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/pods/encode.yml
rename to tests/jinad/integration/test_index_query_with_shards/pods/encode.yml
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/pods/index.yml b/tests/jinad/integration/test_index_query_with_shards/pods/index.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/pods/index.yml
rename to tests/jinad/integration/test_index_query_with_shards/pods/index.yml
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/pods/slice.yml b/tests/jinad/integration/test_index_query_with_shards/pods/slice.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/pods/slice.yml
rename to tests/jinad/integration/test_index_query_with_shards/pods/slice.yml
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/test_integration.py b/tests/jinad/integration/test_index_query_with_shards/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/test_integration.py
rename to tests/jinad/integration/test_index_query_with_shards/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/.env b/tests/jinad/integration/test_local_flow_local_remote_local/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/.env
rename to tests/jinad/integration/test_local_flow_local_remote_local/.env
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/__init__.py b/tests/jinad/integration/test_local_flow_local_remote_local/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/__init__.py
rename to tests/jinad/integration/test_local_flow_local_remote_local/__init__.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/craft.yml b/tests/jinad/integration/test_local_flow_local_remote_local/craft.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/craft.yml
rename to tests/jinad/integration/test_local_flow_local_remote_local/craft.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/docker-compose.yml b/tests/jinad/integration/test_local_flow_local_remote_local/docker-compose.yml
similarity index 87%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/docker-compose.yml
rename to tests/jinad/integration/test_local_flow_local_remote_local/docker-compose.yml
index e986338954350..f0be812c2ef55 100644
--- a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/docker-compose.yml
+++ b/tests/jinad/integration/test_local_flow_local_remote_local/docker-compose.yml
@@ -15,7 +15,7 @@ services:
       - "45005:45005"
       - "45006:45006"
     env_file:
-      - tests/jinad/integration/distributed/test_local_flow_remote_local_remote/.env
+      - tests/jinad/integration/test_local_flow_remote_local_remote/.env
     expose:
       - 10000-60000
     networks:
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/dummy-crafter.py b/tests/jinad/integration/test_local_flow_local_remote_local/dummy-crafter.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/dummy-crafter.py
rename to tests/jinad/integration/test_local_flow_local_remote_local/dummy-crafter.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/dummy-encoder.py b/tests/jinad/integration/test_local_flow_local_remote_local/dummy-encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/dummy-encoder.py
rename to tests/jinad/integration/test_local_flow_local_remote_local/dummy-encoder.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/encode.yml b/tests/jinad/integration/test_local_flow_local_remote_local/encode.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/encode.yml
rename to tests/jinad/integration/test_local_flow_local_remote_local/encode.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/flow.yml b/tests/jinad/integration/test_local_flow_local_remote_local/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/flow.yml
rename to tests/jinad/integration/test_local_flow_local_remote_local/flow.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/index.yml b/tests/jinad/integration/test_local_flow_local_remote_local/index.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/index.yml
rename to tests/jinad/integration/test_local_flow_local_remote_local/index.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/test_integration.py b/tests/jinad/integration/test_local_flow_local_remote_local/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/test_integration.py
rename to tests/jinad/integration/test_local_flow_local_remote_local/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/.env b/tests/jinad/integration/test_local_flow_remote_local_remote/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/.env
rename to tests/jinad/integration/test_local_flow_remote_local_remote/.env
diff --git a/tests/jinad/integration/distributed/test_index_query_with_shards/pods/__init__.py b/tests/jinad/integration/test_local_flow_remote_local_remote/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_index_query_with_shards/pods/__init__.py
rename to tests/jinad/integration/test_local_flow_remote_local_remote/__init__.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/craft.yml b/tests/jinad/integration/test_local_flow_remote_local_remote/craft.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/craft.yml
rename to tests/jinad/integration/test_local_flow_remote_local_remote/craft.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/docker-compose.yml b/tests/jinad/integration/test_local_flow_remote_local_remote/docker-compose.yml
similarity index 81%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/docker-compose.yml
rename to tests/jinad/integration/test_local_flow_remote_local_remote/docker-compose.yml
index b5bce8ce3930c..12369dda8f986 100644
--- a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/docker-compose.yml
+++ b/tests/jinad/integration/test_local_flow_remote_local_remote/docker-compose.yml
@@ -15,7 +15,7 @@ services:
       - "45005:45005"
       - "45006:45006"
     env_file:
-      - tests/jinad/integration/distributed/test_local_flow_remote_local_remote/.env
+      - tests/jinad/integration/test_local_flow_remote_local_remote/.env
     expose:
       - 10000-60000
     networks:
@@ -27,7 +27,7 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_local_flow_remote_local_remote/.env
+      - tests/jinad/integration/test_local_flow_remote_local_remote/.env
     networks:
       testing_net:
         ipv4_address: 172.28.1.2
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/dummy-crafter.py b/tests/jinad/integration/test_local_flow_remote_local_remote/dummy-crafter.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/dummy-crafter.py
rename to tests/jinad/integration/test_local_flow_remote_local_remote/dummy-crafter.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/dummy-encoder.py b/tests/jinad/integration/test_local_flow_remote_local_remote/dummy-encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/dummy-encoder.py
rename to tests/jinad/integration/test_local_flow_remote_local_remote/dummy-encoder.py
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/encode.yml b/tests/jinad/integration/test_local_flow_remote_local_remote/encode.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/encode.yml
rename to tests/jinad/integration/test_local_flow_remote_local_remote/encode.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/flow.yml b/tests/jinad/integration/test_local_flow_remote_local_remote/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/flow.yml
rename to tests/jinad/integration/test_local_flow_remote_local_remote/flow.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/index.yml b/tests/jinad/integration/test_local_flow_remote_local_remote/index.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/index.yml
rename to tests/jinad/integration/test_local_flow_remote_local_remote/index.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/test_integration.py b/tests/jinad/integration/test_local_flow_remote_local_remote/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/test_integration.py
rename to tests/jinad/integration/test_local_flow_remote_local_remote/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/.env b/tests/jinad/integration/test_simple_distributed/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/.env
rename to tests/jinad/integration/test_simple_distributed/.env
diff --git a/tests/jinad/integration/distributed/test_local_flow_local_remote_local/__init__.py b/tests/jinad/integration/test_simple_distributed/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_local_remote_local/__init__.py
rename to tests/jinad/integration/test_simple_distributed/__init__.py
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/docker-compose.yml b/tests/jinad/integration/test_simple_distributed/docker-compose.yml
similarity index 67%
rename from tests/jinad/integration/distributed/test_simple_distributed/docker-compose.yml
rename to tests/jinad/integration/test_simple_distributed/docker-compose.yml
index 4c801985a56c4..16fdc60d28c22 100644
--- a/tests/jinad/integration/distributed/test_simple_distributed/docker-compose.yml
+++ b/tests/jinad/integration/test_simple_distributed/docker-compose.yml
@@ -9,7 +9,7 @@ services:
       - "8000:8000"
       - "45678:45678"
     env_file:
-      - tests/jinad/integration/distributed/test_simple_distributed/.env
+      - tests/jinad/integration/test_simple_distributed/.env
     expose:
       - 10000-60000
   encoder:
@@ -18,11 +18,11 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_distributed/.env
+      - tests/jinad/integration/test_simple_distributed/.env
   indexer:
     image: test_simple_distributed
     expose:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_distributed/.env
+      - tests/jinad/integration/test_simple_distributed/.env
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/flow.yml b/tests/jinad/integration/test_simple_distributed/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/flow.yml
rename to tests/jinad/integration/test_simple_distributed/flow.yml
diff --git a/tests/jinad/integration/distributed/test_local_flow_remote_local_remote/__init__.py b/tests/jinad/integration/test_simple_distributed/pods/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_local_flow_remote_local_remote/__init__.py
rename to tests/jinad/integration/test_simple_distributed/pods/__init__.py
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/pods/dummy-encoder.py b/tests/jinad/integration/test_simple_distributed/pods/dummy-encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/pods/dummy-encoder.py
rename to tests/jinad/integration/test_simple_distributed/pods/dummy-encoder.py
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/pods/encode.yml b/tests/jinad/integration/test_simple_distributed/pods/encode.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/pods/encode.yml
rename to tests/jinad/integration/test_simple_distributed/pods/encode.yml
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/pods/index.yml b/tests/jinad/integration/test_simple_distributed/pods/index.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/pods/index.yml
rename to tests/jinad/integration/test_simple_distributed/pods/index.yml
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/test_integration.py b/tests/jinad/integration/test_simple_distributed/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/test_integration.py
rename to tests/jinad/integration/test_simple_distributed/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_simple_distributed_with_shards/.env b/tests/jinad/integration/test_simple_distributed_with_shards/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed_with_shards/.env
rename to tests/jinad/integration/test_simple_distributed_with_shards/.env
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/__init__.py b/tests/jinad/integration/test_simple_distributed_with_shards/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/__init__.py
rename to tests/jinad/integration/test_simple_distributed_with_shards/__init__.py
diff --git a/tests/jinad/integration/distributed/test_simple_distributed_with_shards/docker-compose.yml b/tests/jinad/integration/test_simple_distributed_with_shards/docker-compose.yml
similarity index 65%
rename from tests/jinad/integration/distributed/test_simple_distributed_with_shards/docker-compose.yml
rename to tests/jinad/integration/test_simple_distributed_with_shards/docker-compose.yml
index f7831b6c7f022..bb95711ceea35 100644
--- a/tests/jinad/integration/distributed/test_simple_distributed_with_shards/docker-compose.yml
+++ b/tests/jinad/integration/test_simple_distributed_with_shards/docker-compose.yml
@@ -9,7 +9,7 @@ services:
       - "8000:8000"
       - "45678:45678"
     env_file:
-      - tests/jinad/integration/distributed/test_simple_distributed_with_shards/.env
+      - tests/jinad/integration/test_simple_distributed_with_shards/.env
     expose:
       - 10000-60000
   pod1:
@@ -18,11 +18,11 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_distributed_with_shards/.env
+      - tests/jinad/integration/test_simple_distributed_with_shards/.env
   pod2:
     image: test_simple_distributed_with_shards
     expose:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_distributed_with_shards/.env
+      - tests/jinad/integration/test_simple_distributed_with_shards/.env
diff --git a/tests/jinad/integration/distributed/test_simple_distributed_with_shards/flow.yml b/tests/jinad/integration/test_simple_distributed_with_shards/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed_with_shards/flow.yml
rename to tests/jinad/integration/test_simple_distributed_with_shards/flow.yml
diff --git a/tests/jinad/integration/distributed/test_simple_distributed_with_shards/test_integration.py b/tests/jinad/integration/test_simple_distributed_with_shards/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed_with_shards/test_integration.py
rename to tests/jinad/integration/test_simple_distributed_with_shards/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_simple_hub_pods/.env b/tests/jinad/integration/test_simple_hub_pods/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_hub_pods/.env
rename to tests/jinad/integration/test_simple_hub_pods/.env
diff --git a/tests/jinad/integration/distributed/test_simple_distributed/pods/__init__.py b/tests/jinad/integration/test_simple_hub_pods/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed/pods/__init__.py
rename to tests/jinad/integration/test_simple_hub_pods/__init__.py
diff --git a/tests/jinad/integration/distributed/test_simple_hub_pods/docker-compose.yml b/tests/jinad/integration/test_simple_hub_pods/docker-compose.yml
similarity index 74%
rename from tests/jinad/integration/distributed/test_simple_hub_pods/docker-compose.yml
rename to tests/jinad/integration/test_simple_hub_pods/docker-compose.yml
index 5dccb3cc7b8fd..f2421df9d3273 100644
--- a/tests/jinad/integration/distributed/test_simple_hub_pods/docker-compose.yml
+++ b/tests/jinad/integration/test_simple_hub_pods/docker-compose.yml
@@ -9,7 +9,7 @@ services:
       - "8000:8000"
       - "45678:45678"
     env_file:
-      - tests/jinad/integration/distributed/test_simple_hub_pods/.env
+      - tests/jinad/integration/test_simple_hub_pods/.env
     expose:
       - 10000-60000
     volumes:
@@ -20,7 +20,7 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_hub_pods/.env
+      - tests/jinad/integration/test_simple_hub_pods/.env
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
   pod2:
@@ -29,6 +29,6 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_hub_pods/.env
+      - tests/jinad/integration/test_simple_hub_pods/.env
     volumes:
       - /var/run/docker.sock:/var/run/docker.sock
diff --git a/tests/jinad/integration/distributed/test_simple_hub_pods/flow.yml b/tests/jinad/integration/test_simple_hub_pods/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_hub_pods/flow.yml
rename to tests/jinad/integration/test_simple_hub_pods/flow.yml
diff --git a/tests/jinad/integration/distributed/test_simple_hub_pods/test_integration.py b/tests/jinad/integration/test_simple_hub_pods/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_hub_pods/test_integration.py
rename to tests/jinad/integration/test_simple_hub_pods/test_integration.py
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/.env b/tests/jinad/integration/test_simple_local_remote/.env
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_local_remote/.env
rename to tests/jinad/integration/test_simple_local_remote/.env
diff --git a/tests/jinad/integration/distributed/test_simple_distributed_with_shards/__init__.py b/tests/jinad/integration/test_simple_local_remote/__init__.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_distributed_with_shards/__init__.py
rename to tests/jinad/integration/test_simple_local_remote/__init__.py
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/docker-compose.yml b/tests/jinad/integration/test_simple_local_remote/docker-compose.yml
similarity index 81%
rename from tests/jinad/integration/distributed/test_simple_local_remote/docker-compose.yml
rename to tests/jinad/integration/test_simple_local_remote/docker-compose.yml
index 3ccd8ed3ee77d..1c6036bc3c760 100644
--- a/tests/jinad/integration/distributed/test_simple_local_remote/docker-compose.yml
+++ b/tests/jinad/integration/test_simple_local_remote/docker-compose.yml
@@ -11,7 +11,7 @@ services:
       - "45001:45001"
       - "45002:45002"
     env_file:
-      - tests/jinad/integration/distributed/test_simple_local_remote/.env
+      - tests/jinad/integration/test_simple_local_remote/.env
     expose:
       - 10000-60000
     networks:
@@ -23,7 +23,7 @@ services:
       - 8000
       - 10000-60000
     env_file:
-      - tests/jinad/integration/distributed/test_simple_local_remote/.env
+      - tests/jinad/integration/test_simple_local_remote/.env
     networks:
       testing_net:
         ipv4_address: 172.28.1.2
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/dummy-encoder.py b/tests/jinad/integration/test_simple_local_remote/dummy-encoder.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_local_remote/dummy-encoder.py
rename to tests/jinad/integration/test_simple_local_remote/dummy-encoder.py
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/encode.yml b/tests/jinad/integration/test_simple_local_remote/encode.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_local_remote/encode.yml
rename to tests/jinad/integration/test_simple_local_remote/encode.yml
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/flow.yml b/tests/jinad/integration/test_simple_local_remote/flow.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_local_remote/flow.yml
rename to tests/jinad/integration/test_simple_local_remote/flow.yml
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/index.yml b/tests/jinad/integration/test_simple_local_remote/index.yml
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_local_remote/index.yml
rename to tests/jinad/integration/test_simple_local_remote/index.yml
diff --git a/tests/jinad/integration/distributed/test_simple_local_remote/test_integration.py b/tests/jinad/integration/test_simple_local_remote/test_integration.py
similarity index 100%
rename from tests/jinad/integration/distributed/test_simple_local_remote/test_integration.py
rename to tests/jinad/integration/test_simple_local_remote/test_integration.py
diff --git a/tests/unit/flow/test_flow.py b/tests/unit/flow/test_flow.py
index bf015e7fa2574..56ab40536124a 100644
--- a/tests/unit/flow/test_flow.py
+++ b/tests/unit/flow/test_flow.py
@@ -2,9 +2,6 @@
 
 import numpy as np
 import pytest
-import requests
-
-from jina import JINA_GLOBAL
 from jina.enums import SocketType
 from jina.executors import BaseExecutor
 from jina.flow import Flow
@@ -160,7 +157,6 @@ def test_flow_identical():
 
 @pytest.mark.parametrize('restful', [False, True])
 def test_flow_no_container(restful):
-
     f = (Flow(restful=restful)
          .add(name='dummyEncoder', uses=os.path.join(cur_dir, '../mwu-encoder/mwu_encoder.yml')))
 
@@ -168,53 +164,9 @@ def test_flow_no_container(restful):
         f.index(input_fn=random_docs(10))
 
 
-def test_flow_log_server():
-    f = Flow.load_config(os.path.join(cur_dir, '../yaml/test_log_server.yml'))
-    with f:
-        assert hasattr(JINA_GLOBAL.logserver, 'ready')
-
-        # Ready endpoint
-        a = requests.get(
-            JINA_GLOBAL.logserver.address +
-            '/status/ready',
-            timeout=5)
-        assert a.status_code == 200
-
-        # YAML endpoint
-        a = requests.get(
-            JINA_GLOBAL.logserver.address +
-            '/data/yaml',
-            timeout=5)
-        assert a.text.startswith('!Flow')
-        assert a.status_code == 200
-
-        # Pod endpoint
-        a = requests.get(
-            JINA_GLOBAL.logserver.address +
-            '/data/api/pod',
-            timeout=5)
-        assert 'pod' in a.json()
-        assert a.status_code == 200
-
-        # Shutdown endpoint
-        a = requests.get(
-            JINA_GLOBAL.logserver.address +
-            '/action/shutdown',
-            timeout=5)
-        assert a.status_code == 200
-
-        # Check ready endpoint after shutdown, check if server stopped
-        with pytest.raises((requests.exceptions.ConnectionError, requests.exceptions.ReadTimeout)):
-            requests.get(
-                JINA_GLOBAL.logserver.address +
-                '/status/ready',
-                timeout=5)
-
-
-@pytest.mark.parametrize('restful', [False, True])
-def test_shards(restful):
-    f = (Flow(restful=restful)
-         .add(name='doc_pb', uses=os.path.join(cur_dir, '../yaml/test-docpb.yml'), parallel=3, separated_workspace=True))
+def test_shards():
+    f = Flow().add(name='doc_pb', uses=os.path.join(cur_dir, '../yaml/test-docpb.yml'), parallel=3,
+                   separated_workspace=True)
     with f:
         f.index(input_fn=random_docs(1000), random_doc_id=False)
     with f:
diff --git a/tests/unit/flow/test_flow_with_sse.py b/tests/unit/flow/test_flow_with_sse.py
deleted file mode 100644
index 7521f592b4482..0000000000000
--- a/tests/unit/flow/test_flow_with_sse.py
+++ /dev/null
@@ -1,13 +0,0 @@
-import pytest
-
-from jina.flow import Flow
-
-
-@pytest.mark.timeout(60)
-@pytest.mark.repeat(10)
-def test_flow_with_sse():
-    f = Flow(logserver=True). \
-        add(uses='BaseExecutor', parallel=1).add(uses='BaseExecutor', parallel=2).build()
-    with f:
-        assert hasattr(f, '_sse_logger')
-        pass
diff --git a/tests/unit/flow/test_flow_yaml_parser.py b/tests/unit/flow/test_flow_yaml_parser.py
index 4929a94e8d4dc..08810db011e05 100644
--- a/tests/unit/flow/test_flow_yaml_parser.py
+++ b/tests/unit/flow/test_flow_yaml_parser.py
@@ -81,13 +81,11 @@ def test_load_flow_from_yaml():
 
 
 def test_flow_yaml_dump():
-    f = Flow(logserver_config=str(cur_dir.parent / 'yaml' / 'test-server-config.yml'),
-             optimize_level=FlowOptimizeLevel.IGNORE_GATEWAY,
+    f = Flow(optimize_level=FlowOptimizeLevel.IGNORE_GATEWAY,
              no_gateway=True)
     f.save_config('test1.yml')
 
     fl = Flow.load_config('test1.yml')
-    assert f.args.logserver_config == fl.args.logserver_config
     assert f.args.optimize_level == fl.args.optimize_level
     rm_files(['test1.yml'])
 


Code review comments:
diff_hunk: @@ -82,10 +82,10 @@ def document_generator(num_docs, num_chunks, num_chunks_chunks):
         yield doc
 
 
-@pytest.mark.parametrize('request_batch_size', [5, 100, 500])
+@pytest.mark.parametrize('request_batch_size', [8, 16, 32])
 @pytest.mark.parametrize('driver_batch_size', [8, 16, 64])
path: tests/integration/encode_driver_batching/test_encode_driver_batching.py
text: @user1:
reduce also driver batch sizes so that most of the tests the driver batch size is smaller than the request size