Repository: sqlfluff/sqlfluff

Issue:
Better name for --parallel?
I feel that --threads instead of --parallel is clearer in terms of what the input value does. We can also then change the help string to specify that the number passed is the number of threads used.

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) Report
> Merging [#1133](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) (6e513b9) into [master](https://codecov.io/gh/sqlfluff/sqlfluff/commit/35673a95698273046996b0b5096e26fd8fc52286?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) (35673a9) will **not change** coverage.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133/graphs/tree.svg?width=650&height=150&src=pr&token=maM8kLgCmn&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff)](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff)

```diff
@@           Coverage Diff           @@
##           master    #1133   +/-   ##
=======================================
  Coverage   95.98%   95.98%           
=======================================
  Files         123      123           
  Lines        8200     8200           
=======================================
  Hits         7871     7871           
  Misses        329      329           
```

| Flag | Coverage Δ | |
|---|---|---|
| dbt018-py38 | `62.10% <75.00%> (ø)` | |
| dbt019-py38 | `62.10% <75.00%> (ø)` | |
| py36 | `94.93% <100.00%> (ø)` | |
| py37 | `94.97% <100.00%> (ø)` | |
| py38 | `94.95% <100.00%> (ø)` | |
| py39 | `94.95% <100.00%> (ø)` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff) | Coverage Δ | |
|---|---|---|
| [src/sqlfluff/core/\_\_init\_\_.py](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff#diff-c3JjL3NxbGZsdWZmL2NvcmUvX19pbml0X18ucHk=) | `100.00% <ø> (ø)` | |
| [src/sqlfluff/core/linter/linter.py](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff#diff-c3JjL3NxbGZsdWZmL2NvcmUvbGludGVyL2xpbnRlci5weQ==) | `97.07% <ø> (ø)` | |
| [src/sqlfluff/cli/commands.py](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff#diff-c3JjL3NxbGZsdWZmL2NsaS9jb21tYW5kcy5weQ==) | `87.84% <100.00%> (ø)` | |
| [src/sqlfluff/core/linter/runner.py](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff#diff-c3JjL3NxbGZsdWZmL2NvcmUvbGludGVyL3J1bm5lci5weQ==) | `92.55% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff). Last update [35673a9...6e513b9](https://codecov.io/gh/sqlfluff/sqlfluff/pull/1133?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=sqlfluff).


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1d9d4c9e6..ded0327fd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -13,20 +13,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ANSI dialect which defaults to the semicolon, but allows it to be more intuitive
   when overriden in a specific child dialect (mysql) [#901](https://github.com/sqlfluff/sqlfluff/issues/901))
 - Added support for the DELIMITER statement in the mysql dialect [#901](https://github.com/sqlfluff/sqlfluff/issues/901))
+- Added support for additional delimiters by creating a new DelimiterSegment in the
+  ANSI dialect which defaults to the semicolon, but allows it to be more intuitive.
 
 ### Changed
+- Renamed --parallel CLI argument to --processes to be more accurate.
 - L034 now ignores select statements which contain macros.
 - L034 now ignores select statements part of a set expression, most commonly a union.
 - Fix bug [#1082](https://github.com/sqlfluff/sqlfluff/issues/1082), adding
-  support for BigQuery `select as struct '1' as bb, 2 as aa` syntax
+  support for BigQuery `select as struct '1' as bb, 2 as aa` syntax.
 - Fix bug [#1079](https://github.com/sqlfluff/sqlfluff/issues/1079), addressing
-  issues with L025 and L026 with BigQuery column references involving `STRUCT`
+  issues with L025 and L026 with BigQuery column references involving `STRUCT`.
 - Fix bug [#1080](https://github.com/sqlfluff/sqlfluff/issues/1080), add
   SET SCHEMA and DROP SCHEMA support to ANSI dialect.
-- Added support for additional delimiters by creating a new DelimiterSegment in the 
-  ANSI dialect which defaults to the semicolon, but allows it to be more intuitive 
   when overriden in a specific child dialect (mysql) [#901](https://github.com/sqlfluff/sqlfluff/issues/901))
-- Added support for the DELIMITER statement in the mysql dialect [#901](https://github.com/sqlfluff/sqlfluff/issues/901))
 
 Contributors:
 - [@bolajiwahab](https://github.com/bolajiwahab) ([#1063])(https://github.com/sqlfluff/sqlfluff/pull/1063)
diff --git a/src/sqlfluff/cli/commands.py b/src/sqlfluff/cli/commands.py
index e7c8701be..e7266d438 100644
--- a/src/sqlfluff/cli/commands.py
+++ b/src/sqlfluff/cli/commands.py
@@ -288,16 +288,15 @@ def dialects(**kwargs):
 )
 @click.option(
     "-p",
-    "--parallel",
+    "--processes",
     type=int,
     default=1,
-    help="If set to a value higher than 1, run SQLFluff in parallel, "
-    "speeding up processing.",
+    help="The number of parallel processes to run.",
 )
 @click.argument("paths", nargs=-1)
 def lint(
     paths,
-    parallel,
+    processes,
     format,
     nofail,
     disregard_sqlfluffignores,
@@ -344,7 +343,7 @@ def lint(
                 paths,
                 ignore_non_existent_files=False,
                 ignore_files=not disregard_sqlfluffignores,
-                parallel=parallel,
+                processes=processes,
             )
         except OSError:
             click.echo(
@@ -410,14 +409,13 @@ def do_fixes(lnt, result, formatter=None, **kwargs):
     "--fixed-suffix", default=None, help="An optional suffix to add to fixed files."
 )
 @click.option(
-    "--parallel",
+    "--processes",
     type=int,
     default=1,
-    help="If set to a value higher than 1, run SQLFluff in parallel, "
-    "speeding up processing.",
+    help="The number of parallel processes to run.",
 )
 @click.argument("paths", nargs=-1)
-def fix(force, paths, parallel, bench=False, fixed_suffix="", logger=None, **kwargs):
+def fix(force, paths, processes, bench=False, fixed_suffix="", logger=None, **kwargs):
     """Fix SQL files.
 
     PATH is the path to a sql file or directory to lint. This can be either a
@@ -449,7 +447,7 @@ def fix(force, paths, parallel, bench=False, fixed_suffix="", logger=None, **kwa
     click.echo("==== finding fixable violations ====")
     try:
         result = lnt.lint_paths(
-            paths, fix=True, ignore_non_existent_files=False, parallel=parallel
+            paths, fix=True, ignore_non_existent_files=False, processes=processes
         )
     except OSError:
         click.echo(
diff --git a/src/sqlfluff/core/__init__.py b/src/sqlfluff/core/__init__.py
index e6d23b30a..2e56f3354 100644
--- a/src/sqlfluff/core/__init__.py
+++ b/src/sqlfluff/core/__init__.py
@@ -26,7 +26,7 @@ from sqlfluff.core.errors import (
 from sqlfluff.core.timing import TimingSummary
 
 
-# This is for "sqlfluff lint" and "sqlfluff fix" multiprocessing (--parallel)
+# This is for "sqlfluff lint" and "sqlfluff fix" multiprocessing (--processes)
 # support. If an exception (i.e. runtime error) occurs in a worker process, we
 # want to return the tracebook to the main process and report it there, as part
 # of the normal output. However, anything returned from a multiprocessing.Pool
diff --git a/src/sqlfluff/core/linter/linter.py b/src/sqlfluff/core/linter/linter.py
index ee51c8779..9e7566f30 100644
--- a/src/sqlfluff/core/linter/linter.py
+++ b/src/sqlfluff/core/linter/linter.py
@@ -763,7 +763,7 @@ class Linter:
         fix: bool = False,
         ignore_non_existent_files: bool = False,
         ignore_files: bool = True,
-        parallel: int = 1,
+        processes: int = 1,
     ) -> LintedDir:
         """Lint a path."""
         linted_path = LintedDir(path)
@@ -779,7 +779,7 @@ class Linter:
         runner = get_runner(
             self,
             self.config,
-            parallel=parallel,
+            processes=processes,
             allow_process_parallelism=self.allow_process_parallelism,
         )
         for linted_file in runner.run(fnames, fix):
@@ -796,7 +796,7 @@ class Linter:
         fix: bool = False,
         ignore_non_existent_files: bool = False,
         ignore_files: bool = True,
-        parallel: int = 1,
+        processes: int = 1,
     ) -> LintingResult:
         """Lint an iterable of paths."""
         # If no paths specified - assume local
@@ -813,7 +813,7 @@ class Linter:
                     fix=fix,
                     ignore_non_existent_files=ignore_non_existent_files,
                     ignore_files=ignore_files,
-                    parallel=parallel,
+                    processes=processes,
                 )
             )
         result.stop_timer()
diff --git a/src/sqlfluff/core/linter/runner.py b/src/sqlfluff/core/linter/runner.py
index 087059667..b72151daa 100644
--- a/src/sqlfluff/core/linter/runner.py
+++ b/src/sqlfluff/core/linter/runner.py
@@ -105,9 +105,9 @@ class ParallelRunner(BaseRunner):
     # don't pickle well.
     pass_formatter = False
 
-    def __init__(self, linter, config, parallel):
+    def __init__(self, linter, config, processes):
         super().__init__(linter, config)
-        self.parallel = parallel
+        self.processes = processes
 
     def run(self, fnames, fix):
         """Parallel implementation.
@@ -118,7 +118,7 @@ class ParallelRunner(BaseRunner):
         and linting work out to the threads.
         """
         with self._create_pool(
-            self.parallel,
+            self.processes,
             self._init_global,
             (self.config,),
         ) as pool:
@@ -212,22 +212,22 @@ class DelayedException(Exception):
 def get_runner(
     linter,
     config,
-    parallel: int,
+    processes: int,
     allow_process_parallelism: bool = True,
 ) -> BaseRunner:
     """Generate a runner instance based on parallel and sytem configuration."""
     # Python multiprocessing isn't supported in 3.6 and before.
     # The library exists but we get pickling errors with LintedFile.
-    if parallel > 1 and sys.version_info >= (3, 7):
+    if processes > 1 and sys.version_info >= (3, 7):
         # Process parallelism isn't really supported during testing
         # so this flag allows us to fall back to a threaded runner
         # in those cases.
         if allow_process_parallelism:
-            return MultiProcessRunner(linter, config, parallel=parallel)
+            return MultiProcessRunner(linter, config, processes=processes)
         else:
-            return MultiThreadRunner(linter, config, parallel=parallel)
+            return MultiThreadRunner(linter, config, processes=processes)
     else:
-        if parallel > 1:
+        if processes > 1:
             linter_logger.warning(
                 "Parallel linting is not supported in Python %s.%s.",
                 sys.version_info.major,


Test patch:
diff --git a/test/core/linter_test.py b/test/core/linter_test.py
index 1eac62a5e..d71ba23f7 100644
--- a/test/core/linter_test.py
+++ b/test/core/linter_test.py
@@ -186,8 +186,8 @@ def test__linter__linting_result_check_tuples_by_path(by_path, result_type):
     isinstance(check_tuples, result_type)
 
 
-@pytest.mark.parametrize("parallel", [1, 2])
-def test__linter__linting_result_get_violations(parallel):
+@pytest.mark.parametrize("processes", [1, 2])
+def test__linter__linting_result_get_violations(processes):
     """Test that we can get violations from a LintingResult."""
     lntr = Linter()
     result = lntr.lint_paths(
@@ -195,7 +195,7 @@ def test__linter__linting_result_get_violations(parallel):
             "test/fixtures/linter/comma_errors.sql",
             "test/fixtures/linter/whitespace_errors.sql",
         ],
-        parallel=parallel,
+        processes=processes,
     )
 
     all([type(v) == SQLLintError for v in result.get_violations()])
@@ -234,7 +234,7 @@ def test__linter__linting_parallel_thread(force_error, monkeypatch):
     lntr = Linter(formatter=CallbackFormatter(callback=lambda m: None, verbosity=0))
     result = lntr.lint_paths(
         ("test/fixtures/linter/comma_errors.sql",),
-        parallel=2,
+        processes=2,
     )
 
     all([type(v) == SQLLintError for v in result.get_violations()])
@@ -247,7 +247,7 @@ def test_lint_path_parallel_wrapper_exception(patched_lint):
     Test on MultiThread runner because otherwise we have pickling issues.
     """
     patched_lint.side_effect = ValueError("Something unexpected happened")
-    for result in runner.MultiThreadRunner(Linter(), FluffConfig(), parallel=1).run(
+    for result in runner.MultiThreadRunner(Linter(), FluffConfig(), processes=1).run(
         ["test/fixtures/linter/passing.sql"], fix=False
     ):
         assert isinstance(result, runner.DelayedException)