Repository: pantsbuild/pants

Issue:
Using a newer version of `pylint`
I'm trying to use a newer version of `pylint` with Pants. Even though I've got `pylint` set to version 2.8.2 in `requirements.txt`/`poetry.lock`, it seems that it kept getting built with version `>=2.4.4,<2.5`. After looking around [in the docs](https://www.pantsbuild.org/v2.0/docs/reference-pylint#advanced-options), I discovered that that's the default `pylint` version in Pants. There's apparently an option to set an alternative `version` in `pants.toml`, which I'm able to use to lint my codebase using a more recent version of `pylint`.

However, I'd like to understand:
1. Why does Pants have a default version of `pylint` at all instead of always using the most recent version (or the version specified in e.g. `requirements.txt`)?
2. Related to the previous question: what, if any, are the risks of using a newer `pylint` version in Pants?


Discussion:
Hey @martimlobao, welcome! Good questions.

> Why does Pants have a default version of pylint at all 

Mostly for stability for users, that we want to avoid the problem of your CI breaking overnight because of a new release. If it's unconstrainted, it's more susceptible to breaking on you.

We periodically update the default version, though, and then indicate that in the changelog as a User API Change.

> what, if any, are the risks of using a newer pylint version in Pants?

There shouldn't be :) In fact, we'd love a patch to update the default version of Pylint if you're interested! It's nothing more than updating this line to be `>=2.8.2,<2.9`

https://github.com/pantsbuild/pants/blob/955ee9509348b4d31395d03b145f410d0d3ba40d/src/python/pants/backend/python/lint/pylint/subsystem.py#L21

See https://www.pantsbuild.org/docs/contributor-overview for a contributor guide. You could skip the parts about building the rust engine and testing things locally, given that this is a one line change.

(No worries if not interested)
Awesome, I'd love to submit a PR :)

Resolving patch:
diff --git a/src/python/pants/backend/python/lint/bandit/subsystem.py b/src/python/pants/backend/python/lint/bandit/subsystem.py
--- a/src/python/pants/backend/python/lint/bandit/subsystem.py
+++ b/src/python/pants/backend/python/lint/bandit/subsystem.py
@@ -15,7 +15,7 @@ class Bandit(PythonToolBase):
     options_scope = "bandit"
     help = "A tool for finding security issues in Python code (https://bandit.readthedocs.io)."
 
-    default_version = "bandit>=1.6.2,<1.7"
+    default_version = "bandit>=1.7.0,<1.8"
     default_extra_requirements = [
         "setuptools<45; python_full_version == '2.7.*'",
         "setuptools; python_version > '2.7'",
diff --git a/src/python/pants/backend/python/lint/black/subsystem.py b/src/python/pants/backend/python/lint/black/subsystem.py
--- a/src/python/pants/backend/python/lint/black/subsystem.py
+++ b/src/python/pants/backend/python/lint/black/subsystem.py
@@ -17,7 +17,7 @@ class Black(PythonToolBase):
     help = "The Black Python code formatter (https://black.readthedocs.io/)."
 
     # TODO: simplify `test_works_with_python39()` to stop using a VCS version.
-    default_version = "black==20.8b1"
+    default_version = "black==21.5b2"
     default_extra_requirements = ["setuptools"]
     default_main = ConsoleScript("black")
     register_interpreter_constraints = True
diff --git a/src/python/pants/backend/python/lint/pylint/subsystem.py b/src/python/pants/backend/python/lint/pylint/subsystem.py
--- a/src/python/pants/backend/python/lint/pylint/subsystem.py
+++ b/src/python/pants/backend/python/lint/pylint/subsystem.py
@@ -18,7 +18,7 @@ class Pylint(PythonToolBase):
     options_scope = "pylint"
     help = "The Pylint linter for Python code (https://www.pylint.org/)."
 
-    default_version = "pylint>=2.4.4,<2.5"
+    default_version = "pylint>=2.6.2,<2.7"
     default_main = ConsoleScript("pylint")
 
     @classmethod


Test patch:
diff --git a/src/python/pants/backend/python/lint/bandit/rules_integration_test.py b/src/python/pants/backend/python/lint/bandit/rules_integration_test.py
--- a/src/python/pants/backend/python/lint/bandit/rules_integration_test.py
+++ b/src/python/pants/backend/python/lint/bandit/rules_integration_test.py
@@ -43,7 +43,6 @@ def run_bandit(
     rule_runner.set_options(
         [
             "--backend-packages=pants.backend.python.lint.bandit",
-            "--bandit-version=bandit==1.6.2",
             *(extra_args or ()),
         ],
         env_inherit={"PATH", "PYENV_ROOT", "HOME"},
@@ -110,21 +109,24 @@ def test_uses_correct_python_version(rule_runner: RuleRunner) -> None:
             ),
         }
     )
+    py2_args = [
+        "--bandit-version=bandit>=1.6.2,<1.7",
+    ]
     py2_tgt = rule_runner.get_target(Address("", target_name="py2", relative_file_path="f.py"))
-    py2_result = run_bandit(rule_runner, [py2_tgt])
+    py2_result = run_bandit(rule_runner, [py2_tgt], extra_args=py2_args)
     assert len(py2_result) == 1
     assert py2_result[0].exit_code == 0
     assert "f.py (syntax error while parsing AST from file)" in py2_result[0].stdout
 
     py3_tgt = rule_runner.get_target(Address("", target_name="py3", relative_file_path="f.py"))
-    py3_result = run_bandit(rule_runner, [py3_tgt])
+    py3_result = run_bandit(rule_runner, [py3_tgt], extra_args=py2_args)
     assert len(py3_result) == 1
     assert py3_result[0].exit_code == 0
     assert "No issues identified." in py3_result[0].stdout
 
     # Test that we partition incompatible targets when passed in a single batch. We expect Py2
     # to still fail, but Py3 should pass.
-    combined_result = run_bandit(rule_runner, [py2_tgt, py3_tgt])
+    combined_result = run_bandit(rule_runner, [py2_tgt, py3_tgt], extra_args=py2_args)
     assert len(combined_result) == 2
 
     batched_py2_result, batched_py3_result = sorted(
diff --git a/src/python/pants/backend/python/lint/pylint/rules_integration_test.py b/src/python/pants/backend/python/lint/pylint/rules_integration_test.py
--- a/src/python/pants/backend/python/lint/pylint/rules_integration_test.py
+++ b/src/python/pants/backend/python/lint/pylint/rules_integration_test.py
@@ -305,7 +305,7 @@ def test_source_plugin(rule_runner: RuleRunner) -> None:
                 """\
                 python_requirement_library(
                     name='pylint',
-                    requirements=['pylint>=2.4.4,<2.5'],
+                    requirements=['pylint>=2.6.2,<2.7'],
                 )
 
                 python_requirement_library(