Three things your linter shouldn’t tell you
Tags: webdev, python, javascript
Published: 2019-11-21 19:37:00

---
published: true
description: "How we’ve curated our code checks in Bento"
---

My coworker Grayson originally wrote this blog post. It was first posted on [the r2c engineering blog](https://blog.r2c.dev). At r2c we are linter lovers at heart and value tools that help us write better code, faster. However, sometimes we’re left wanting for a bit more from linters…

Oh, ESLint rules aren’t enabled by default? Which is better, Airbnb’s or Google’s rule list? Bandit is giving me thousands of errors? Oh, those are just `asserts`.  Why does Bandit consider `assert` a security violation? 🤔 Shouldn’t this be easier?…

Wouldn’t it be great to have a code check tool where you *could just run `check` and it showed you the results you cared about?*

We’re making a new developer tool called [Bento](https://bento.dev/). Developer time is valuable, and we frequently hear from developers that their linters demand a lot from them. It’s our belief that your linter shouldn’t bother you with things you don’t care about, so in Bento we turn off checks based on these three things:

## Linters Shouldn’t Tell You about Style Violations

Frankly, we don’t care about one or two lines between function definitions or whether there should be spaces around binary operators. With deterministic formatters like [**Black**](https://pypi.org/project/black/) for Python and [**Prettier**](https://prettier.io/) for JavaScript, there is **no need** to spend valuable development time manually addressing style violations when a formatter can do this automatically.

Therefore, Bento’s default configuration explicitly disables all style and preference checks. **Bento** only reports violations that might** actually be a problem**.

For ESLint (with the [Airbnb config](https://github.com/airbnb/javascript)), this amounts to disabling all checks labeled “style” by both [ESLint](https://eslint.org/docs/rules/#stylistic-issues) and [Airbnb](https://github.com/airbnb/javascript/blob/master/packages/eslint-config-airbnb-base/rules/style.js), checks with “prefer” in the name, and a handful of checks that our manual examination determined did not indicate the presence of an actual bug.

For Flake 8, this amounts to most [`pycodestyle`](https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes) checks.

## Linters Shouldn’t Tell You about Unpopular Checks

We hypothesized we could get a good estimate for checks that developers like and dislike by mining the configuration files for code checking tools in open source code. Using our at-scale code analysis platform, [app.r2c.dev](https://app.r2c.dev/), we mined the ESLint configurations of the top 1,000 npm packages (as of April 2019), and 128 Bandit configurations on GitHub.

#### Disabled ESLint checks:
![Disabled ESLint checks](https://blog.r2c.dev/media/disabled-eslint-checks.png)

Perhaps unsurprisingly, about 38% of repositories globally disabled `no-console`. Apparently, people like `console.log`, which makes sense! We disable this check in Bento.

#### Disabled Bandit checks:
![Disabled Bandit checks](https://blog.r2c.dev/media/disabled-eslint-checks.png)

Of the Bandit configs, nearly 80% disable `assert_used - B101`! We disable this check in Bento, too. About 45% also disable `random - B311`, so we disable it as well.

As you might imagine, this is the first step in **tailoring checks to your code**.  In the future, **Bento** will show you the checks **you want to see** by identifying which checks get fixed… and which get ignored. This will be incredibly powerful—and necessary—as we add support for more code checks.

## Linters Shouldn’t Tell You about “Bad Checks”

Granted, the definition of “bad” is subjective! 🤗 Generally speaking, our definition of a “bad check” is a check with a high false-positive rate and a low chance of being a bug.

As an example, `B105` checks for `hardcoded_password_strings`. In theory, this check is great!

* it prevents checking in hard-coded secrets, and
* it prevents easily guessable defaults — for example, `admin:password`.

However, in *practice*, this check is very noisy. The [check itself](https://github.com/PyCQA/bandit/blob/1661456292b8b909833dd2bd9d7179636ca5aa1a/bandit/plugins/general_hardcoded_password.py#L41) looks for assignments or comparison operators that “look like a password.” This check commonly has a false positive on the variable name `token`, frequently used in string parsing operations.

We use our [static analysis platform](https://app.r2c.dev/) to test checks on real-world code. We can test on thousands of open source repositories at once to see where a check yields a true or false positive. Usually, it is easy to spot false positives—such as `token` in string parsers.

```python
#https://github.com/cobbler/cobbler/blob/91e3ff1d2a05eeb24107ef72e09c1a3d800ae304/cobbler/web/templatetags/site.py#L245
while not self.at_end():
    token = self.get_token()
    if token == 'not':    # B105 fires here
        if self.at_end():
            raise self.error_class('No variable provided after "not".')
        token = self.get_token()
        negate = True
```

To be more scientific, we can measure the accuracy of the check. After rigorously reviewing 20 repositories, including [Django](https://github.com/django/django) and [Ansible](https://github.com/ansible/ansible), `B105` hits nothing of importance 93% of the time. While the theory is good, this check is rather noisy. Therefore, Bento disables this check. (FYI, neither Django nor Ansible use hardcoded passwords!…except in their test files. 😉)

Checks we ship in **Bento** are reviewed this way. If a check is “bad,” it’s disabled. If “badness” cannot be determined, we leave it enabled. We figure our users will tell us if it’s bad! 😅

## Run Bento on your code
We care a lot about code quality at r2c, and we love our linters! We’ve made it even easier to write better code faster by filtering out the checks that we don’t want linters to tell us about. You can run *Bento* on your code to take advantage of these checks by following this link:  https://bento.dev/.

If you have feedback or comments for us, [email us](mailto:hello@r2c.dev) or [make an issue on GitHub](https://github.com/returntocorp/bento/issues).