The JSON trick 25% of Python devs don't know about
Tags: python, programming, webdev, tutorial
Published: 2022-02-09 01:06:13

Reading and writing JSON files is a common chore in Python. In fact our analysis of 888 codebases found that 17% of repositories either read from or wrote to JSON files. Of those 150 codebases 25% were making their lives harder than necessary by not using a built-in JSON feature. A key part of our job as devs is to reduce the complexity of the code we write, so this is a good area of improvement for one in four Python devs.

At the end of this article is a link to the stats. You may want to check the stats. You never know, your repo might be one of the 888 we checked!

Many of the occurrences were in big open source projects, so this is not just a rookie mistake. For fun we listed some notable examples near the end of the article.

### Reading JSON files

Have you ever wondered why [`json.loads`](https://docs.python.org/3/library/json.html#json.loads) ends with a "s"? The reason is because that method is specifically for working on a string, and there is another related method for working on files: [`json.load`](https://docs.python.org/3/library/json.html#json.load)

`json.loads` is for deserialising a string containing JSON, while `json.load` (no "s") is for deserialising a file containing JSON.

The methods are very related. In fact, [`json.load` is a wrapper around `json.loads`](https://github.com/python/cpython/blob/d7c567b08f9d7d6aef21b881340a2b72731129db/Lib/json/__init__.py#L293). It's a method provided out of the box by Python to simplify the task of reading JSON from file-like objects. For example, these result in the same outcome:

```py
# bad
with open('some/path.json', 'r') as f:
    content = json.loads(f.read())

# good
with open('some/path.json', 'r') as f:
    content = json.load(f)
```

So why not choose the easiest to read and write?


### Writing JSON files

Similarly to `json.loads`, [`json.dumps`](https://docs.python.org/3/library/json.html#json.dumps) also ends with an "s" because it's specifically for working on a string and there is another related method: [`json.dump`](https://docs.python.org/3/library/json.html#json.dump)

`json.dumps` is for JSON serialising an object to a string, while `json.dump` (no "s") is for JSON serialising an object to a file.

Again similar to the relationship between `json.load` and `json.loads`, [`json.dump` is a wrapper around `json.dumps`](https://github.com/python/cpython/blob/d7c567b08f9d7d6aef21b881340a2b72731129db/Lib/json/__init__.py#L120). It's a method provided out of the box by Python to simplify the task of writing JSON to a file-like object.

For example, these result in the same outcome:

```py
# bad
with open('some/path.json', 'w') as f:
    f.write(json.dumps({'foo': 'bar'}))

# good
with open('some/path.json', 'w') as f:
    json.dump({'foo': 'bar'}, f)

```

So why not choose the easiest to read and write?

### Reduce complexity

Python core developers are very careful about what features are included in the Python built-in modules because once they expose an API to devs it's very hard for them to remove it. So we've been given some tools that simplify our handling of JSON files so it makes sense to utilise the convenience methods and not reinvent the wheel. For that reason when [Code Review Doctor](https://codereview.doctor) sees devs not using `json.load` and `json.dump` this advice is given:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iayuhyqmr67j53sigx2u.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h4osa534var1q4eqaqz6.png)

### Examples

Many of the occurrences were in big open source projects, so this is not just a mistake rookies make, but rather is a often overlooked feature of Python:

- [Microsoft](https://github.com/microsoft/dpu-utils/blob/72cac67490ccf19fede6b86269c57843549b526c/python/tests/utils/test_richpath.py#L71) 
- [Ansible](https://github.com/ansible/ansible/blob/5bddecb048aae0a7fc84e38e086ac90445c4edb4/test/units/galaxy/test_api.py#L1062)
- [Unicef](https://github.com/unicef/etools/blob/5b5fb14806ebc6696f03e15f6a0b0fd23746e105/src/etools/applications/partners/tests/test_api_prp.py#L58) 
- [Sentry](https://github.com/getsentry/sentry/blob/bdae0c34b042467f15dd73d9da5fb8749dc59a58/src/sentry/utils/distutils/commands/build_js_sdk_registry.py#L29)
- [UK Department for International Trade](https://github.com/uktrade/directory-tests/blob/5aabf7bb0e923fd9afccdc99a0f1be90e7cce111/update_results.py#L22) 


### Stats

You can read the gists: 
Reading JSON: [json.load vs json.loads](https://gist.github.com/code-review-doctor/f6cd072becd256fe7c81b24ab3db58d3)
Writing JSON: [json.dump vs json.dumps](https://gist.github.com/code-review-doctor/b457f8e9020124cdd294f0bdf443deb9)

How did we find this statistic? At [CodeReview.doctor](https://codereview.doctor) we run static analysis checks on codebases to find bugs and mistakes most humans miss. During this process we collect stats on the public repos. We tracked every line we could find where developers interacted with JSON files. We were surprised by the results!

If you would like to check if you're affected by issues like this one, check your codebase instantly for free at [CodeReview.doctor](https://codereview.doctor).