How to handle JSON in Python?. In this article, we will learn and... | by Tanu N Prabhu | Towards Data Science

JSON is the abbreviation for JavaScript Object Notation: one of the easiest data formats for humans being like you and me to read and write. JSON is often declared as a text format that is completely language independent. It is based on a subset of the JavaScript Programming Language Standard ECMA-262 3rd Edition — December 1999.
Quick heads up, the entire code of this tutorial can be found in my GitHub repository given below:
github.com
The JSON is built on two structures:
A collection of name/value pairs. In many programming languages, we often refer to this as an object, record, struct, dictionary, hash table, etc.An ordered list of values. In many programming languages, we refer to this as an array, vector, list, or sequence.
A collection of name/value pairs. In many programming languages, we often refer to this as an object, record, struct, dictionary, hash table, etc.
An ordered list of values. In many programming languages, we refer to this as an array, vector, list, or sequence.
An object is an unordered set of name/value pairs. Every time an object begins with {} (left brace and ends with a right brace). Each name is followed by : a colon and the name/value pairs are separated by a , comma.
For example:
example = '{"name": "Jake", "programming languages": ["Python", "Kotlin"]}'
For extra details please refer to JSON official docs given below:
www.json.org
Let us stick to a single dataset throughout this article for consistency. I recommend bookmarking, fork, or star this repository because this repository has one of the mind-blowing JSON format datasets.
github.com
From the above list of JSON datasets, I hence chose the smallest JSON dataset.
The details of the dataset up to an extent is as given below:
Pss: How did I get this, please refer to the bonus example below:
Name: Neighbourhoods in UK (Leicestershire)Link: https://data.police.uk/api/leicestershire/neighbourhoodsRangeIndex: 6 entries, 0 to 5Data columns (total 2 columns):  #   Column  Non-Null Count  Dtype ---  ------  --------------  -----   0   id      6 non-null     object  1   name    6 non-null     object dtypes: object(2) memory usage: 224.0+ KB
This is how a JSON format data looks. Well, I know it looks hideous but not all the time the datasets are supposed to look neat; it is your job to make them look neat.
I have only taken a few instances of the dataset, because of its size.
[{"id":"NC04","name":"City Centre"},{"id":"NC66","name":"Cultural Quarter"},{"id":"NC67","name":"Riverside"},{"id":"NC68","name":"Clarendon Park"},{"id":"NE09","name":"Belgrave South"},{"id":"NE10","name":"Belgrave North"}]
Now let’s just store the above JSON dataset in a variable named json_data. You will understand the whole purpose of the later article.
json_data = '[{"id":"NC04","name":"City Centre"},{"id":"NC66","name":"Cultural Quarter"},{"id":"NC67","name":"Riverside"},{"id":"NC68","name":"Clarendon Park"},{"id":"NE09","name":"Belgrave South"},{"id":"NE10","name":"Belgrave North"}]'
We actually don’t need to install any external modules. It is one of those built-in libraries wherein you actually don’t need to type.
pip install json
So all you have to do is just say:
import json
Note: Remember import json is a Python module, so to use it you need to import it first.
Parsing a JSON string using loads(). The loads() takes a JSON string as input and it returns a dictionary or a list.
Remember, if there is a single entry in the dataset then the loads() will return a dictionary otherwise for multiple entries as the above dataset the loads() will return a list.
To be specific, a dataset that has only a single row is called a single entry, whereas a dataset with over two rows is called multiple entries. The example given below makes things more clear.
The syntax of loads() is pretty trivial. We pass a single parameter most of the time which is the json data itself.
variable = json.loads(json_data)
Let's use the above stored json_data as our base dataset and then pass it to loads().
# Importing json moduleimport json# Passing the json data to the loads methodneighbourhood = json.loads(json_data)# Printing the result - listprint(neighbourhood)print("-----------------------------------------------------")# Checking the type of the variableprint(type(neighbourhood))print("-----------------------------------------------------")# Accessing the first element in the list - Using indexprint(neighbourhood[0])
Like I mentioned earlier, the loads() might return a list or a dictionary depending on the JSON format data. In this case, it is a list, so the above output would be:
[{'id': 'NC04', 'name': 'City Centre'}, {'id': 'NC66', 'name': 'Cultural Quarter'}, {'id': 'NC67', 'name': 'Riverside'}, {'id': 'NC68', 'name': 'Clarendon Park'}, {'id': 'NE09', 'name': 'Belgrave South'}, {'id': 'NE10', 'name': 'Belgrave North'}]----------------------------------------------------- <class 'list'> ----------------------------------------------------- {'id': 'NC04', 'name': 'City Centre'}
Parsing a Python object using dumps(). The dumps() takes a Python object as input and it returns a JSON string.
The syntax of dumps() is pretty trivial. We pass a single parameter most of the time which is the Python Object (list) itself.
variable = json.dumps(neighbourhood)
Let’s use the above stored neighbourhood as our base dataset which is a Python list and then pass it to dumps().
# Importing json moduleimport json# Passing the python data to the dumps methodneighbourhood_list = json.dumps(neighbourhood)# Checking the type of the variableprint(type(neighbourhood_list))print("-----------------------------------------------------")# Printing the result - stringprint(neighbourhood_list)
Now the dumps() takes the python object as input and then returns a JSON string as output as shown below:
<class 'str'> ----------------------------------------------------- [{"id": "NC04", "name": "City Centre"}, {"id": "NC66", "name": "Cultural Quarter"}, {"id": "NC67", "name": "Riverside"}, {"id": "NC68", "name": "Clarendon Park"}, {"id": "NE09", "name": "Belgrave South"}, {"id": "NE10", "name": "Belgrave North"}]
Now, for this conversion, let us use the most used Python objects on a daily basis such as:
Dictionary
List
String
Integer
Note: Remember for the conversion of the Python objects into a JSON string you need to use dumps().
First, let us create four different types of variables which can hold the above python data
dictt = {"name": "John Depp", "age": 48}listt = ["John", "Depp", 48]strr = "John Depp"intt = 48
Now let's feed these values to the dumps() and then see the conversion into a JSON string.
# Python Object 1 - Dictionaryprint(json.dumps(dictt))# Checking the typeprint(type(json.dumps(dictt)))print("-----------------------------------------------------")# Python Object 2 - Listprint(json.dumps(listt))# Checking the typeprint(type(json.dumps(listt)))print("-----------------------------------------------------")# Python Object 3 - Stringprint(json.dumps(strr))# Checking the typeprint(type(json.dumps(strr)))print("-----------------------------------------------------")# Python Object 4 - Integerprint(json.dumps(intt))# Checking the typeprint(type(json.dumps(intt)))print("-----------------------------------------------------")
Now no matter what, the type is always gonna be a str.
{"name": "John Depp", "age": 48} <class 'str'> ----------------------------------------------------- ["John", "Depp", 48] <class 'str'> ----------------------------------------------------- "John Depp" <class 'str'> ----------------------------------------------------- 48 <class 'str'>-----------------------------------------------------
Similarly, try to do the same with float, tuple, and true, or false objects and let me know the results in the comment section below.
In this bonus example, you will get to know the conversion of JSON data into a data frame. For this understand the scenario below:
Now think that you are doing a data science project for whoever it is. The details of the project are not so important in this case. So the first step is to gather the data, now most of the time on Kaggle, data.gov or etc, the data would already in a CSV format, but in this case, your boss or professor gives you data which is in a JSON format (happened to me). Now how to tackle this situation?
The solution is pretty simple, it’s not rocket science, rather you need to use loads() and the pandas.DataFrame() methods. For this example, I will use the Neighbourhoods in UK (Leicestershire) dataset from above.
# Import both JSON and Pandas libraryimport jsonimport pandas as pd# Use the `loads()` method to load the JSON datadf = json.loads(json_data)# Pass the generated JSON data into a pandas dataframedf = pd.DataFrame(df)print(df)print("-----------------------------------------------------")# Checking the type of the data print(type(df))
After executing the above code the result would be a neat looking dataframe
The type of data now will be as shown below:
<class 'pandas.core.frame.DataFrame'>
This is how you can convert a not so looking neat JSON data into a beautiful-looking DataFrame. To get more information about the dataset use df.info(), this is how I got “Details of the dataset”
Congratulations guys you have successfully completed reading/implementing this beautiful article “How to handle JSON in Python?”. This tutorial covers all the concepts along with specific examples to understand the handling of JSON data in Python. If you guys have any comments or suggestions, please use the comment section wisely. I hope you guys have learned something new today. Stay tuned for more updates. Until then, see you next time. Have a good day and stay safe!