How do I extract Nested Dictionary Data in Python?
Tags: Extract, Dictionary, Python, Nested, Json

Tutorial

How do I extract Nested Dictionary Data in Python?

JSON: Dictionary and List Data Structures (Types), Image by Author.

“Life is a like an onion, you peel it off one layer at time, and sometimes you weep” ― Carl Sandberg

I suppose the same could be said of extracting values from nested JSON structures. Even the most skilled programmer can be brought to tears when working with a JSON object that consists of a mix of deeply nested data structures. The process of extracting the values can feel messy and disorganized at best. The more data there is, the bigger the mess.

In this tutorial, I’ll walk you through a step-by-step method to extract the values you need from any JSON. A word of warning: this tutorial is not meant for newbies to JSON, lists or dictionaries. If you’ve never heard of a list index or a dictionary key-value pair, I would suggest reviewing one of the many great tutorials available on the web or YouTube. Once you feel more comfortable with the subject, come back to continue learning and growing.

Housekeeping

JSON vs. Lists vs. Dictionaries

First things first, when it comes to the terms “JSON”, “list” and “dictionary”, we have to do some important housekeeping. JSON, or JavaScript Object Notation, is a broader format used to encompass dictionary and list structures as shown in the image below.

JSON: List and Dictionary Structure, Image by Author.

The technical documentation says a JSON object is built on two structures: a list of key-value pairs and an ordered list of values. In Python Programming, key-value pairs are dictionary objects and ordered list are list objects. In practice, the starting point for the extraction of nested data starts with either a dictionary or list data structure. When extracting Nested Data the question(s) should be: Is the data nested in a Dictionary or List data structure? What is the combination of data structures used? Is the first data structure used a dictionary or a list?

“It has long been an axiom of mine that the little things are infinitely the most important.” — Sir Arthur Conan Doyle,

If it seems like I’m making a big deal about the terminology, it is because I am. When it comes to extracting nested data the details matter. Data structures change the deeper the data is nested in the JSON structure and knowing those distinctions are important. The initial data structure may be a list but then change to a dictionary as the data is extracted. The key to extracting data from a JSON object is recognizing the mix of data structures used to store the data. If you struggle to recognize the data structure in a JSON object, it’s likely that you’ll struggle to extract the values you want. In most cases, this results in applying the wrong extraction technique.

The table below is a brief refresher on the techniques used to extract data from a JSON structure.

Data Types and Extraction Methods, Image by Author

One final note before starting our example. In Python Programming, the term “data structure” is rarely used when describing lists and dictionary. The commonly used term is “data type”. I use the terms data type and data structure interchangeably throughout this tutorial. I use the term data structure because it conveys the idea that the data structures are the fundamental building blocks of the JSON object. The usage of the term data type in Python is not of less importance however it does not convey the same meaning as a key to understanding nested data extraction.

Real World Data

Let's Get Started