How to Serialize a Class Object to JSON in Python
By: Changsin Lee
Tags: Python Programming, Json, Python, Serialization

Serializing your own class objects is quite straightforward in Python. Here are tips.

Photo by Claudio Schwarz on Unsplash

Serialization is the process of converting an object into a medium that can be saved and retrieved later: e.g., saving the state of an object into a file. For any mildly complex projects, serialization is something all developers have to do sooner or later. One of the nice things about the Python language is its ease of use in many common programming tasks. File IO, plotting diagrams, and accessing the web contents are all easy with just a few lines of code. Serialization is also easy in Python, except when you are trying to serialize your own custom class. In this article, I want to share what I found to be the best practice in serializing your own class objects to JSON objects. The entire code sample is shared here.

1. Class Definition

Let‘s start with the definition of a sample class.

class Label:

def __init__(self, label, x, y, width, height):

self.label = label

self.x = x

self.y = y

self.width = width

self.height = height

If you want to serialize (e.g., to print out the object), you will get an esoteric message like this:

label = Label("person", 10, 10, 4, 10)

print(label) >> <__main__.Label object at 0x000002C3913EB2E0>

Python json library provides a handy method called json.dumps() to turn any Python objects into JSON. This is good so you call the method and pass the object. Sound simple so let’s do that.

import json



print(json.dumps(label)) >>...

/usr/lib/python3.7/json/encoder.py in default(self, o)

177

178 """

--> 179 raise TypeError(f'Object of type {o.__class__.__name__} '

180 f'is not JSON serializable')

181



TypeError: Object of type ImageLabel is not JSON serializable

What happened? json.dumps() calls encoder for a custom object and when raises an error for the class object because there is no encoder.

2. Requirements

We can implement an encoder for this class, but let’s find a more general solution that meets the following requirements:

Outputs a proper JSON: Serialization results are proper JSON objects. print() just works: print() on the object does not require special handling. json.dumps() works: json.dumps() is the default way of turning a Python object into a JSON object and it behaves as expected. Generic and scalable: It is easy to fix for a particular class but we want our solution to work for all class types and relationships.

With these requirements in mind, let’s dive in and explore the options.

3. Solutions

Method 1: Use json.dumps() and __dict__

The simplest way to serialize is to use the built-in __dict__ method to display the object’s content:

label = Label("Brake", 0, 867, 328, 36)

print(label.__dict__)

print(json.dumps(label.__dict__)) >{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}

>{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}

Both print() and json.dumps() show the content of the object in JSON format.

Method 2: Implement __str__ and __repr__

However, the above method is limited because you need to call __dict__ on the object every time. To make print simple, you need to implement the __str__ or __repr__ method on the class.

Both __str__ and __repr__ returns the string representation of the object. The difference is the audience: __str__ is supposed to be human-readable and __repr__ for machines (e.g., processed by other programs). For our purposes, they are the same so __repr__ just relays the call to __str__.

label = Label("person", 10, 10, 4, 10)

print(label) # print(json.dumps(label))

But json.dumps() still does not work because it still needs the encoder.

Method 3: Implement JSON Encoder

To support the json.dumps use case, one way is to implement a custom encoder class by subclassing from JSONEncoder. In our case, since we want the object to be in JSON dictionary format, we are just returning the dictionary.

JSONEncoder

Here is the output.

# outputs of a LabelSimple class object

# which does not have print method

{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10} <__main__.LabelSimple object at 0x7f5e3d8dc2d0>

{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10} # outputs of a Label class object

{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10} {"label": "person", "x": 10, "y": 10, "width": 4, "height": 10} {"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}

You can see that MyEncoder works even for LabelSimple which does not have __str__ or __repr__ methods.

The pros of this approach are that you don’t have to implement extra methods like __str__ or __repr__ and support a wide variety of class object types. The disadvantage is that you have to implement a custom class.

4. Handling complex objects

The above methods, however, do not work if you need to handle more complex cases. Here is a sample JSON output we want to see:

The Label class objects need to be contained in another class. Let’s define a containing class called ‘ImageLabelCollection’ and implement the __str__ method that will return the JSON object of the collection object. Note that we used MyEncoder which will return the dictionary object of Label objects.

The output looks pretty good:

# output from the __str__ method

{"version": 1, "type": "bounding-box-labels", "boundingBoxes": {"image1.jpg": [{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}, {"label": "car", "x": 20, "y": 20, "width": 5, "height": 11}]}} # output from json.dumps() method

{"version": 1, "type": "bounding-box-labels", "bboxes": {"image1.jpg": [{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}, {"label": "car", "x": 20, "y": 20, "width": 5, "height": 11}]}}

However, if you look carefully, the key for the collection of Label objects is different in the output of the two methods. In the case of __str__, the key is “boundBoxes” whereas json.dumps returns “bboxes” instead. The reason is because json.dumps() does not call __str__ method and instead simply returns the dictionary of the object itself. So instead of calling the __str__ which in turn calls __iter__ method to get the correct key, “boundingBoxes”, it simply takes the class variable name “bboxes” as the key for the collection object.

Method 4: Implement to_json() method

To properly serialize an object so that it works even for json.dumps(), you need to write a custom to_json() method. The first instinct is to replace the default method in JSONEncoder with to_json() and have it call __str__.

JSONEncoder with a custom to_json() method

After modifying the classes, we tried testing with different combinations, but none of the results meets our requirements. They all have double or triple-quote problems because contained Label objects are already made into string objects in their own json.dumps() call so calling them again will add extra quotation marks on the string objects. Arghhh! We made the problem worse!

# __str__ output

{"version": 1, "type": "bounding-box-labels", "boundingBoxes": {"image1.jpg": ["{\"label\": \"person\", \"x\": 10, \"y\": 10, \"width\": 4, \"height\": 10}", "{\"label\": \"car\", \"x\": 20, \"y\": 20, \"width\": 5, \"height\": 11}"]}} # calling to_json() directly

{"version": 1, "type": "bounding-box-labels", "boundingBoxes": {"image1.jpg": ["{\"label\": \"person\", \"x\": 10, \"y\": 10, \"width\": 4, \"height\": 10}", "{\"label\": \"car\", \"x\": 20, \"y\": 20, \"width\": 5, \"height\": 11}"]}} # json.dumps() with JSONEncoder

"{\"version\": 1, \"type\": \"bounding-box-labels\", \"boundingBoxes\": {\"image1.jpg\": [\"{\\\"label\\\": \\\"person\\\", \\\"x\\\": 10, \\\"y\\\": 10, \\\"width\\\": 4, \\\"height\\\": 10}\", \"{\\\"label\\\": \\\"car\\\", \\\"x\\\": 20, \\\"y\\\": 20, \\\"width\\\": 5, \\\"height\\\": 11}\"]}}" # json.dumps() on to_json() output

"{\"version\": 1, \"type\": \"bounding-box-labels\", \"boundingBoxes\": {\"image1.jpg\": [\"{\\\"label\\\": \\\"person\\\", \\\"x\\\": 10, \\\"y\\\": 10, \\\"width\\\": 4, \\\"height\\\": 10}\", \"{\\\"label\\\": \\\"car\\\", \\\"x\\\": 20, \\\"y\\\": 20, \\\"width\\\": 5, \\\"height\\\": 11}\"]}}"

Method 5: Implement a custom to_json() method

To solve the double-quotation mark problem, you need to implement a custom to_json() method in the container class. To make the implementation and calling to_json easier, we are now defining a global method called ‘default’.

customized to_json method in the container class

The work is a little tedious because you have to handle the serialization of each containing class object yourself, but it is worth it to have simple and uniform handling of serialization for all object types. The output is exactly what we wanted:

# __str__ method is called:

{"version": 1, "type": "bounding-box-labels", "boundingBoxes": {"image1.jpg": [{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}, {"label": "car", "x": 20, "y": 20, "width": 5, "height": 11}]}} # json.dump() method calls to_json directly

{"version": 1, "type": "bounding-box-labels", "boundingBoxes": {"image1.jpg": [{"label": "person", "x": 10, "y": 10, "width": 4, "height": 10}, {"label": "car", "x": 20, "y": 20, "width": 5, "height": 11}]}}

Conclusion

Serialization should be easy in Python, but I often stumble odd problems with JSON serialization and thus my reason for writing an article. You can decide whether you want __str__ to behave the same as json.dumps() or you want to have it return the containing objects in double quotations. You now have complete control and knowledge on how to do it. The complete source code is shared here.