Working with a JSONB Array of Objects in PostgreSQL
By: Gravity Well, Rob Tomlin
Tags: Api Development, Database, Postgres, NoSQL, Json

Let’s Get Started!

Building Our Data

Open pgAdmin and create a database as desired. Right-Click on the database name and choose Query Tool. Run the snippet below to create a simple table that will have an id, purchaser name and a jsonb column that stores an array of json objects, which will store items purchased.

CREATE TABLE public.purchases

(

id serial PRIMARY KEY,

purchaser varchar(50),

items_purchased jsonb

);

4. Run the snippet below to insert four records in to the table.

INSERT INTO purchases (purchaser,items_purchased) VALUES ('Bob',

'[{

"productid": 1,

"name": "Dell 123 Laptop Computer",

"price": 1300 },

{

"productid": 2,

"name": "Mechanical Keyboard",

"price": 120 }

]'); INSERT INTO purchases (purchaser,items_purchased) VALUES('Carol',

'[{

"productid": 3,

"name": "Virtual Keyboard",

"price": 150 }, {

"productid": 1,

"name": "Dell 123 Laptop Computer",

"price": 1300 },

{

"productid": 8,

"name": "LG Ultrawide Monitor",

"price": 190 }

]'); INSERT INTO purchases (purchaser,items_purchased) VALUES ('Ted',

'[{

"productid": 6,

"name": "Ergonomic Keyboard",

"price": 90 },

{

"productid": 7,

"name": "Dell 789 Desktop Computer",

"price": 120 }

]'); INSERT INTO purchases (purchaser,items_purchased) VALUES('Alice',

'[{

"productid": 7,

"name": "Dell 789 Desktop Computer",

"price": 120 },

{

"productid": 2,

"name": "Mechanical Keyboard",

"price": 120 }

]');

5. To see our data, run the query below.

SELECT * FROM purchases;

purchases with a jsonb column of items purchased

Our Queries

Getting All Items Purchased For An Account

This is the most straightforward query.

Run the query below to get all items purchased by Ted.

SELECT items_purchased FROM purchases

WHERE id=3

Array of json objects

Front-End

Thinking of this an API end-point result, this is perfect for the front-end developer to use Array.map(), Array.filter() or Array.forEach() and others operations for processing on the front-end.

Getting A Specific Item Purchased For An Account

Here we have several options depending on the request.

Looking at Carol, id=2, we see she has purchased,

[

{

"name": "Virtual Keyboard",

"price": 150,

"productid": 3

},

{

"name": "Dell 123 Laptop Computer",

"price": 1300,

"productid": 1

},

{

"name": "LG Ultrawide Monitor",

"price": 190,

"productid": 8

}

]

Let’s start by running the query below and examining what we get.

SELECT arr.position,arr.item_object

FROM purchases,

jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2;

Pulling each json object out along with it’s ordinal position in the array

This gives us each json object (item purchased) out along with it’s ordinal position in the array.

jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

We passed to jsonb_array_elements the jsonb column with an array of objects. This breaks the array in to separate json objects.

the jsonb column with an array of objects. This breaks the array in to separate json objects. We included with ordinality . This tells PostgreSQL to include the ordinal position. Note this starts at 1. But when dealing with array, it starts at 0.

. This tells PostgreSQL to include the ordinal position. Note this starts at 1. But when dealing with array, it starts at 0. The name “arr” is arbitrary.

The name “item_object” is arbitrary and references a purchase json object in our case.

The word position is a key word.

is a key word. We can access the position and the item_object by using the syntax, arr.position and arr.item_object. We actually do not need to include the “arr” prefix, we can just use position and item_object.

More on jsonb_to_array() may be found here.

Now that we see how the data is transformed, if we want the second purchase by Carol, we simply create a Where clause specifying the ordinal position in the array.

2. If we know the ordinal position in the Array, we can just specify it in the Where Clause.

Run the query below to get just the second item purchased by Carol, id=2.

SELECT arr.item_object

FROM purchases,jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2 and arr.position=2;

Second item purchased

3. If we do not know the position but we know the productid (or any Key) we can use a sub query that finds the ordinal position.

Run the query below to produce the same result as above.

SELECT arr.item_object

FROM purchases,jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2 and arr.position=Cast((select arr.position FROM purchases, jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2 and arr.item_object->>'productid' = '1') as int)

The ->> operator will return the value of the Key (productid) contained in item_object as text. That is why we use ‘1’ even though productid is a number.

Even though the ordinal position is a bigint, we must Cast it to int or bigint.

Front-End

Thinking of this an API end-point result, this is perfect for the front-end developer. We get back a nice clean json object they can manipulate or display.

4. As a bonus we will incorporate a technique from my last article to get the highest price item purchased by Carol, id=2.

SELECT arr.item_object,arr.item_object->>'price'

FROM purchases,

jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2

and arr.item_object->>'price' = Cast((Select max(item_prices.price) as p

FROM purchases,jsonb_to_recordset(purchases.items_purchased) as item_prices(price int)

WHERE id=2) as varchar);

I have included “arr.item_object’->>’price’ in the Select clause so we can see that the data type returned is text. Hence the need for the Cast() function.

Highest priced item

Without explaining too much from my prior article, the jsonb_to_recordset() function allows us to extract data out in to relational columns, against which we can query.

More on jsonb operators can be found here.

Adding A New Purchase To An Account

If we look at Bob’s purchases, we see he has purchased,

[

{

"name": "Dell 123 Laptop Computer",

"price": 1300,

"productid": 1

},

{

"name": "Mechanical Keyboard",

"price": 120,

"productid": 2

}

]

Bob is about to purchase a monitor.

{

"name": "LG Ultrawide Monitor",

"price": 190,

"productid": 8

}

Run the query below to add a purchase to the items_purchased array for Bob, id=1.

UPDATE purchases SET items_purchased = items_purchased ||

'{"name": "LG Ultrawide Monitor",

"price": 190,

"productid": 8}' ::jsonb

WHERE id=1;

Bob has now purchased a monitor.

Bob’s new purchase

Here we have used the concatenation operator, ||, to concatenate on to the array a jsonb objects.

Front-End

Thinking of this an API end-point, this is perfect use for an HTTP PATCH request. The front-end sends a PATCH request and the json object to be added in the Body.

2. Special case — no existing purchases. If items_purchased is null, we need to use an additional function, COALESCE.

UPDATE purchases SET items_purchased = COALESCE(items_purchased, '[]'::jsonb) ||

'{"name": "LG Ultrawide Monitor",

"price": 190,

"productid": 8}' ::jsonb

WHERE id=1;

We often use the COALESCE function to substitute a default value for null values when we querying the data

In our example, we substituted and empty array and then concatenated because you cannot concatenate on to null.

More on COALESCE can be found here.

Removing A Purchase From An Account

Important Note: Previously we used “with ordinality” to get “position” in the array. This works fine when using the jsonb_array_elements() function. But it is important to note that while this ordinal value starts with the number 1, a jsonb array of objects starts at 0.

When the position in the array is known, this is as simple as using the ‘-’ operator.

The “-” operator description in the documentation is a bit vague in my opinion.

Delete key/value pair or string element from left operand. Key/value pairs are matched based on their key value.

Bob returned his Mechanical Keyboard. Let’s remove item number 2 from his items_purchased. First query for Bob’s items to see he has three, and then remove item 2.

SELECT * FROM purchases where id=1;

Bob has three items

Item number 2, Mechanical Keyboard, is at array position 1.

UPDATE purchases SET items_purchased = items_purchased - 1

WHERE id=1;

Bob without item number 2 or array position 1

What if the position is not known? Maybe the productid is known. To our rescue comes a previously used query.

The query below gives the ordinal position of the Dell 123 Laptop, productid=1, for Carol, id=2.

SELECT position FROM purchases, jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2 and item_object->>'productid' = '1';

This will return the number 2 since the Dell 123 Laptop for Carol is in the second position in the array. However, if we use this to help us remove an item, it will remove the third item since, as mention in the note above, the array position starts with 0. The solution is easy, just use position-1

2. Finding the position of the item in the array and then removing the item.

Carol. Dell 123 Laptop in ordinal position 2 but array position 1.

UPDATE purchases SET items_purchased = items_purchased -

Cast((SELECT position - 1 FROM purchases, jsonb_array_elements(items_purchased) with ordinality arr(item_object, position)

WHERE id=2 and item_object->>'productid' = '1') as int)

WHERE id=2;

Carol without productid=1. This had ordinal position 2 but array position 1.

We used a a query to find the position in Carol’s items_purchased for productid=1. This returned ordinal position 2. Therefore we used position-1 to get array location 1.

Even though the ordinal position is a bigint, we must Cast it to int or bigint.

Also recall that the ->> returns a string so in item_object->>’productid’ = ‘1’ we compare against the string version of the productid, even though it is a number in the json.

Indexing

Based on a couple of questions from readers, I have added indexing the items_purchased column and a Key value in the JSON.

Indexing the Column

CREATE INDEX idx_purchases ON public.purchases (items_purchased);

This creates a btree index.

Indexing a Key in the JSONB Column

For a specific Key, name here, in the field use,

CREATE INDEX idx_purchases_name

ON public.purchases

((items_purchased ->> ‘name’));

This creates a btree index on name.

In our small data set you may not see the index being used when you do an Explain Analyze.

When an index is used in a SELECT query, first the position of the requested rows is fetched from the index (instead of from the table directly). Then, the requested rows (a subset of the total) are actually fetched from the table. It’s a two step process. Given a large enough number of requested rows (such as all the posts with an id greater than 10), it is actually more efficient to go fetch all those rows from the table directly rather than to go through the extra step of asking the index for the position of those rows and next fetch those specific rows from the table. If the percentage of the rows is smaller than 5-10% of all rows in the table the benefit of using the information stored in the index outweighs the additional intermediate step.

Full article here.

Conclusion

To recap. In this article we continued the work done in the prior article of querying the jsonb array of objects as a recordset by adding the ability to get specific array objects, add a new array object and remove array objects.

PostgreSQL has many functions and operators for JSON data and we will explore some of these others in future articles.

I am using them everyday as I build a RESTful API for a PERN stack.

If you found this useful, start exploring json, jsonb and the other operators and functions for JSON as well as the regular SQL operations available in PostgreSQL.

Thank you for reading and coding along!

You may also enjoy,