Simple guide to Postgres Partitioning
By: Rishi Sharma
Tags: Database, Postgresql, Sql, Query Optimization, Query Performance

In this article, we’ll be using PostgreSQL 11.

Most of the systems which are built are guided by some process or flows. Each process can have multiple stages/statuses, like when we initiate a process then status might be START and then it move to IN_PROGRESS and multiple intermediary status and then finally to DONE / COMPLETE status. So for general purpose here we’ll take table name as process .

Note: Never keep table name as process as it’s a verb

We’ll start with creating 2 tables:

process — Normal table process_partition — Partition table with partition key as status

Both these process tables will contain

id — Auto-incremented id name status — possible values for status OPEN , IN_PROGRESS and DONE

Let’s start by creating process table first

CREATE TABLE process (

id bigserial,

name character varying(255) ,

status character varying(255) ,

CONSTRAINT process_pk_id PRIMARY KEY (id)

);

Now, we’ll create process_partition table

Partitioning of process_partition table based on status

CREATE TABLE process_partition (

id bigserial,

name character varying(255) ,

status character varying(255) NOT NULL,

CONSTRAINT process_partition_pk_id PRIMARY KEY (id, status)

) PARTITION BY LIST (status);



-- Partitions SQL



CREATE TABLE process_partition_done PARTITION OF process_partition

FOR VALUES IN ('DONE');



CREATE TABLE process_partition_in_progress PARTITION OF process_partition

FOR VALUES IN ('IN_PROGRESS');



CREATE TABLE process_partition_open PARTITION OF process_partition

FOR VALUES IN ('OPEN');

Here we have created a master table process_partition and perform a list partition on it with partition key as status . We have created 3 partition of the master table based on each status, process_partition_open for OPEN , process_partition_in_progress for IN_PROGRESS and process_partition_done for DONE status.

If we want to use status as partition key, then we are forced to add it in primary key also. Similarly if we have any Unique key constraint, then we’ll need to add the partition key there also. If we don’t, then we’ll get error while creating table.

Error : partition key is not added in primary key

Now, we’ll add around 10000 rows first in each status of the 3 status.

DO $$

BEGIN

FOR i in 1..10000 LOOP

INSERT INTO process(name, status) VALUES ('Process Name', 'OPEN');

INSERT INTO process(name, status) VALUES ('Process Name', 'IN_PROGRESS');

INSERT INTO process(name, status) VALUES ('Process Name', 'DONE');

END LOOP;

END; $$

30000 rows added in process table

Similarly we’ll add in process_partition table also.

DO $$

BEGIN

FOR i in 1..10000 LOOP

INSERT INTO process_partition(name, status) VALUES ('Process Name', 'OPEN');

INSERT INTO process_partition(name, status) VALUES ('Process Name', 'IN_PROGRESS');

INSERT INTO process_partition(name, status) VALUES ('Process Name', 'DONE');

END LOOP;

END; $$

30000 rows added in process_partion table

10000 rows in process_partition_open for open status

process_partition table has 0 rows

Here we see that, when we count only process_partition table then there are 0 rows. This is because all the rows which we inserted are split into 3 partition tables process_partition_open , process_partition_in_progress and process_partition_done .

So here we saw that we executed insert statement on the master table process_partition , but based on the value of the status column each row is added to respective partition and total rows of master table are the aggregation of all partition tables.

So now let’s insert a new row and then try to change status and observe the behaviour.

INSERT INTO process_partition(name, status) VALUES ('Moving process', 'OPEN');

Moving process row is added to process_partition_open with id 30001

We add this row to master table which has id : 30001 and as it’s in OPEN status, so it automatically moves to process_partition_open .

Now, we’ll update this row to IN_PROGRESS status.

UPDATE process_partition SET status = 'IN_PROGRESS' WHERE name = 'Moving process' AND status = 'OPEN';

after changing status row moves to process_partition_in_progress

Now, if we query the process_partition_open table that row is not there instead that row has moved to process_partition_in_progress table. So, we observe that there is movement of rows from one partition to other when there is change in status, i.e partition key.

Below are the results for 30000 rows and when data is increased 10x times to 300000 rows.