The Postgres Performance Tuning Manual: Query Plans
By: Kishan Sharma
Tags: Postgres, How To, Tech, Database, Postgresql

Postgres is one of the most widely used open source databases in the world. At GOJEK, a lot of our products depend on Postgres as well. However, when you’re a building and operating a Super App at scale, the volume of data passing through the pipelines can slow down even the most efficient systems.

Sometimes, we witness degraded performance on Postgres too. That’s not a sustainable thing for a startup like GOJEK.

We needed to optimise things to enhance performance. So we targeted the optimizer.

The optimizer is the brain of the database, which interprets queries and determines the fastest method of execution. A single query optimization technique can increase database performance drastically.

Postgres query lifecycle

This post outlines how to analyse Postgres performance using tools such as EXPLAIN and ANALYZE (which Postgres provides).

Meet EXPLAIN & ANALYZE

EXPLAIN gives an exact breakdown of a query. The execution plan is based on the statistics about the table, and it identifies the most efficient path to the data. This takes different database paradigms (such as indexes) into consideration. EXPLAIN only guesses a plan that it thinks it will execute.

This is where ANALYZE comes into the picture. ANALYZE basically runs a query to find the processing time to execute a query.

To quickly summarise, EXPLAIN and ANALYZE commands help improve database performance in Postgres by:

a) Displaying the execution plan that the PostgreSQL planner generates for the supplied statement b) Actually running the command to show the run time.

Finding the framework

Let’s consider we have a table named schemes .

EXPLAIN SELECT * FROM schemes;

QUERY PLAN | Seq Scan on schemes (cost=0.00..64 rows=328 width=479)

SELECT * FROM pg_class where relname = 'schemes';

pg_class has the metadata about the tables.

Cost in the query plan is calculated using the following formula

COST = (disk read pages * seq_page_cost) + (rows scanned * cpu_tuple_cost).

Disk read pages and rows scanned are the properties of pg_class .

. Seq page cost is the estimated cost of disk read fetch.

Cpu tuple cost is the estimated cost of processing each row.

e.g. COST (54 * 1.0) + (1000 * .01) = 64

Let’s see how the query plan changes when we apply filters in the select statement.

EXPLAIN SELECT * FROM schemes WHERE status = ‘active’;

QUERY PLAN | Seq Scan on schemes (cost=0.00..66.50 rows=960 width=384) Filter: ((status)::text = ‘active’::text)

The estimated cost here is higher than the previous query, this is because Postgres is performing a seq scan over 1000 rows first and then filtering out the rows based on the WHERE clause.

Planner cost constants

The cost variables described in this section are measured on an arbitrary scale. Only their relative values matter, hence scaling them all up or down by the same factor will result in no change in the planner’s choices. By default, these cost variables are based on the cost of sequential page fetches; that is, seq_page_cost is conventionally set to 1.0 and the other cost variables are set with reference to that. But you can use a different scale if you prefer, such as actual execution times in milliseconds on a particular machine.

Unfortunately, there is no well-defined method for determining ideal values for the cost variables. They are best treated as averages over the entire mix of queries that a particular installation will receive. This means that changing them on the basis of just a few experiments is very risky. (Refer planner cost constants).

EXPLAIN SELECT schemes.rules FROM scheme_rules JOIN schemes ON (scheme_rules.scheme_id = schemes.id ) where scheme_name = 'weekend_scheme';

The query planner sometimes decides to use a two-step plan. The reason for using two-plan node is the first plan sorts the row locations identified by the index into physical order before reading them, and the other plan actually fetches those rows from the table.