How to Boost PostgreSQL Cache Performance
By: Calvin Li
Tags: Database, Postgresql, Performance, Postgres, Cache

How to Boost PostgreSQL Cache Performance

When thinking about caching data for incredibly fast retrieval, technologies like Redis and Memcached often come to a developer’s mind. I, however, want to discuss a less commonly acknowledged cache that also helps with improving data retrieval time: a cache within a database.

The ability to fetch data quickly is crucial for any high throughput application and service. For the Notifications team at Redfin (the team I work on), it ensures that we can capture the updates in the US and Canadian real estate markets and inform our users within minutes of them occurring. A difference of several tens of milliseconds in process time per update event is enough to disrupt our pipelines and prevent us from delivering updates to our users in a timely manner.

In fact, that was exactly what happened to our email and push sending pipelines earlier last year. They couldn’t process the events quickly enough during times of peak traffic, so notifications were delayed to our customers. After some investigation, we discovered that the main bottleneck was the performance of retrieving data from our PostgreSQL database replica.

Although introducing external caching stores like Redis or Memcached to our applications would have been beneficial, we wanted to make sure that we were properly utilizing our database. In particular, we discovered in our investigation that our configurations for our database could have been modified for improved performance. Let’s walk through some of the techniques we did to discover and fix this.

Investigating bottlenecks in the query performance

Before looking for a solution, it is important to first understand why the queries are slow. There can be an assortment of reasons why, and some of them may not be related to the database. To narrow down the possibilities, you should look into the general health of the machines your applications are running on and the performance of all the various queries being made.

For instance, any CPU or network bottlenecks on the machine your application is running on can negatively impact its ability to quickly fetch data from the database. Other possibilities are that the queries are poorly written or that the tables don’t have certain indices that can help speed up those queries. If only one or a few queries are slow, that’s indicative that improving those queries will resolve the problem. If all of the queries are slow, even primary key lookups, then that’s a strong signal that the problem is unrelated to the query.

In the case that all queries are slow, you should examine the health of the machine that the database runs on. Bottlenecks in the machine’s computational power or I/O means that you will need to scale up your database. Good ways to examine this is to look at the CPU utilization of the machine, disk I/O, and network I/O.

One way to determine whether or not your database is maxing out on its processing ability is to take samples of the number of active queries running. To sample the number of active queries running on our PostgreSQL database, we can query the pg_stat_activity table:

select count(*) from pg_stat_activity where state='active'

If that number consistently exceeds the number of CPU cores of the machine, then that’s a good sign that your database is under-resourced.

Investigating the database’s internal cache performance

If your investigation leads you to believe that there are database performance issues, then you should consider analyzing the cache performance of your database. PostgreSQL databases use both an internal cache and the machine’s page cache for storing commonly requested data. That way, instead of pulling the data from disk, which can take milliseconds, the database replica can fetch the data in-memory in sub-millisecond times.

Postgres provides cache hit rate statistics for all tables in the database in the pg_statio_user_tables table. The two useful columns in that table are the heap_blks_read , defined as the “number of disk blocks read from this table” and the heap_blks_hit , defined as the “number of buffer hits in this table”. If we run the following query,

select heap_blks_read, heap_blks_hit from pg_statio_user_tables where relname='<table_name>';

we get the number of disk block reads and buffer hits for that given table since the creation of the table. If we store those values and run the above query again after some period of time, we can calculate the cache hit rate for that time period. To simplify doing the math every time, you can run the below query:

select

heap_blks_read — <result from prev heap_blks_read> as misses,

heap_blks_hit — <result from prev heap_blks_hit> as hit,

(heap_blks_hit — <result from prev heap_blks_hit>)::float/(heap_blks_read — <result from prev heap_blks_read> + heap_blks_hit — <result from prev heap_blks_hit>) as hit_ratio

from pg_statio_user_tables

where relname='<table_name>';

Ideally, you will want the hit_ratio to be greater than 90%. Otherwise, you may need to tune your database configurations to improve the cache hit rate.

Improving Postgres internal cache performance

One of the main reasons why the cache hit rate for a table is low is that the database doesn’t have enough space allocated in its internal cache buffer. This internal cache buffer is used for loading the table rows from disk to memory, and if there isn’t enough space allocated, then the database server will constantly have to fetch the data from disk.

For PostgreSQL databases, the cache buffer size is configured with the shared_buffer configuration. It tells the database how much of the machine’s memory it can allocate for storing data in memory. The default is incredibly low (128 MB) because some kernels do not support more without changing the kernel settings. The optimal value ultimately depends on the data access patterns for a given database, but PostgreSQL recommends that you should initially configure it to 25% of the machine’s total memory if you are using a dedicated database server. More memory can be beneficial for larger databases, but there’s a limit to the optimization since PostgreSQL databases also use the system’s cache. The more memory you allocate to the shared_buffer, the less memory will be available for the machine’s cache.

Final Thoughts

There are also many other database configurations, such as the effective_cache_size , that are worth looking into and tuning if you still are seeing database performance issues. The optimal values for those configurations will depend on the data access patterns of the database, so having a good understanding of what tables are commonly queried and rows commonly fetched can help with guiding the decisions for those configurations. Ultimately, knowing how to tune them or whether they will have a significant impact is hard to say, so it’s always good to test some changes out and see their effects.

As mentioned earlier, a possible alternative would be to place an external cache (e.g. Redis, Memcached) in front of the database. It may come with the additional cost of maintaining the cache operationally or concerns with fetching data. However, for services and applications that share commonly fetched data, it can be a quick and cheap way of reducing data fetching latency.