Just Released: Google Cloud SQL for PostgreSQL 13
By: James Green
Tags: Database, Google Cloud Platform, Data Science, Postgresql, Postgres

Just Released: Google Cloud SQL for PostgreSQL 13

We explore Cloud SQL PostgreSQL 13 — A fully-managed instance of the world’s most advanced open-source relational database

Image licensed to author

New to PostgreSQL?

PostgreSQL is a powerful, open-source object-relational database system. It shares many commonalities with other relational database platforms, including Microsoft SQL Server and MySQL.

New to Google Cloud SQL?

Part of the Google Cloud Platform ecosystem, Cloud SQL is a fully-managed database service designed to make it easy to set up, manage, and administer relational databases on Google Cloud Platform. At the time of writing, Cloud SQL supports MySQL, PostgreSQL, and SQL Server.

Did you know?

PostgreSQL started out in life in 1986 as a military-sponsored project, codenamed POSTGRES.

Developed at the University of California at Berkeley, the project received significant backing from Defense Advanced Research Projects Agency (DARPA), the Army Research Office (ARO), the National Science Foundation (NSF), and ESL, Inc.

Excerpt from A Brief History of PostgreSQL

PostgreSQL 13

This much-anticipated release appears to be heavily centered around scalability, helping the platform to cope with ever-increasing storage volumes.

PostgreSQL 13 includes significant improvements to its indexing and lookup system that benefit large databases, including space savings and performance gains for indexes, faster response times for queries that use aggregates or partitions, better query planning when using enhanced statistics, and more.

PostgreSQL 13 release note

You can really tell from this, that the engineers at PostgreSQL are going after addressing some of the challenges with query performance and storage size that users with large databases will be facing. And it’s good timing too, as we see more and more clients managing databases that now fall into this “large” category as they ingest more and more data.

Table index optimization

Building on work from the previous PostgreSQL release, PostgreSQL 13 can efficiently handle duplicate data in B-tree indexes, the standard database index. This lowers the overall space usage that B-tree indexes require while improving overall query performance.

PostgreSQL 13 release note

We think this will benefit many users of the platform, as B-tree index duplicates are very common in any relational database. Duplicate data occurs when multiple leaf-nodes in the index (leaf-nodes point to a physical row in a table) contain the same value as another leaf-node and both point to the same physical row.

Duplicates include NULL values, so if you have an index that covers a sparsely populated column, you should see some good benefits from this change.

Deduplication is enabled by default but can be disabled. We note the process is managed lazily, and therefore should not impact DML performance.

A more intelligent query engine

PostgreSQL 13 introduces incremental sorting, where sorted data from an earlier step in a query can accelerate sorting at a later step. Additionally, PostgreSQL can now use the extended statistics system (accessed via CREATE STATISTICS) to create improved plans for queries with OR clauses and IN/ANY lookups over lists.

PostgreSQL 13 release note

Extended statistics

This is going to be a useful tool to have in the box for solving individual queries that are running slowly. Historically, database developers were very much at the mercy of the PostgreSQL optimiser; whilst it’s easy to view execution plans, it was often difficult to prevent the optimiser from making a “mistake” (for example, in underestimating the number of rows returned from an operation). And these can lead to slow running queries that are difficult to tune.

Extended statistics allow database developers and administrators to provide their own statistics, to complement the statistics built offline and served to the optimiser. For example, you can indicate which columns in a table are highly correlated (that is, generally have a 1–1 mapping, such as an employee id and their employee title id).

This is a great article by Jared Rulison, an engineer at Affinity, who shows how his team used extended statistics to “achieve a 3000x speedup”.

Smart sorting

PostgreSQL 13 introduces incremental sorting, where sorted data from an earlier step in a query can accelerate sorting at a later step.

PostgreSQL 13 release note

This one is difficult to quantify in terms of impact, but it’s good to say another improvement to the query engine; DBAs will certainly be pleased and so too, the users of more responsive applications.

More efficient vacuuming

Image licensed to author

Vacuuming is an essential part of PostgreSQL administration, enabling the database to reclaim storage space after rows are updated and deleted. This process can also present administrative challenges, though prior PostgreSQL releases have done work to ease the overhead of vacuuming. PostgreSQL 13 continues to improve the vacuuming system with the introduction of parallelized vacuum for indexes. In addition to the vacuuming performance benefits it offers, the use of this new feature can be tuned to specific workloads as administrators can select the number of parallel workers to run. In addition to these performance benefits, data inserts can now trigger the autovacuum process.

PostgreSQL 13 release note

These kinds of administrative tasks can quickly become a burden on DBA teams. I remember during my many years developing on MS SQLServer, frequently approaching DBAs for performance tuning help, for example by running overnight index rebuilds.

So I’m sure DBAs will be pleased to see the autovacum process, one less thing to administer. Great to see the overall vacum process has been optimised too.

We are spoilt working with modern analytic platforms such as Google BigQuery, which are zero-ops. This means, we no longer have to spend any time doing administrative operations like this; in fact, we don’t even have to create indexes anymore! I wonder how long it will be for relational data stores to catch up?

New data types, more flexible partitioning

Partitioning has received a boost, and partitioned tables can now be replicated. Great news for large multi-instance Enterprise deployments. Also, partitioned tables can now support BEFORE row-level triggers. Whilst not frequently used, these are handy, for example, to enforce business logic that prevents certain updates from taking place on a table.

PostgreSQL 13 includes a new DateTime() function to convert (valid) strings to a native type. A small change, but should make code easier to read. The GEN_RANDOM_UUID() is now callable without the need to install additional extensions. One last thing to ask DBAs to do!

Security enhancements

Previously, PostgreSQL extensions had to be installed by a database superuser. In 13, this has been relaxed slightly, and superusers can now mark an extension as “trusted”, allowing that extension to be installed by non-superusers.

This should please DBAs, who can now (safely) delegate to development teams and reduce the number of incoming requests to their teams.

A good set of features in all. Let’s hope 13 doesn’t live up to its name :)

“PostgreSQL 13 showcases the collaboration and dedication of our global community in furthering the abilities of the world’s most advanced open source relational database. The innovations that each release brings along with its reputation for reliability and stability is the reason why more people choose to use PostgreSQL for their applications.”

Peter Eisentraut, a PostgreSQL Core Team member

Get started with PostgreSQL on Google Cloud SQL

If you are a data scientist, developer, or engineer looking for a fully-managed, Cloud-based instance of PostgreSQL, it’s really easy to get started with Cloud SQL.

In fact, it can be free too…

“New customers get $300 in free credits to spend on Google Cloud during the first 90 days.”

The $300 can be used for lots of other services too, such as Google BigQuery. You also get to use the full products too, so really is a great way to try out the platform.

What happens when my $300 runs out?

You can review the Cloud SQL pricing to get an idea of your monthly spend.

Why Google Cloud SQL?

There are many good reasons to use Cloud SQL to manage your PostgreSQL instance. We have drawn out the 3 that we think are most compelling, taken from the official Google Cloud SQL documentation.

Secure and compliant. For peace of mind with sensitive data, all Cloud SQL data is encrypted at rest and in transit. Private connectivity with Virtual Private Cloud and user-controlled network access with firewall protection. Compliant with SSAE 16, ISO 27001, PCI DSS, and HIPAA. Scale as you go. Scale your instances effortlessly with a single API call, whether you start with simple testing or you need a highly available database in production. Set up in minutes. Standard connection drivers and built-in migrational tools allow you to create and connect to your first database in just a few minutes.

How to set up PostgreSQL in Cloud SQL

We finish with a short showcase, so you can see for yourself, just how simple creating a new instance is. These steps are from the Google Get Started documentation, linked here:

Before you start

1. You will first need a Google Cloud project. A project can contain any of the services you wish to take advantage of and not just Cloud SQL.

To create a project, in a browser launch the Google Cloud console and select Create Project

2. Make sure that billing is enabled for your Google Cloud project. Learn how to confirm billing is enabled for your project

3. Make sure you have the Cloud SQL Admin and Compute Viewer roles on your user account. Go to the IAM page

Learn more about PostgreSQL roles and permissions in Google Cloud.

Create a PostgreSQL instance in Cloud SQL

1. Go to the Cloud SQL Instances page in the Google Cloud Console

2. Click Create instance, shown below in blue

Cloud SQL Instance Page (author image)

3. Select PostgreSQL and then click Next

4. Enter a name for your new instance

5. Enter a password for the postgres user

6. Under Location, chose a zone and region. We always advise choosing a region local to you.

7. Under Configuration options, update settings for your instance. Learn more about instance settings

8. Click Create

And you are done!

Image licensed to author

Next steps

1. Read the Google Cloud SQL PostgreSQL release note

2. Read the Google Cloud SQL documentation

3. Learn more about Ancoris Data, Analytics & AI

4. Connect with the author