PostgreSQL Europe Conference summary

This is a quick recap of some good talks I visited at [pgconfeu 2018](https://www.postgresql.eu/events/pgconfeu2018) in Lisbon in October this year. I went to more than these talks obviously, but that's all the notes I have.

The [conference schedule](https://www.postgresql.eu/events/pgconfeu2018/schedule/) also contains links to slides.

My TOP3 talks are about HyperLogLog algorithm for counting, various index types in PG and spatial analysis with PostGIS.

### Training: Dig the WAL

[Slides](https://www.slideshare.net/loxodata/dig-thewal-pgconfeu2018) [Description](https://www.postgresql.eu/events/pgconfeu2018/sessions/session/2069-dig-the-wal/)

In the half-day training, we were looking into WAL details and how to use this to recover certain catastrophic situations. The speakers had automated examples of recovering clusters using PITR, understanding conflicts with logical replication and using pg_rewind to recover split brain situations.

Apart from the rather high-level discussion, my take aways were

* Logging DDL statements along with txids is tremendously helpful https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5502
* Group commit can considerably reduce amount of fsyncs required https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5503 
* `\gset` is really useful for scripting with `psql`

### TOP3 Talk: Location - the universal foreign key. Past, present and future of spatial PostgreSQL

[Slides](https://docs.google.com/presentation/d/1xyXA4-0wmNX7WfiLeH9h10bIkZxrej278-mMaClagys/edit?usp=sharing) [Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2020-location-the-universal-foreign-key-past-present-and-future-of-spatial-data-in-postgresql/)

The talk was given by Paul Ramsey (himself!) and he beautifully managed to build a bridge for database folks into GIS land. The spatial analysis examples were awesome, for example he correlated starbucks locations with income data and more. If you never worked with spatial data before:

* How to get spatial data: ogr2ogr
* How to visualize it: gqis
* How to load, how to analyze: postgis

Personally, it was a good reminiscence of my bachelor thesis covering different spatial databases.

### TOP3 Talk: Cleaning out crocodiles teeth with PostgreSQL indexes - a story on all the index types in PG

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2119-cleaning-out-crocodiles-teeth-with-postgresql-indexes-a-story-on-all-the-index-types-in-pg/) 

Great talk by Louise Grandjonc (Citus Data): Covered implementation details of b-tree, GIN, GiST, SP-GiST, BRIN and Hash indexes and how they compare.

### TOP3 Talk: The HyperLogLog Algorithm: How it works and why you will love it

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2210-the-hyperloglog-algorithm-how-it-works-and-why-you-will-love-it/) 

Theory behind HyperLogLog algorithm to implement efficient `COUNT(DISTINCT)` queries in PostgreSQL. 

If you always wondered how HLL works - this is the talk to read: [Slides](https://docs.google.com/presentation/d/1Bn_LOUwaZiKAcat_LtyyeqY-NxRohFqo6uJYrMvpnM8/edit) 

### Talk: PostgreSQL worst practices

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2169-postgresql-worst-practices/)

Countless worst practices presented by a database consultant. Excerpts for bingo play:

* Disallow access to the production database to developers. Not even read-only.
* Always use a ORM
* Count everything precisely (this is like all over the place in GitLab, counting is really hard at scale and we have a few anti patterns implemented that do not scale, see https://gitlab.com/gitlab-org/gitlab-ce/issues/52096 and related issues/discussions)
* In-memory joins
* Be in trend, be schemaless, use JSONB
* Be agile, use EAV
* (and many more)

### Talk: Advanced logical replication

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2173-advanced-logical-replication/)

This talk detailed how to implement a transactional, at-least-once message queue using logical decoding in PostgreSQL (based on `pg_logical_emit_message`). This seems relevant to our needs regarding [Geo design](https://gitlab.com/gitlab-org/gitlab-ee/issues/7420) - it would be a great way of introducing a transactional queue without introducing another dependency (to like Kafka or others).

The talk also covered online upgrades ([relevant to us](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/4937)) and bi-directional replication.

### Talk: Around the World With Extensions

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2000-around-the-world-with-extensions/)

Great overview of available extensions including cool stuff like [HyperLogLog counting](https://github.com/citusdata/postgresql-hll), [TOP-N](https://github.com/citusdata/postgresql-topn), [pg_partman](https://github.com/pgpartman/pg_partman) or sharding with citus.

### Talk: High Performance pgBackRest

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2013-high-performance-pgbackrest/)

Details of pgBackRest and how it implements parallelism for backup and restore. Claims to be able to push 20 TB worth of WAL per hour (async parallel push) - if network keeps up with it.

This is relevant to us discussing replacing wal-e for various reasons ([design pending](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5022)).

### Talk: Advanced PostgreSQL backup and recovery methods

[Description](https://www.postgresql.eu/events/pgconfeu2018/schedule/session/2098-advanced-postgresql-backup-and-recovery-methods/)

Great overview of different backup/recover tools available for PostgreSQL and how they compare.

Story points: 1.0