Drastically reduce Sidekiq concurrency (from 15 to 3 threads).

This app's jobs are generally not bound by access to the DB, but rather by the speed of external RSS servers responding. My intention was to have many workers running concurrently sharing just a few DB connections with a very high timeout when retrieving a DB connection from the pool. Supposedly then I could have workers accessing many external servers simultaneously and DB connections in the pool would be shared among them. The high timeout should mean that after getting an RSS XML workers would wait if necessary to access the DB to write any new entries and such. In this process accessing the DB is the fastest step, so the slowdown caused by sharing few connections among many workers shouldn't bee too noticeable.

However I'm getting frequent instances of workers giving a timeout error after 5 seconds attempting to get a connection from the pool, despite the timeout in database.yml being much higher (5 minutes).

I'm not sure of the root cause. It could be that when a thread gets a DB connection, it never returns it to the pool. Or it could be that I'm misinterpreting the meaning of the "timeout" option in database.yml; perhaps it's not the timeout when retrieving a connection from the pool, but rather the timeout for DB operations.

In any case the solution for now seems to be to have as many DB connections in the pool as running workers. I estimate that with my current VPS I can afford 3 connections for Sidekiq (postgresql connections are expensive in memory use!), so I've reduced Sidekiq concurrency to 3 workers.

In the future I'll probably use a native pooling solution like pgbounce. That may support setting a very big pool size in Rails, while actually using just a few DB connections. But that requires some more study and testing; for now, a reduced Sidekiq concurrency seems to be the way to go.