diaspora/diaspora — pull request #5217 by jhass (2014-09-13 23:38:44+00:00)
Title: [WIP] Global hydra

Try reducing sidekiq/typhoeus/http_multi memory usage

* Only fetch needed attributes of persons
* Discard person objects as early as possible
* Use only one global hydra instance
* Limit number of cached connections

While being highly experimental, this seems to run stable on my pod for the last two days (in combination with #5209).

Only ever allocating one hydra and limiting the number of connections cached by it drastically reduces and stabilizes memory usage for usage patterns on my pod, I'm stable under 300M RSS now. Though this has the potential to totally wreck outgoing federation performance on different (= higher) load patterns, if this goes in the settings for `sidekiq_concurrency`, `typhoeus_concurrency` and the new `typhoeus_connects` likely need adjusted defaults. Figuring out sensible defaults for these is the hard part of this PR.

Another possible optimization is detecting if one Sidekiq worker is already driving the hydra and not blocking additional ones after they submitted the additional requests. This requires figuring out a reliable way of detecting whether the hydra is already running after the new requests have been queued. That would also requires a complete rework of the retry logic, so I'd like to avoid it if doable.

I'm not sure what more testing could be done from my side since I have no other setups available.