Load users from DB on demand like tweets, rather than all at start up.
Change udc_ptr to intrusive_ptr, with eviction policy same as tweets.
Read in all user IDs at start up.
Read in all user objects with DM sets, and account users at start up.
Mark users loaded at startup and DM users non-purgable.
Support deferred parsing of user objects.
Various minor changes to support above.
Change dbconn::SyncPurgeProfileImages to scan DB, run at most once per day.
De-duplicate functionality shared with dbconn::SyncPurgeMediaEntities.
DB memory management improvements.
Use smart-pointer type abstractions to handle freeing memory
and tracking buffer sizes.
Reduce the number of allocations/copies that need to be made.
Give compressed and uncompressed data distinct types to avoid
mixups and improve type safety.
Use insertion hints for DB loads of sorted ID sets.
Significantly speeds up ID set loading at start up.
Use a varint encoding of diffs for compressed DB sets instead of deflate.
Varint encoding is 2 bits in upper position of first byte,
signals 30, 38, 46 or 64 bit integer using 4, 5, 6 or 9 bytes
respectively. This choice of sizes is due to analysis of sample data,
(IDs of all tweets in test DB), ~99% of diffs fitted in 5 or 6 bytes.
Uses zigzag of incremental diff for not strictly sorted sets.
Uses direct encoding of decremental diff for sorted descending sets.
This is much faster than deflate, and gives about the same performance
of ~6 bytes/id.
Bump DB version.
No longer write UserCountHint to settings table.
It is not used anywhere.
Cache the set of all tweet IDs in the DB as a blob.
This is to speed up startup time by avoiding a table scan.