learningequality/ka-lite — pull request #773 by bcipolli (2013-09-28 20:31:19+00:00)
Title: The beauty of counter=None on change...

Background:
Securesync uses "counter" to determine which records to sync. The maximum value "counter" sent over the wire becomes the starting counter for the next round-trip of syncing.

This means that when records are sent over the wire, anything with a lower counter position than what's sent must be sent along with it--otherwise, it will never be requested to be sent in the future. This logic causes dependencies between models, not just for their ForeignKey values, but for their counter values.

Issue:
In both the current version, and develop version of the syncing code, this counter dependency is not taken into account explicitly. This is problematic.

Severity:
Low in the past, high now. Because UserLogSummary records are sent over the wire before ExerciseLog records, but always have higher counters, this becomes an issue. In addition, if a Facility or FacilityUser record were ever edited and saved, this could skip unsynced records. There's some fudge factor, due to "limit" and "boost" in the master code that smooths out some of the issue--but guarantees nothing.

Changes:
* Make sure that if a model is going to be sent over the wire, that all models with a lower counter are sent along with it.
* With the new logic to set counters to None, this actually helps avoid this tremendously. However, if we're sending something over wire with counter=None, then we have to send ALL models with a non-None counter that hasn't been set yet, as updating the None counter to a value would skip over those models.
* To mitigate this, make sure all records that have a deferred counter (Facility*, *Log) have it set to None. Previously, on create it was NOT set to None, as that value is used for calculating the object's ID. To solve this, I double-save the record--once to create the ID, once to eliminate the counter.