alphagov/transition — pull request #172 by jennyd (2014-02-06 13:51:51+00:00)
Title: Update counts on existing hits rows if data changed

When we process logs, we will have data for partial days at the ends
of periods covered because the logs will not have been rotated at
exactly midnight. When subsequent days' logs are processed,
pre-transition-stats will update the hits file containing the partial
day's data with the full day's data, so long as the intermediate
cache file is not deleted between processing runs. This change to the
hits import allows us to re-import hits files containing updated counts
and have the counts on the already-imported rows updated, rather than
no change being made to rows which already exist.

This change appears to make the hits import take a couple of seconds
longer to run, but it's a pretty insignificant difference.

It's worth noting that both INSERT IGNORE and ON DUPLICATE KEY UPDATE
are MySQL-specific, so this change doesn't make it any harder to
move to PostGRES than it was before.