alphagov/backdrop — pull request #391 by tombooth (2014-12-15 19:41:26+00:00)
Title: Allow _start_at and _end_at fields on a record

Sometimes we store data that represents an event that happened at a
point in time, sometimes that data represents an aggregate period. We
should be able to store both kinds without having to falsify it by running
a period query over already aggregated data.

We should use the _start_at and _end_at fields as this brings parity
with the result of querying event data with a `period` parameter, as this
outputs synthesized `_start_at` and `_end_at` fields. If one was to query
already aggregated data with `_start_at` and `_end_at` fields already on
the records backdrop will helpfully overwrite these values with whatever
higher level period you have aggregated into.

I talked to @robyoung to get some history as he was one half of team
that pulled this out:

https://github.com/alphagov/backdrop/commit/c88beba20805ef7d1d7929e144b0327928bc93ae

Previously it was accepted to store aggregated chunks of data and
indicate this by using `_start_at` and `_end_at`. After some internal debate
this was switched to using a `period` field, which had the goal of being
used to infer the end of a aggregated record using its `_timestamp` as
the beginning.

While this could have been a reasonable change of direction, all of the
code written both in backdrop and spotlight has been built around using
`_start_at` and _end_at` fields to denote aggregated data points and
only in a few places has this `period` field been used, and only in
collection.