Project: springxd
Issue XD-1047: Allow Aggregate Counter to use timestamp field in data.
Currently the aggregate counter aggerates by the current time. However the data may already have a timestamp in it (eg streams from activity events on a website). It would be useful as an alternative approach to be able to specify this field to aggregate on. This would have the following benefits: 1) The aggregate counts would be more accurate as they would reflect the acutal event times and not have any lag from an intermediate messgaging system they might have passed through. 2) If for whatever reason XD is down, comes back up and starts pulling queued messages from the messaging system the aggregate counter will reflect the correct event time. Currently you would get a gap and then a spike as a backlog of messages would get allocated to the current aggregate count. 3) Old data could be rerun through XD still creating the correct aggregate counts. Configuration would be something like stream create --name mytap --definition "tap:mystream > aggregatecounter --name=mycount --timestampField=eventtime" without the timestampfield it would behave as currently.
Story points: 5