Project: springxd
Issue XD-1048: Extend aggregate counter to dynamically aggregate by field values in addition to time.
This would be a combination of the existing aggregate counter and field value counter functionality. For example if the stream data was for car purchases some fields might be colour, make and model. When analysing the aggregate data I dont just want to know how many were sold on Monday, but how many of each make or how many of each colour, or how many of a particular colour, make AND model. This would allow a dashboard type client to 'drill down' into each dimension or combination of dimensions (in real time without executing batch queries against the raw data) Ideally the aggregate counter would be specified as stream create --name mytap --definition "tap:mystream > aggregatecounter --name=mycount --fieldNames=colour,make,model" The keys would be dynamically created according to the field values in each record (ie in a similar way to the field value counter you would not need to predefine field values) and keys would be created for all combinations of the fields specified eg the record { "colour":"silver" , "make":"VW" , "model" : "Golf" } would increment the following key counters (in addtion to the existing time buckets) colour:black make:VW model:Golf colour:black.make:VW colour:black.model:Golf make:VW.model:Golf colour:black.make:VW.model:Golf ie the actual keys would look something like aggregatecounters.mycount.make:VW.model:Golf.201307 etc This may seem like it would generate a lot of key combinations but in practice the data generated will still be massively less than the raw data, and keys will only be created if that combination occurs in a time period. Also some fields may be dependent on each other (such as make and model in the above example) so the amount of possibilites for those composite keys would be a lot less that the number of one times the number of the other.
Story points: 5