Patent ID: 8949810
Filing Date: 2015-02-03
Classification: G06F

Abstract:
1. A method for optimizing data stream processing, comprising the steps of: employing a split, aggregate and join architectural pattern on an application, wherein the pattern facilitates splitting of one or more incoming streams and distributing processing of the application across multiple per-group operators, wherein each of the one or more incoming streams contain data belonging to multiple groups multiplexed together, and wherein the number of groups is not known a priori; employing the multiple per-group operators to perform multiple simultaneous group-independent split, aggregation and join operations on the one or more streams via (i) de-multiplexing the data from the one or more incoming streams into multiple physical streams, each physical stream corresponding to one of the multiple groups, (ii) aggregating data from the multiple groups to form multiple aggregates and (iii) joining the multiple aggregates with additional data incoming from a same and one or more other groups, and wherein said employing the multiple per-group operators comprises applying a distinct window for performing the simultaneous group-independent split, aggregation and join operations for each of the multiple groups, wherein each said distinct window triggers performance of said operations and termination of the performance of said operations based on the tuples that have been accumulated for the corresponding group; generating code, wherein the code facilitates mapping of the application onto a computational infrastructure to enable workload partitioning; using the multiple per-group operators to decompose the application into one or more granular components; using the code to reassemble the one or more granular components into one or more deployable blocks to map the application to the computational infrastructure, wherein reassembling the one or more granular components to map the application to the computational infrastructure optimizes data stream processing of the application; and refining the level of granularity associated with the one or more granular components based on said computational infrastructure.