Patent ID: 7844976

Claim:
A method for processing data across a distributed network of a plurality of machines, comprising: receiving one or more data items in at least one computing device; filtering the one or more data items at a first package in the at least one computing device, wherein the first package filters the one or more data items based on a user-supplied transform; sorting the filtered one or more data items at a second package in the at least one computing device, wherein the second package sorts the filtered one or more data items based on a designated key; summing frequencies related to the sorted one or more data items at a third package in the at least one computing device, wherein a frequency is a number of times an event occurs for a data item, wherein the third package sums up a number of times the sorted one or more data items are found; and distributing the one or more data items to the plurality of machines using a cluster API, wherein the cluster API uses a source and destination transform in order to distribute the one or more data items to the plurality of machines, and wherein the first, second, and third packages are distributed across the plurality of machines.