Patent ID: 9648095
Date: 2017-05-09
CPC Classifications: G06F,H04L

Claim:
1. A method for processing a plurality of data feeds, the method comprising: acquiring said plurality of data feeds from one or more content servers via a network; adding a respective task of a plurality of tasks to each of said plurality of data feeds, wherein each respective task of said plurality of tasks is a request for processing a data feed of said plurality of data feeds; storing and queuing said plurality of tasks in a persistent queue at a queuing database; fetching each of said plurality of tasks queued at said queuing database by one or more worker servers; asynchronously processing said plurality of tasks by said one or more worker servers; and engaging additional worker servers when traffic load of the tasks increases, and disengaging one or more of the worker servers when the traffic load of the tasks decreases, to maintain a ratio of traffic load of the tasks to number of engaged worker servers; wherein a push mechanism enables acquiring said plurality of data feeds from said one or more content servers; wherein a pull mechanism enables acquiring said plurality of data feeds from said one or more content servers; wherein the persistent queue is created by taking a snapshot of data stored in the queuing database for recovery in event of the queuing database being inaccessible.