Patent ID: 9147373
Filing Date: 2015-09-29
Classification: G06F,G09G

Abstract:
1. A system for executing a map reduce sequence, comprising: a processor; a map reduce module operable to execute on the processor, and further operable to execute all jobs in the sequence by a collection of a plurality of processes with a process running one or more mappers, combiners, partitioners and reducers for a job respectively, the processes comprising virtual machine processes wherein at least one of the jobs comprises at least a mapper and a reducer run in a same virtual machine process; and an in-memory cache for transparently sharing of heap state between the jobs, wherein the map reduce module is further operable to coordinate completion of map, shuffle and reduce phases, and completion of said all jobs in the sequence, wherein for all of the mappers, a mapper has a tree map for each destination reducer associated with the mapper, the tree map storing data output by the mapper destined for said each destination reducer, and responsive to all of the mappers terminating, merging the tree map of each of the mappers into one tree for said each destination reducer by one or more of the combiners, the tree map buffered as a data structure in the in-memory cache.