Patent ID: 9189290
Filing Date: 2015-11-17
Classification: G06F

Abstract:
1. A method comprising: distributing a plurality of tasks among a plurality of worker nodes to perform a processing operation on an array, each task of the plurality of tasks being associated with a set of at least one data block of the array and an order of the tasks being defined by an array-based programming language, wherein distributing the plurality of tasks comprises: for at least one of the worker nodes, selectively reordering the order defined by the array-based programming language to regulate an amount of data transferred to the worker node; selecting a given worker node of the plurality of worker nodes to perform a given task of the plurality of tasks; communicating at least one block of data associated with the given task with the given worker node; and using the given worker node to determine a format of the communicated at least one block of data, wherein using the given worker node comprises selecting a given candidate format based at least in part on a time estimated by the worker node to perform a format conversion into the given candidate format and a cost of communicating using the candidate format.