Many large-scale scientific experiments are carried out in collaboration to leverage expertise and high-tech infrastructures to perform experiments quicker. Data produced by such experiments are thus replicated and cached at multiple geographic locations. This necessitates new techniques for selection of data and compute resources so that execution of applications is time and cost efficient. Existing heuristic techniques select 'best' data source for retrieving data to a compute resource and subsequently process task-resource assignment. However, this approach of scheduling, which is based only on single source data retrieval, may not give time efficient schedules when:1) tasks are interdependent on data, 2) the average size of data processed by most tasks is large, and 3) data transfer time exceeds task computation time by at least one order of magnitude. To achieve time efficient schedules, we leverage the presence of replicated data sources and retrieve data in parallel from multiple locations. In this article, we propose two multisource data retrieval based scheduling heuristic that assigns interdependent tasks to compute resources based on both data retrieval time and task-computation time. We carry out experiments using real applications and deploy them on emulated as well as real testbed environments. With a combination of data retrieval and task-resource mapping technique, we show that our heuristic achieves time-efficient schedules that are better than existing heuristic based techniques for scheduling application workflows.
Pandey et al. address in REF the problem of single source data retrieval, which is not optimal in cases like: tasks are interdependent on data, the average size of data processed by most tasks is large and data transfer time exceeds task computation time.
O nly Scheduling Workflow Applications based on Multi-Source Parallel Data Retrieval in Distributed Computing Networks
Comput. J.
Comput. J.
Computer Science