# Dynamic Task Scheduling Scheme for Processing Real-Time Stream Data in Storm Environments
## 3 The Proposed Dynamic Task Scheduling Scheme  
### Characteristics  
Conventional schemes have the problem of delay in real-time processing since the redistribution of tasks and increased processing costs of worker nodes due to the computational complexities of the tasks and varying performances of worker nodes in a distributed environment are not considered. The states of worker nodes, task information, and processing costs need to be measured periodically to resolve such a problem. A dynamic task scheduling scheme is required in which the cause of loads on worker nodes and task priorities must be determined based on their information and tasks are redistributed from loaded nodes to free nodes based on the cause of loads.  
In this paper, we propose a dynamic task scheduling scheme in which real-time loads on worker nodes in the Storm environment are considered. The proposed scheme distinguishes the loads on worker nodes based on their usage and throughput of the CPU, free memory size, and network load, following which the tasks are redistributed based on the load to balance the loads on worker nodes. Hence, the stream data are processed in real time to prevent bottlenecks. Figure 1 shows the system architecture of the proposed scheme, which consists of a Nimbus node, which plays the role of the master in the Storm environment, worker nodes, and a zookeeper node, which stores the state information of the worker nodes. The Nimbus node consists of a load table, which stores the information of the state of the worker nodes and the tasks processed in the worker nodes, and a scheduler, which proceeds with scheduling based on the load table. The worker nodes consist of executors that perform tasks and an NMonitor, which inspects the loads on the worker nodes. The workers execute an actual spout or bolt within a thread slot. NMonitor periodically transmits the task information to the zookeeper node. The zookeeper node creates the state information of a respective worker node based on the task information. The load table of the Nimbus node uses the zookeeper information to examine the load state of the worker nodes and redistributes the tasks. In Figure 1, the tasks are concentrated on worker node 1, and the last input task, E, is redistributed to worker node 2 for balancing the loads.  
Figure 2 illustrates the overall processing procedure of the proposed scheme. When stream data are input, Nimbus, which is the master node, checks the loads using a load management module. Static scheduling is performed if the load is below a threshold, whereas dynamic scheduling is performed if the load exceeds the threshold. Static task scheduling uses R-Storm, developed in a previous study [12]. Dynamic scheduling is performed based on the load types, which are memory load, CPU load, and CPU/memory load. Dynamic scheduling redistributes the tasks based on the deadlines of the tasks being processed and allocates the newly input tasks to suitable worker nodes. Each worker node collects the information of the state and the tasks as well as the task throughput in real time. The data are collected as the NMonitor of each worker node sends task information and processing costs to the zookeeper node. Subsequently, the zookeeper node updates the load table of Nimbus every time the load state of a worker node changes based on the delivered information. Nimbus consistently performs static and dynamic scheduling based on the load table.  
Figure 1: Overall architecture of the proposed local event detection scheme.