# Dynamic Task Scheduling Scheme for Processing Real-Time Stream Data in Storm Environments
## 3 The Proposed Dynamic Task Scheduling Scheme
### Collection and Management of Loads  
Collecting and managing the load information of each worker node while processing real-time stream data in a distributed environment are crucial for identifying problems and load type. These processes are also used for identifying and solving various problems such as the CPU usage increase when there is a delay in a task or a task loss due to an increase in the memory usage. Furthermore, managing the loads on worker nodes and processing the costs of tasks, or the task information, are equally important as collecting loads on the worker nodes. This is because the loads on worker nodes become the selection criteria when the tasks are redistributed in a task scheduler. The loads on worker nodes are balanced as the task schedule redistributes the tasks based on the states of worker nodes and task information by collecting and managing the loads.  
Table 1 summarizes the load table managed by Nimbus. The load table manages the CPU, memory, and network load information based on the tasks being processed by the worker nodes. Node\({}_{\text{id}}\) is an identifier of a worker node. CPU\({}_{\text{use}}\) and CPU\({}_{\text{avg-th}}\) represent the current CPU usage and the average throughput of the corresponding worker node. Memory\({}_{\text{total}}\), Memory\({}_{\text{free}}\), and Network\({}_{\text{load}}\) are the total memory size, free memory size, and network load, respectively. The load table is updated at every pre-determined unit (1 s). The CPU usage increases when the computation complexity of the task is high, whereas the memory usage increases when an excessive amount of stream data is input. Accordingly, loads can be examined based on the state of a worker node on which dynamic scheduling can be performed based on the load type.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Notation** & **Definition** \\ \hline Node\({}_{\text{id}}\) & ID of worker node \\ CPU\({}_{\text{use}}\) & CPU usage of worker node \\ CPU\({}_{\text{avg-th}}\) & Average CPU throughput of worker node \\ Memory\({}_{\text{total}}\) & Total memory size of worker node \\ Memory\({}_{\text{free}}\) & Free memory size of worker node \\ Network\({}_{\text{load}}\) & Network load \\ \hline \hline \end{tabular}
\end{table}
Table 1: Load table information.  
Figure 2: Scheduling process.  
Each worker node manages their own CPU usage, throughput, average throughput, and network load. Table 2 presents the information of the NMonitor managed by each worker node. NMonitor maintains the resource usage of the worker nodes used in each task. Task\({}_{\text{id}}\) and N\({}_{\text{id}}\) represent the identifiers of each task and worker node, respectively. Time\({}_{\text{start}}\) represents the starting time of a task and Time\({}_{\text{dline}}\) represents the deadline of the task, such that the task must be completed within \(d\) seconds from the starting time. NMonitor additionally records the resource usage of each task. For resource usage, CPU\({}_{\text{use}}\), Memory\({}_{\text{use}}\), and Network\({}_{\text{use}}\), which are the usages of the CPU, memory, and network are stored. When dynamic scheduling is required, a task to be migrated to a different worker node can be selected based on such information. Similar to the load table, the relevant information is managed by the zookeeper at a particular period to ensure Nimbus can examine the state of a worker node.