## 4 System and Problem Definition for Network Aware Server Consolidation
### Optimizing the Inter-Consolidation Time  
Due to the arrival of new tasks and the completion of existing tasks, the consolidated utilization profile of the servers may change over time. Therefore, the power consumption of the HPC data center may be adversely affected over time. Hence, the BCC consolidation algorithm should be repeated periodically. Repeating the consolidation too often might not reduce the total power consumption enough to justify the additional network traffic introduced as a result of the consolidation. On the contrary, delaying the consolidation can adversely affect the potential opportunity to save power. Hence, to determine the optimal time interval between two consecutive consolations, an appropriate cost function, to capture the trade-off between power savings and network traffic is required. We define the expected value of the time-dependent cost function \(C(t)\) for inter consolidation time interval as  
\[C(t)=\kappa E[A(t)]-E[B(t)], \tag{6}\]  
where \(A(t)\) is migration cost related to the network traffic which represents the total traffic movement for the consolidation operation, \(B(t)\) is the total power saving due to the consolidation, \(t\) represents the time interval between two consecutive consolidation operation, and \(\kappa\) is a scaling constant which captures the relative significance of network traffic and power savings. \(E[\cdot]\) represents the expected value and is necessary as random task arrivals and completion make \(A(t)\) and \(B(t)\) random processes. At the optimal inter-consolidation time interval of \(t^{*}\), the cost \(C(t)\) should have the minimum value, that is,  
\[t^{*}=\operatorname*{argmin}_{t\in\mathbb{R}}C(t) \tag{7}\]  
Figure 3 represents the timeline for the consolidation operation which shows two consecutive consolidation operations. \(\mathbf{u}\) denotes the utilization profile of all the servers in the HPC data center, where \(\mathbf{u}=[u_{1},u_{2},...,u_{N}]^{T}\in\mathbb{R}_{+}^{N}\) if \(N\) is the total number of servers in the data center. At time \(t_{0}\), when the utilization profile is \(\mathbf{u}_{0}\), first consolidation operation takes place, and immediately after the consolidation, at time \(t_{1}\), the utilization profile of the servers becomes \(\mathbf{u}_{1}\). After \(t\) seconds at \(t_{2}\) the utilization profile of the servers becomes \(\mathbf{u}_{2}\) and a second consolidation is carried out which is completed at \(t_{3}\) with a final utilization profile of \(\mathbf{u}_{3}\). Hence, it can be written that, \(\mathbf{u}_{1}=\Gamma\left(\mathbf{u}_{0}\right)\) and \(\mathbf{u}_{3}=\Gamma\left(\mathbf{u}_{2}\right)\), where \(\Gamma\) represents the consolidation operation. Furthermore, it holds that,  
\[\mathbf{u}_{2}=\mathbf{u}_{1}+\delta t, \tag{8}\]  
where \([\delta]_{i}\) is the task increase rate. \(A(t)\) is directly related to the amount of traffic transferred through the network for the migration. If the average size of traffic per migration is \(\
u\), then \(A(t)\) can be represented by,  
\[A(t)=\
u\Big{(}\left\|\mathbf{u}_{1}+\delta t\right\|_{0}-\left\|\Gamma(\mathbf{u}_{1 }+\delta t)\right\|_{0}\Big{)}, \tag{9}\]  
where \(\left\|\cdot\right\|_{0}\) represents the \(\ell_{0}\)-norm and returns the number of non-zero entries of its vector argument that is the total number of active servers. Therefore the difference between the \(\ell_{0}\)-norms capture the total number of VMs migrating as a result of the consolidation. Let, \(\eta_{0}\), \(\eta_{1}\), \(\eta_{2}(t)\), and \(\eta_{3}(t)\) represent the number of idle servers at time \(t_{0}\), \(t_{1}\), \(t_{2}\), and \(t_{3}\), respectively, where  
\[\eta_{0} =N-\left\|\mathbf{u}_{0}\right\|_{0}, \tag{10}\] \[\eta_{1} =N-\left\|\Gamma(\mathbf{u}_{0})\right\|_{0},\] \[\eta_{2}(t) =N-\left\|\mathbf{u}_{1}+\delta t\right\|_{0},\] \[\eta_{3}(t) =N-\left\|\Gamma(\mathbf{u}_{1}+\delta t)\right\|_{0},\]  
and \(N\) is the total number of servers in the data center.