# Online Learning Algorithms for the Real-Time Set-Point Tracking Problem
## 2 Mathematical Framework of Online Decision Making
Figure 3: Comparison of the loss values for each method. ### Case II

The only thing that differs in this case is the set-point signal fluctuation size, which is assumed to be defined as \(s_{t}=\frac{155N}{100}+15 sin(0.1t)\). In this case, similar results and sub-linear regret are achieved for both algorithms as shown in Figures 6-8. Figure 5: Regret analysis of both approaches in case I.

Figure 6: Comparison of the loss for both approaches in case II. Figure 10: Comparison of the loss for both approaches in case III with a logarithmic \(y\) axis. Figure 9: Comparison of the loss for both approaches in case III. ## 6 Discussion

In the previous section, several study cases were presented, and the performance of two algorithms based on OGD and OFW approaches were investigated. It can be seen that, generally, these two algorithms performed satisfactorily in the sense that both achieved sub-linear regret in all three case studies. A more detailed analysis regarding a quantitative comparison in terms of the output of these two algorithms could be found in Table 2. Four different values are given for the three cases acquired by each algorithm. First, the final value of the loss at the end of the optimization period is demonstrated. It can be seen that, in case I, the OGD-based method achieved lower values; specifically, if we calculate the average of all three cases, the OFW loss was 2.24-times higher than for OGD. This behavior was repeated when evaluating the average loss over the period of 20 to 100 (ignoring the initial fluctuation period), meaning that, for the average loss, OGD could output lower values. However, OFW depicted much better behavior in terms of decreasing the cost in the first five time periods, thus, showing that OFW was faster to approach the targeted signal. Especially in case II, the difference was almost ten-times less for OFW. Finally, OFW reached smaller minimum losses. For instance, the minimum cost reached by OFW was much lower than the OGD counterpart. By calculating the average of the fourth row, it can be seen that OFW achieved a four times lower minimum value compared to OGD. Considering the setting of the optimization problem, the time consumption was also of importance for comparing the performances, which is addressed in the next subsection. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**OGD**} & \multicolumn{3}{c}{**OFW**} \\ \hline  & Case I & Case II & Case III & Case I & Case II & Case III \\ Final loss (T) & 0.021 & 0.268 & 0.24 & 0.29 & 0.204 & 0.77 \\ Average loss (t = [20,100]) & 0.098 & 0.082 & 0.086 & 0.271 & 0.21 & 0.147 \\ Loss (t = 5) & 989.7 & 3614.3 & 486.9 & 323.4 & 317.6 & 853.3 \\ Minimum achieved loss & 0.003 & 0.00007 & 0.0009 & 0.00007 & 0.0008 & 0.00009 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Quantitative comparison of different values of the loss achieved by the OGD and OFW algorithms. Figure 11: Regret analysis of both approaches in case III. ### Scalability Analysis

Another important comparison is represented in Figure 12 where, for four different numbers of loads, the time consumption of each approach is measured and demonstrated. Clearly, the OFW approach is the faster one. Especially in the case of \(N\) = 10,000, the difference is more noticeable. In addition, Table 3 gives the time consumption ratio of the two algorithms for better comparison. It can be seen that the OGD algorithm always consumed more time. When the number of loads increased, the difference between the two algorithms was stronger. Especially in the case of \(N\) = 10,000 OGD, the time consumption was twenty-nine percent more. This means that when we are dealing with a large number of controllable loads and we are aiming at utilizing them in a fast ancillary service scheme regarding the delays in the signals, the OFW approach becomes more favorable. ### Implementation Discussion

It is important to note that both algorithms were carried out with theoretically recommended step sizes.