# Online Learning Algorithms for the Real-Time Set-Point Tracking Problem
###### Abstract
The proposed control policy has a threshold structure and achieves sub-optimal, however, efficient performance regarding a lack of data about future uncertain parameters. The authors in [16] investigated voltage regulation problems for the electric distribution systems implementations, and the utilized model incorporated online optimizations regarding the structure of the decision-making. In [17; 18], the authors studied the online decision-making framework to set pricing strategies for DR programs. In [17], online pricing was proposed for DR programs. It was assumed that a utility company was in charge of N users and communicated with them only through a price signal. The user response is known afterward, and no negotiation was considered, meaning that there was only one round of optimization at each time step. This assumption makes sense as, in real-time, considering the communication delay, the time for the required optimizations is very limited. An online algorithm was proposed in this work that utilized the concept of regret for learning and as a performance metric. Regret is a common term in online optimization and is frequently used to determine the performance of an online algorithm. The regret is greater than zero and is dependent on the algorithm performance and the process of updating the decision of the next step. A good algorithm should achieve a sub-linear regret, or in other words, when the time increases, the average regret should tend to zero. Study [18] aimed to utilize online learning while designing a load scheduling learning (LSE) algorithm for multi-residential users participating in a real-time price-based DR program. In this study, the focus was on the long-term instead of users who aimed to minimize their cost in a short-term period (i.e., a day). In doing so, the interaction among the users could change the overall consumption and, as a result, the pricing schemes made by the utility within the demand response program. The DR program was considered to be price incentive-based, where prices are generally IBR-based but can also change over time. Several assumptions were considered regarding the prices and users' behavior, mainly to shift the problem and reassert it as a Markov decision process and guarantee that the Markov perfect equilibrium exists. In addition, it was presumed that the price parameters were generated according to a hidden Markov model. With this assumption, it was demonstrated how to estimate the necessary action probabilities and profile observation. Finally, the authors proposed a model including the interactions among users and the utility leading to the decreased cost values in the long-term run, where they considered the sequential decision in the process. Studies [19; 20] have designed online algorithms to solve decision-making issues for the online optimization of an end-user customer participating in a real-time pricing DR program. In [19], an algorithm was proposed in which the decision was updated regarding the gradient of the cost function. The demand response model utilized as the base case for the study was the one presented in [22]. They demonstrated that, through a day, the proposed online algorithm could perform satisfactorily compared to the existing approaches that utilized an offline approach combined with the rolling window concept [22]. This work was extended in [20], and two online algorithms were proposed based on the receding horizon concept that incorporates a very limited window of prediction in the online algorithm. It was demonstrated that these algorithms could perform better than the rolling-window robust optimization approach, and the acquired results were achieved only in seconds where the offline method requires minutes to solve the optimization. For the residential section, due to a lack of heavy computation capability, the necessity to adapt to multiple fast signals from markets and household appliances makes online decision-making more attractive.