# Online Learning Algorithms for the Real-Time Set-Point Tracking Problem
###### Abstract  
With the recent advent of technology within the smart grid, many conventional concepts of power systems have undergone drastic changes. Owing to technological developments, even small customers can monitor their energy consumption and schedule household applications with the utilization of smart meters and mobile devices. In this paper, we address the power set-point tracking problem for an aggregator that participates in a real-time ancillary program. Fast communication of data and control signal is possible, and the end-user side can exploit the provided signals through demand response programs benefiting both customers and the power grid. However, the existing optimization approaches rely on heavy computation and future parameter predictions, making them ineffective regarding real-time decision-making. As an alternative to the fixed control rules and offline optimization models, we propose the use of an online optimization decision-making framework for the power set-point tracking problem. For the introduced decision-making framework, two types of online algorithms are investigated with and without projections. The former is based on the standard online gradient descent (OGD) algorithm, while the latter is based on the Online Frank-Wolfe (OFW) algorithm. The results demonstrated that both algorithms could achieve sub-linear regret where the OGD approach reached approximately 2.4-times lower average losses. However, the OFW-based demand response algorithm performed up to twenty-nine percent faster when the number of loads increased for each round of optimization. set-point tracking; online optimization; uncertainty; smart grids 2021 14 July 2021 19 July 2021 1  
solution frameworks, the issue of computational time was not properly considered as, normally, decisions are made some time prior to the real-time implementation. However, this is not the case for DR, especially where the residential loads are engaged. The decision-making intervals depending on the type of DR program could be reduced to seconds. Clearly, utilizing the existing approaches that rely on the high-computational demand and future prediction becomes almost impossible to implement since the optimization period and also computational capability is very limited, and there are several loads where the signals need to be read in time [8; 9]. The limitation mentioned above, which comes from the requirements to have rapid yet competent decisions with limited information, necessitates a specific framework: _online decision-making_. In the context of online decision-making, the whole horizon of the decision-making is divided into small intervals for which separate data of uncertain parameters are available and for each interval value of decision variables are needed to be decided consequently [10]. Indeed, when optimization time is limited, in many practical cases, the procedure of decision-making can become complex, and, as a result, devising a complete model and employing existing classical optimization algorithms will not be possible [11; 12]. Online optimization-based methods deal with these deficiencies by implementing online algorithms that incorporate learning from experience and feedback as optimization proceeds and are already popular in control science and applications, such as communication networks and the internet of things [13; 14]. Recently, these algorithms have been investigated in power systems and DR contexts as well [15; 16; 17; 18; 19; 20]. Note that another way to deal with such problems is to utilize deterministic artificial intelligence-based approaches [21]. However, considering the setting of our problem, the focus is on the first option in this study. Optimal frequency control was addressed in [15]. An online optimization framework was defined and utilized in order to extract an online policy that can react on time when required in the frequency control optimization problem.