Water heating is a major component of domestic electrical energy usage, in some countries contributing to 25 % of the residential sector energy consumption. Demand response strategies can reduce the time-of-use costs and overall electrical energy consumption. We present a method to reduce the electrical energy usage itself. Our novel heating schedule control minimises the electric water heater's energy usage without com- promising user convenience. We achieve optimal control, while taking into account the natural temperature strati cation of the water in the tank, using the A* search algorithm. Since previous research assumes a one-node thermal model, we also assess the e ect of excluding strati cation. We match three optimal control strategies to a baseline: the standard \always on" thermostat control. The rst two strategies respectively match the temperature and the energy of the hot water supplied by the water heater. The third, a variation on the second, includes a method of preventing the growth of Legionella bacteria. We tested 77 water heaters over four weeks, a week for each season, and all three strategies saved energy. The median savings were 6:3 % for temperature-matching, 21 :9 % for energy-matching and 16 :2 % for energy-matching with Legionella prevention. Taking strati cation into account increased these savings by 1 :2 %, 5:4 % and 5:5 % respectively. Keywords: Domestic energy saving; Electric water heater; Energy usage prediction; Legionella ; Optimal control; Scheduled control; Water heater temperature control 1. Introduction Energy usage by domestic water heaters can be reduced by optimal control strategies. These take into account the pattern of actual hot water usage and the user's convenience. But the savings have only been demonstrated and quanti ed using optimisation for unstrati ed thermal models. Strati cation, i.e. layers of di erent temperatures in the tank because of the di erent densities of cold and hot water, is known to occur in water heaters. A control strategy that takes this into account could well improve the savings. Much of the household electricity demand results from water heating (Hohne et al., 2019; Amirirad et al., 2018). Water heating accounts for 18 % of the residential energy consumption in the USA and 25 % in the UK (Liu et al., 2017; Singh et al., 2010). Furthermore, the residential energy used in the USA accounts for 20 % of their greenhouse gas emissions (Goldstein et al., 2020). Water heaters supply water, and consequently energy, in a cyclical pattern. They are thus well suited for using demand response to manage peak load on the grid. Those with storage tanks are particularly suitable because they can conserve thermal energy for long times with relatively little heat loss. (Ericson, 2009). The thermal energy they retain can be stored for delayed use in schemes that perform peak-shifting power scheduling (Du and Lu, 2011; Diduch et al., 2012; Shaad et al., 2012). Such schemes must take into account the heater's thermal behaviour, and the customer's water draw patterns and comfort and convenience (Gholizadeh and Aravinthan, 2016; Roux et al., 2018). Thermal models for water heaters, and algorithms for their control, have been thoroughly covered by the literature for smart grid applications (Goh and Apt, 2004; Nehrir et al., 2007; Du and Lu, 2011; Lu et al., 2011; Diao et al., 2012; Diduch et al., 2012; Booysen et al., 2013; Boudreaux et al., 2014; Nel et al., 2016a; Kepplinger et al., 2015; Gholizadeh and Aravinthan, 2016; Zuniga et al., 2017; Ahmed et al., 2018; Hohne et al., 2018; Jack et al., 2018; Kapsalis et al., 2018; Lunacek et al., 2018; Kepplinger et al., 2019a; Gerber et al., 2019). However, very few studies have proposed Corresponding author Email address: mjbooysen@sun.ac.za (M.J. Booysen ) Preprint submitted to Energy for Sustainable Development February 3, 2021 models explicitly designed to reduce the overall energy used for water heating. Most have proposed models designed to manage peak load through time-of-use cost optimisation for the bene t of the generator of the customer. In a recent study, Braas et al. (2020) developed a method for generating heat pro les for domestic water heaters to nd the most cost-ecient heating solution. They note the importance of draw-o pro les and appropriate time intervals. Pomianowski et al. (2020), reviewing the current state of work on improving the energy performance of domestic water heaters, highlight the importance of measured energy data and spatial distribution of water usage for optimisation, and they note that advanced control strategies can cleverly adjust heating systems to decrease energy loss, increase user comfort and minimise Legionella risks. Legionella bacteria can ourish in water heaters at lower water temperatures. They pose a health risk to humans, causing diseases collectively referred to as Legionellosis (Stone et al., 2019). Users in some countries, typically paying a time-dependent at fee per kWh rather than a tari based on time-of-use or congestion, resort to schedule control to reduce their monthly costs (Nel et al., 2016b; Hohne et al., 2019). These users thus bear the burden of any increased energy usage resulting from demand management schemes (Roux et al., 2018). Demand response strategies that focus on reducing the total energy consumption of a household can therefore reduce costs and minimise greenhouse gas emissions. Simulating a variety of hot water usage pro les, such as one shower per day or two baths per day, using a one-node (lumped-mass) model for the water heater, Roux et al. (2018) found that scheduled control had the largest e ect, achieving energy savings of 9 to 18%. 1.1. Research gaps Table 1 lists the relevant literature, illustrating what still needs to be achieved in saving energy through scheduled control of storage-based water heaters. This section summarises the challenges remaining (and implies the strategies to be explored in the paper). We review the studies listed in Table 1, describing their data, methods and results (energy savings), and any limitations or shortcomings. Fanney and Dougherty (1996) evaluate electrical water heater thermal eciency. They simulate six combinations of usage patterns and heating schedules. Their metric, thermal eciency, is not ideal as a standalone metric to assess savings: a water heater's eciency varies for high volume and low volume use, and if it is switched o it has a thermal eciency of 100 %. They predict savings of 4 % and 6 % but do not state the real energy savings explicitly. Goh and Apt (2004), Gholizadeh and Aravinthan (2016), Booysen et al. (2013), Nel et al. (2016a), Booysen and Cloete (2016) and Cloete (2016) use schedule control. Goh and Apt (2004) nd savings of 5 to 8%. Gholizadeh and Aravinthan (2016) nd savings of 5 :9 to 6:4% with the addition of temperature control. Both studies use simulations that do not take into account the outlet temperature and energy used. They use only predicted, not actual, consumption patterns. Booysen et al. (2013) nd an energy saving of 14 % to 17 %. They do a small experiment to con rm the results. They do not take into account the e ects of the possibly reduced temperature (i.e. reduced energy) of the water drawn from the tank. Their estimates are based on a simple lumped-mass analytical physics model (i.e. not step-wise simulated) with an assumed usage pattern for only one water heater. Nel et al. (2016a) present a more accurate model. Their model is based on horizontally mounted water heaters with thermostat-based and schedule-based heating control. Booysen and Cloete (2016) use the same model and conduct a controlled eld trial of four water heaters and a controlled laboratory experiment with one heater. They nd savings of 29 %. They do not determine the savings in the scenario where the output energy (and temperature) are matched to the baseline condition. This probably invalidates the conclusion, as the utility extracted from the water heater is not comparable before and after the intervention that resulted in the savings. The metric used to measure savings, the change in energy per litre, does not represent the savings fairly, since it compares the electrical energy (kWh) used per litre of hot water delivered under the two conditions but does not take into account the temperature, and therefore utility, of that volume of hot water. Cloete (2016) conducts a follow-up controlled study, repeating the experiment in a laboratory with longer xed heating times. This study nds 16 % savings. Cloete then uses an iterative adjustment of the set-point to match the output temperature retrospectively for a fairer comparison. This reduces the savings to only 6 %. Kepplinger et al. (2014) and Booysen et al. (2019) use dynamic programming for optimal schedule control. 2 Table 1: Remaining challenges in the literature on energy saving through scheduled water heating. Field- measured hot water1Optimal controlTemperature matched output2Energy matched output3Legionella - constrained controlSavings classi ed4Reported energy savings (%) Fanney & Dougherty (1996) 7 7 7 3 7 7 4{6 Goh & Apt (2004) 7 7 7 7 7 7 5{6 Booysen et al. (2013) 7 7 7 7 7 7 14{17 Kepplinger et al. (2014) 7 3 7 3 7 7 5{13 Kepplinger et al. (2015) 7 3 7 3 7 7 10{12 Kepplinger et al. (2016) 7 3 7 3 3 7 12 Booysen & Cloete (2016) 3 7 7 7 7 7 29{34 Gholizadeh & Aravinthan (2016) 7 7 7 7 3 7 6 Cloete (2016) 3 7 3 7 7 7 6 Nel et al. (2018) 7 7 7 7 7 7 9{18 Booysen et al. (2019) 3 3 3 3 3 7 8-18 Xiang et al. (2019) 7 7 7 7 7 7 4 1Field-measured hot water usage used in simulation to determine savings. 2Temperature-matched hot water used in simulation with purported savings. 3Energy-matched hot water used in simulation with purported savings 4Electrical energy savings split into reduced thermal losses and reduced alternative losses. Kepplinger et al. (2014) uses hourly control to minimise cost and energy usage. They use synthesised usage patterns from Jordan et al. (2001). They nd energy savings of 4 :5 to 13:3%. Kepplinger et al. (2015), their subsequent study, uses an auto-scheduling method. They simulate a strati ed thermal model, similar to the model presented here in our paper. They nd energy savings of 10 :5 to 12:4% In the case of cold events the state constraint approach is not satis ed. The model ensures equal delivery of matched energy, but does not consider the need to match temperature so that the required temperature at the start of each usage event will match the temperature achieved by thermostat control at the start of the same event. Kepplinger et al. (2016) extends the 2015 study and also runs a eld trial. They nd 12.3% energy savings. Booysen et al. (2019) uses minutely control to minimise energy usage. They use real hot-water usage patterns. They include strategies that achieve target delivery temperatures and energy usages. They also include Legionella sterilisation. As the study is based on a one-node electric water heater model, water strati cation is ignored and not re ected in the results. Xiang et al. (2019) use a novel direct load control method for water heaters without the need for tem- perature information. They do this by using a time-varied weight matrix, generated from hot water usage patterns. The matrix produces a user comfort index which determines how the water heater can be controlled to shift peak loads. 1.2. Novel contributions It is clear from the preceding analysis, which is summarised in Table 1, that many challenges remain. None of the existing literature did all of the following: implement weekly temperature control standards to limit Legionella growth, evaluate losses other than thermal losses and used water usage data sampled at high frequency, i.e. using minutes rather than hours. More importantly, none of the existing work in literature accounted for strati cation, in the heater, which is expected to have a substantial impact. This paper establishes the extent to which electrical energy used for water heating can be reduced for the case of perfectly predicted water draw patterns. We propose a novel optimal water heater control strategy that minimises the electrical energy used while conforming to user's hot water demand pro le and limiting the growth of Legionella pneumophila . Our strategy accommodates external disturbances that vary over time, such as cold water inlet temperature and ambient temperature, and input constraints, such as electricity supply interruptions. In this paper, we formulate the water heater control problem as an optimal control problem. We use the A* algorithm to identify the optimal switching schedule for the heating element to solve the formulated problem. The A* algorithm is a well-established and widely-used search algorithm that we apply to the optimal control problem. The algorithm is typically used for path planning, but it is applied to the optimal energy control for an EWH in this paper. To make the optimal control problem solvable with A*, 3 the electric water heater thermal dynamics is presented as a two-node lumped-mass model. The water in the tank is split into two distinct volumes or layers, termed \nodes" (a top node that is hot, and a bottom node that is cooler). The optimal schedule and temperature control performance is evaluated against the case in which control is unscheduled and performed by the thermostat. To do this, we use one-minute simulation time steps for 77 water heaters over four weeks (a week for each season). To account for the strati cation in the water heater, the simulations also use the two-node lumped-mass model to model the thermal dynamics. The contributions of this paper are as follows: 1. The residential electric water heater control problem is mathematically formulated in a novel way as an optimal control problem with the objective of nding the optimal heating element switching signal and temperature state trajectory to minimise the energy used while satisfying a perfectly predicted hot water usage pro le. 2. A novel optimal schedule and temperature planning technique is developed that applies the A* algorithm to solve the optimal control problem using a two-node lumped-mass model of the electric water heater that takes strati cation into account. 3. A feedback control technique is developed to control the temperature inside the electric water heater to follow the planned optimal temperature trajectory, rejecting disturbances such as unanticipated hot water usage and providing robustness to model uncertainty. 4. A reactive hot water usage simulation model is developed that simulates the user's adjusting the ratio of cold to hot water to obtain the desired temperature. 5. The study determines how much energy can theoretically be saved by using optimal control for electric water heaters instead of the traditional thermostat control. 6. The study determines how much energy can theoretically be saved when taking strati cation into account when performing usage-based optimal energy control for electric water heaters, compared to when not taking strati cation into account. 2. System 2.1. System overview The goal of the system described in this paper is to minimise the electrical energy used by a storage-based electric water heater (EWH) while preventing the user from experiencing cold water temperatures, for a given predicted hot water usage pro le. An overview of the system is shown in Figure 1. The system consists of an optimal temperature schedule planner, a temperature feedback controller, a probabilistic hot water usage model, and a hot water demand predictor. The EWH is modelled using a two-node thermodynamic model that accounts for strati cation. The EWH is controlled by a heating element that can be switched either on or o . The user is modelled using a reactive hot water usage model that simulates the user experiencing the hot water outlet temperature and adjusting the mixing ratio of hot and cold water to obtain the desired temperature. The optimal control sequence for the heating element and the corresponding optimal EWH temperature trajectory are determined by an optimal temperature schedule planner. The optimal temperature planner could be implemented using either a one-node EWH model that does not account for water strati cation, or using a two-node EWH model that does account for water strati cation. If the one-node EWH model that does not account for strati cation is used, then a Dynamic Programming (DP) algorithm is used to solve the optimal control problem (Booysen et al., 2019). If the two-node EWH model that accounts for water strati cation is used, then the optimal control problem is solved using an A* search algorithm, as described in this paper. The temperature feedback controller is used to compensate for deviations between the planned optimal temperature trajectory and the actual temperature trajectory, rejecting disturbances such as unanticipated hot water usage and providing robustness to model uncertainty. The temperature feedback controller controls the water temperature inside the EWH to follow the temperature set point provided by the optimal tem- perature planner by switching the heating element based on feedback from the EWH internal temperature sensor. 4 Figure 1: Flow diagram of optimal electric water heater (EWH) control determined by an optimal temperature planner. Components are indicated as physical or software. Components shown in grey are software components that will be fully explored in future work. The optimal schedule planner uses a predicted hot water demand pro le to plan the optimal EWH temperature trajectory. The predicted hot water demand pro le is provided by the hot water demand predictor, which in turn uses a probabilistic hot water usage model. The probabilistic hot water usage model is obtained by tting a probabilistic model to historical measured hot water usage data obtained from the EWH temperature and ow rate sensors (Ritchie et al., 2020). This paper investigates how much energy can theoretically be saved with optimal control for electric water heaters compared to traditional thermostat control, when perfect foreknowledge of the hot water usage pro le is available. The probabilistic hot water usage model and the hot water demand predictor will therefore not be used in this paper, and will be replaced with perfect foreknowledge of the hot water usage pro le. The practical energy savings that can be achieved with usage-based optimal energy control, when a predicted hot water usage pro le based on historically measured data is used, will be investigated in our next paper. 2.2. Heating control strategies Thermostat control is our baseline heater control strategy. We evaluated three alternative strategies, the third being a variant of the second, to discover which saves the most energy. 0. (Baseline) Thermostat control (TC): Water heaters are designed to be used like this, and most people use them this way. The thermostat maintains the water at a target temperature, usually between 65C and 75C, with a small hysteresis band around the set temperature. This is a wasteful strategy because it keeps the water hot for long periods when hot water is not required. It loses more energy to the environment than the other three strategies. It also means that water is drawn from the tank at a higher temperature than normally required. To correct this, usually to about 40C, the user must adjust the temperature by mixing cold water with the hot water (Armstrong et al., 2014; Jacobs et al., 2018; Kepplinger et al., 2019b). 1. Scheduled control with temperature matching (TM): To save costs and reduce loss of energy, some users turn o heaters for long periods when they do not need hot water (Nel et al., 2016b). Users' methods vary, but ideally they switch the heater on shortly before they need hot water (Booysen et al., 2013; Nel et al., 2016b). Our optimisation constraints can be set so that the same volume of water is drawn at the same temperatures as under thermostat control, but minimising the thermal loss to the environment. This strategy assumes that the user wants water at a temperature as high as 60C to 70C and will add cold water to achieve the desired temperature. In this strategy, the heater delivers the same amount of useful output energy as in the baseline strategy (TC). 2. Scheduled control with energy matching (EM): This strategy assumes that the user does not require the water at such high temperatures but at a lower, more directly usable temperature, so it may not be necessary to add cold water. A lower target temperature during water draw-o s, of say 38C, could be used. We increased the volume drawn from the tank so that the same amount of energy was delivered in the water drawn as the baseline strategy (TC) (Armstrong et al., 2014; Jacobs et al., 2018; Roux et al., 2018). 5 3. Scheduled control with energy matching plus Legionella sterilisation (EML): Strategy EM saves energy, but maintaining and delivering water at low temperatures poses health risks. Legionella pneumophila thrives at temperatures between 32C and 42C and has been found in water heaters (Armstrong et al., 2014; Stone et al., 2019). To sterilise the bacteria, the heater must remain at 60C for 11 min or 70C for 3 min, at least once a day (Stout et al., 1986). Strategy EML ensures that the heater temperature reaches 60C for at least 11 min at least once before the largest water usage event of the day. 2.3. Electric water heater thermodynamics The EWH thermodynamics can be modelled using a one-node or two-node model. The latter models strati cation. In this paper we consider only a vertically oriented tank. We do not consider a horizontally oriented tank because it requires substantially more computational power to model and makes optimisation increasingly dicult Nel (2015). The EWH is modelled according to an energy balance equation to track the energy ow in the tank. In the one-node model, the body of water inside the tank is assumed to be at a uniform temperature, as shown in Figure 2(a). Energy ows into the tank from an electrical heating element situated near the base of the tank. When water is drawn from the tank at a temperature higher than that of the water in the inlet pipe, there is a reduction in thermal energy due to a volumetric ow rate. When there is a temperature di erence between the water inside the tank and the ambient temperature, thermal energy is lost from the tank at a rate determined by the thermal resistance of the tank. The two-node EWH in Figure 2(b) models strati cation by introducing a thermocline that divides the tank into an upper and a lower node which represent the hot and cold water respectively. Water leaving the outlet pipe is at the temperature of the upper node and water entering the inlet pipe is at the ambient temperature. Inter-node energy transfer occurs due to the temperature di erence at the thermocline between the two bodies of water at a rate determined by the thermal resistance of the thermocline. (a) One-node EWH. (b) Two-node EWH with strati cation. Figure 2: Energy ow, thermal resistance, ow rate, temperature and volume in a) one-node and b) two-node EWH. One-node EWH dynamics The thermal dynamics of the one-node lumped-mass model are expressed as follows: _Etank(t) =Pelec(t)Pelec(t)Ploss(t) (1) whereEtankis the thermal energy in the tank, Pelecis the power delivered by the heating element, Pelecis the power leaving the tank when hot water is drawn, and Plossis the power leaving the tank due to losses to the environment. The equation shows that the rate of change of thermal energy is directly in uenced by Pelec,PdrawandPloss. The heating element supplies electrical power at Pratedwhen the switch is on and zero when o and is shown as follows: Pelec(t)2f0;Pratedg (2) 6 The energy supplied by the element is given as follows: Eelec=Ztf t0Pelec(t)dt (3) The power leaving the tank during hot water usage is given as follows: Pdraw(t) =Qdraw(t)[^houtlet (t)^hinlet(t))] (4) whereis the density of the water, Qdraw(t) is the hot water outlet volumetric ow rate, and ^hinletand^houtlet are the speci c enthalpy entering and leaving the water heater, respectively. Under conditions of constant pressure and constant speci c heat capacity, this can be approximated by Pdraw(t)cPQdraw(t)[Toutlet (t)Tinlet(t)] (5) wherecPis the constant pressure-speci c heat capacity of the water, Toutlet is the hot water outlet temper- ature, andTinletis the cold water inlet temperature. The power heat loss to the environment due to the temperature di erence between the water inside the tank and surrounding ambient temperature is calculated as follows: Ploss(t) =1 Rtank[Ttank(t)Tamb(t)] (6) whereRtank is the thermal resistance of the wall of the tank, Ttankis the temperature of the water inside the tank and Tambis the ambient temperature. The temperature of the water inside the tank is assumed to be equal to the outlet temperature Toutlet . The relationship between the EWH energy Etankand the EWH water temperature Ttankrelative to a reference temperature where we de ne energy to be zero, is given as follows: Ttank(t) =Etank(t) cVVtankEtank(t) cPVtank(7) whereVtank is the volume of the EWH, and cVis the constant volume speci c heat capacity, which is approximately equal to cPfor water and henceforth denoted as c. Two-node EWH dynamics When the tank is in a one-node state and water is drawn from the tank at a higher temperature than the inlet water temperature, the tank transitions to a two-node state. When all the hot water is drawn from the tank, the EWH reverts to a one-node state and the temperature of the whole tank is that of the lower node. The EWH also transitions to a one-node state when the lower node temperature reaches the temperature of the upper node. The nodes are referred to as the upper and the lower node and are designated by subscripts U and L respectively. The energy and volumes of the upper and lower nodes are related to the total energy and total volume of the tank by the following equations: Etank(t) =Etank,U (t) +Etank,L (t) (8) Vtank,U (t) +Vtank,L (t) =Vtank (9) The total energy Etank in the tank is the sum of the energy Etank,U in the upper node and the energy Etank,L in the lower node. The sum of the upper node volume Vtank,U and the lower node volume Vtank,L are constrained to equal the total volume of the tank Vtank, which remains constant. The thermal dynamics of the two-node EWH model is described by a set of four di erential equations in terms of the upper node energy, the lower node energy, the upper node volume, and the lower node volume respectively. The rst di erential equation describes the dynamics of the upper node's thermal energy, as follows _Etank,U (t) =Pdraw,U (t)Ploss,U (t)Ptrans,U (t) (10) 7 The rate of change of the upper node's thermal energy _Etank,U is the sum of the power Pdraw,Uleaving the upper node when hot water is drawn, the power Ploss,Uleaving the upper node due to losses to the environment, and the power Ptrans,Uleaving the upper node due to power transfer to the lower node across the thermocline. The powerPdraw,U is determined by the volumetric ow rate and the temperature of the hot water leaving the tank through the outlet: Pdraw,U (t)cPQdraw(t)Ttank,U (t) (11) whereTtank,U is the temperature of the upper node. The power loss Ploss,U of the upper node to the environment is determined by the temperature di erence between the upper node and the environment, and is calculated as follows: Ploss,U (t) =1 Rtank,U[Ttank,U (t)Tamb(t)] (12) whereRtank,U is the thermal resistance of the tank at the upper node. The power transfer Ptrans,U from the upper node to the lower node across the thermocline is determined by the temperature di erence between the upper node and the lower node, and is calculated as follows: Ptrans,U (t) =1 RTH[Ttank,U (t)Ttank,L (t)] (13) whereRTHis the thermal resistance of the thermocline and Ttank,L is the temperature of the lower node. The second di erential equation describes the dynamics of the lower node's thermal energy: _Etank,L (t) =Pelec(t) +Pinlet,L (t)Ploss,L(t)Ptrans,L (t) (14) The rate of change of the lower node's thermal energy _Etank,L is the sum of the electrical power Pelec delivered to the lower node by the heating element, the power Pinlet,L entering the lower node due to the thermal energy in the cold water owing into the inlet, the power Ploss,Lleaving the lower node due to losses to the environment, and the power Ptrans,Lleaving the lower node due to power transfer to the upper node across the thermocline. (If the cold water owing into the inlet is warmer than the ambient temperature, then the inlet power Pinlet,L is positive; if the cold water owing into the inlet is colder than the ambient temperature, then the inlet power Pinlet,L is negative.) The power loss Ploss,L of the lower node to the environment is determined by the temperature di erence between the lower node and the environment, and is calculated as follows: Ploss,L(t) =1 Rtank,L[Ttank,L (t)Tamb(t)] (15) whereRtank,L is the thermal resistance of the tank at the lower node. The power transfer Ptrans,L from the lower node to the upper node across the thermocline is determined by the temperature di erence between the lower node and the upper node, and is calculated as follows: Ptrans,L (t) =1 RTH[Ttank,L (t)Ttank,U (t)] (16) The thermal resistance Rtank,U andRtank,L are calculated as follows: Rtank,U =RAtank Atank,U(17) Rtank,L =RAtank Atank,L(18) whereAtank,Atank,U andAtank,L are the areas of the tank, upper node and lower node respectively. The third and fourth di erential equations describe the dynamics of the upper node volume and the lower 8 node volume, as follows _Vtank,U (t) =Qdraw(t) (19) _Vtank,L (t) =Qdraw(t) (20) rate at which the upper node volume Vtank,U decreases and the rate at which the lower node volume Vtank,L increases both equal the ow rate Qdrawof the hot water leaving the tank, which also equals the ow rate of the cold water into the tank. the lower node volume can be calculated by simply subtracting the upper node volume from the total tank volume, the fourth di erential equation (21) for the lower node volume is not required, and can be replaced by the following constraint equation: Vtank,L (t) =VtankVtank,U (t) (21) 2.4. Temperature feedback control The optimal temperature plan is passed to the temperature feedback controller before determining the input of the EWH at any given time. The controller compares the measured temperature of the EWH with the desired time-varying temperature set-point of the optimal plan. The controller will override the optimal input for the EWH so that the temperature of the EWH follows the optimal temperature (with hysteresis). The temperature feedback control corrects the EWH temperature when it deviates from the optimal plan. Temperature deviations are caused by unexpected water usages (not included here because we assume perfect predictions) and model inaccuracies. 2.5. User and water mixer The user experiences the hot water temperature of the EWH when a usage event is intended. If the initial temperature experienced by user is not the desired temperature, the user will adjust the ratio of hot and cold water using a water mixer to reach such a desired temperature. The water mixer is a model of reality and is used to perform and evaluate simulation tests. This mechanism will also contribute to the energy matching heating control strategies where, if the optimal temperature is di erent to the thermostat set-point temperature for a usage event, the adjustment of the hot water ow rate will provide a matched energy usage from the EWH. 3. Optimal Temperature Planning for an EWH with Strati cation In this section, the algorithm that performs the optimal temperature planning for an EWH with strati - cation is presented. In our previous paper, Booysen et al. (2019), the optimal control problem was formulated for the one-node EWH model without strati cation and was solved using a dynamic programming algorithm. In this paper, the optimal control problem is formulated for the two-node EWH model with strati cation and is solved using an A* search algorithm. 3.1. The optimal control problem formulation Optimal control problem: Given a hot water usage pro le in terms of ow rate Qusage(t) and a desired minimum hot water temperature Tusage(t) as a function of time t, the cold water inlet temperature Tin(t) and the ambient temperature Tamb(t) as time-varying disturbance signals, and scheduled supply-side interruption of the electricity supply to the EWH Pmax(t), we wish to determine the optimal EWH control signal P elec(t) that will satisfy the hot water usage pro le, while minimising the total energy used. System dynamics: We de ne the system dynamics using the nonlinear di erential equations describing the EWH thermal dynamics for both a one-node and a two-node EWH, as described above, and speci cally by equations (1), (2), (5) and (6) for the one-node model and equations (8), (9), (10) (11), (12), (13), (14), (15), (16), (19), (20), and (21) for the dynamics of the two-node model. 9 System state: The system state for the two-node EWH system is represented by three state variables: the thermal energy Etank,U in the upper node, the thermal energy Etank,L in the lower node, and the upper node volume Vtank,L . The system state is therefore represented by the following state vector: x(t) =Etank,U (t)Etank,L (t)Vtank,U (t)T(22) Control input: The control input for the system is the power input Pelecdelivered by the heating element: u(t) =Pelec(t) (23) State constraints: We specify the physical limitations on the thermal energy and volumes of the upper and lower nodes in the EWH by de ning the following set of admissible states: Etank,U (t)2[Emin;Emax] (24) Etank,L (t)2[Emin;Emax] (25) Vtank,U (t)2[0;Vtank] (26) where the minimum energy Eminand the maximum energy Emaxcorrespond to the minimum temperature Tminand the maximum temperature Tmaxspeci ed for the EWH: Emin =cVtankTmin (27) Emax =cVtankTmax (28) Input constraints: We specify the control input constraints by de ning the following admissible inputs: Pelec(t)2f0;Pratedg (29) As the heating element is either o or on, the power input Pelecdelivered by the element is either zero or its power rating Prated. Boundary conditions: We specify the initial and terminal boundary conditions by de ning the follow- ing temperatures: T(ti) =Tstart (30) T(tf) =Tend (31) wheretiis the initial time and tfis the nal time. The temperatures TstartandTendare the desired initial and nal temperatures, respectively. The boundary conditions are selected such that the optimal path be- gins and ends at the corresponding desired temperatures. Both of the boundary conditions are chosen to be equivalent to the set-point temperature of 68 :5C which we use for thermostat control in this paper. The EWH is assumed to be in the one-node state at both the initial time tiand the nal time tf. Cost function: We use the following cost function to represent the objective of minimising the total energy usage: J=Ztf tiPelec(t)dt (32) Temperature pro le constraints (usage and Legionella sterilisation): To represent the objective of satisfying the hot water usage pro le, we use the following time-varying state inequality constraints on the temperature of the water for both nodes inside the EWH: Ttank,U (t)Tpro le,U (t) (33) Ttank,L (t)Tpro le,L (t) (34) 10 To prevent the growth of Legionella bacteria, we need to increase the temperature once a day. We can include this in the time-varying state inequality constraints. We set the upper node pro le temperature Tpro le,U (t) to the desired usage temperature Tusage for times that correspond to a draw, the lower node pro le temperature Tpro le,L (t) to the minimum Legionella sterilisation temperature TLegionella to ensure that the entire tank prevents bacterial growth, and both pro les to the minimum EWH temperature Tminfor all other times: Tpro le,U (t) =Tusage when intentional draw causes Qdraw(t)>0 Tmin otherwise (35) Tpro le,L (t) =TLegionella once per day to prevent Legionella growth Tmin otherwise (36) Electricity supply constraint: We represent the constraint on the EWH, due to the scheduled supply-side interruption of the electricity supply, by the following time-varying input constraint: Pelec(t)Pmax(t) (37) We set the maximum power input to Prated when the supply-side electricity is available, and to zero when it is interrupted, with Pmax(t) =0 when the electricity is interrupted Prated when the electricity is available (38) Temperature pro le constraint: We construct the temperature pro le constraint Tpro le (t) di erently for temperature matching, energy matching and energy matching with Legionella sterilisation. We take into account \unreasonable" hot water usage pro les where hot water cannot be delivered at the minimum desired temperature, even with the heating element permanently switched on. An example of unreasonable usage would be drawing all the hot water from the tank and then expecting more hot water to be available immediately. Optimal temperature matching (TM): We construct the temperature pro le constraint for temperature matching so that the required EWH temperature at the start of each usage event matches the EWH tem- perature achieved by thermostat control at the start of the same event. We construct the constraint on the lower node temperature pro le so that the temperatures will still match if the EWH transitions to a one-node state. Optimal energy matching (EM): We construct the temperature pro le constraint for energy matching so that the EWH upper node temperature remains above 40C for the duration of each usage event. However, we increase the outlet ow rate so that the energy in the volume of hot water delivered at 40C matches the energy in the volume of hot water delivered at the temperature used in thermostat control. Optimal energy matching with Legionella sterilisation (EML): We construct the temperature pro le con- straint for energy matching with Legionella sterilisation just as we did for energy matching (EM), except that once a day, just before the largest usage event of the day, we increase the EWH lower node temperature to 60C for 11 min. \Unreasonable" hot water usage pro les: To accommodate these pro les, we run a forward simulation for the entire usage pattern, assuming that the heating element is always switched on, to determine the optimal temperatures the EWH can deliver for a given usage pro le. We then modify the temperature pro le constraintTpro le,U (t) to be the minimum of the temperature pro le constructed to satisfy the user's demand for hot water and the Legionella sterilisation and the achievable temperatures that were determined from the forward simulation. 3.2. The A* solution The A* algorithm is a well-known and widely-used shortest path search algorithm that can be used to model an optimal control problem as a node-based data structure navigation process to nd the optimal state trajectory and control inputs to minimise the cost function from an initial state to a destination. The 11 algorithm optimises its search time by introducing a heuristic function that estimates the path to a terminal state. However, the eciency of the algorithm depends on the quality of the chosen heuristic function. In the study by Booysen et al. (2019), the optimal temperature plan was designed for a one-node EWH model only. Because the one-node model is a rst order system, dynamic programming was a suitable approach for optimisation. However, the present paper explores optimisation of the two-node model, which is not a rst-order system and makes A* more applicable. The A* algorithm builds a binary search tree with a size determined by the number of possible actions at each node of the tree. For both the one-node and two-node models there are only two possible actions: element on and element o . Discretisation To apply A* to an optimal control problem, we have to break the problem into discrete time instants to represent the decision stages, and into discrete states to represent the decisions to be made at each decision stage. The A* algorithm nds the optimal path by starting at the initial stage and working through inter- mediate stages until it nds the rst admissible path from the initial state to a terminal state. The rst admissible path is also the optimal path due to the way that the paths are sorted in a priority queue. Discrete-time dynamic model: The continuous-time di erential equations describing the system dy- namics are discretised to produce discrete-time di erence equations that describe the state transition from one discrete time instant to the next. For the one-node case, the state transition is described as E(k+ 1) =E(k) + [Pelec(k)Pusage(k)Plosses(k)]t (39) where tis the sampling period of the discrete time instant. For the two-node case, the state transition is described by the following set of di erence equations: EU(k+ 1) =EU(k) +_EU(k)t (40) EL(k+ 1) =EL(k) +_EL(k)t (41) Vtank,U (k+ 1) =Vtank,U (k)Qdraw(k)t (42) where _EU(k) and _EL(k) are respectively given as _EU(k) =Pdraw,U (k)Ploss,U (k)Ptrans,U (k) (43) _EL(k) =Pelec(k) +Pinlet,L (k)Ploss,L(k)Ptrans,L (k) (44) To reduce the number of variables that will be tracked by the A* algorithm, the lower node volume is calculated by subtracting the upper node volume from the constant total volume of the tank, as follows: Vtank,L (k+ 1) =VtankVtank,U (k+ 1) (45) Initial state: The initial state is speci ed at the rst time instant t= 0 and is expressed as follows: xs=Estart (46) wheresindicates the starting state of the algorithm and Estartis equivalent to the energy of the tank at the de- sired starting temperature Tstart. The EWH is also assumed to be in a one-node state at the rst time instant. Goal state: The goal of the algorithm is to nd a path from the initial state xsto a goal state. The goal state is speci ed at the last time instant t=Nand is expressed as follows: xf=Eend (47) 12 where xfindicates the goal state of the algorithm and Eendis equivalent to the energy of the tank at the desired nal temperature Tend. The EWH is also assumed to be in a one-node state at the last time instant. EWH model transition: The EWH's current model is represented by ns=1 if one-node EWH model 2 if two-node EWH model (48) and transitions between one node and two nodes when certain conditions are met. The EWH model transitions can be represented as a state machine that begins as a one-node model. The one-node model will transition to the two-node model when the outlet ow rate is greater than zero: Qusage>0 (49) When the single node splits into two nodes, the lower node is assigned the same temperature as the inlet temperature, the volume of the upper node is reduced by the usage volume, and the volume of the lower node is increased by the usage volume: TL=Tin (50) VU=VtankVusage (51) VL=Vusage (52) The two-node model will transition back to the one-node model if one of two conditions are met: 1. The temperature of the lower node equals or would exceed the temperature of the lower node. 2. The volume of the upper node is reduced to zero while the usage ow rate is greater than zero. If the temperature of the lower node reaches the temperature of the upper node: TLTU (53) then the upper and lower nodes are merged into a single node represented by the upper node. The temperature of the lower node is assigned the temperature of the upper node, the volume of the lower node is set to zero, and the volume of the upper node is set to the volume of the tank. TL=TU (54) VU=Vtank (55) VL= 0 (56) If the volume of the upper node is reduced to zero: VU0whenQusage>0 (57) then the upper and lower nodes are merged into a single node represented by the lower node. The temperature of the upper node is assigned the temperature of the lower node, the volume of the upper node is set to zero, and the volume of the lower node is set to the volume of the tank. TU=TL (58) VU= 0 (59) VL=Vtank (60) Action space: The following binary action space is used to generate the child nodes from a given parent node: Uq=f0;Pratedg (61) 13 Cost-to-come: The cost-to-come is calculated incrementally as nodes are created and added to the search tree. The cost to come is the total energy use so far and is calculated with g(k+ 1) =g(k) +Pelec(k)t (62) whereg(k+ 1) is the total cost-to-come of the child node, g(k) is the total cost to come of the parent node, andPelec(k)tis the incremental energy used. Heuristic search: A heuristic cost function is introduced to accelerate the search algorithm by priori- tising the optimal path as the next iteration of the algorithm execution. This is accomplished by heuristics: estimating the path cost from the next state xj(k+ 1) to the terminal state. Cost-to-go: At any time instant, the EWH must reach the terminal state after the result of thermal energy that is anticipated to still leave the tank. The cost-to-go estimates both the minimum amount of energy that must still be supplied to the tank to reach the terminal energy state as well as how much thermal energy will leave the tank during the remaining water usages from the considered time instant. The cost-to-go is calculated with h(k+ 1) =E(k+ 1) +E(N) +NX n=k+1Pusage(n)t (63) whereE(k+ 1) is the energy at the child node, E(N) is the energy at the nal node, and Pusage(n)tis the predicted thermal energy that will leave the tank due to hot water usage. Because standing losses contribute a relatively small portion of the thermal energy that leaves the tank, the thermal energy loss to the environ- ment is not included in the cost-to-go . The heuristic is still valid, however, since by ignoring the standing losses the heuristic underestimates the actual cost-to-go . Pusage(n)tis the estimated thermal energy that is drawn from the tank at a speci c time instant. It is pre-calculated by performing a forward simulation of the water pro le which acts as a disturbance to the EWH. The simulation is performed such that each water event ends with the outlet temperature remaining aboveTusage. Total path cost: The total cost Jis calculated by the cost-to-come and cost-to-go , and is calculated as follows: J(k+ 1) =g(k+ 1) +h(k+ 1) (64) Binary search tree (BST): This section describes how a BST data structure is used to aid the A* path searching algorithm. We model the navigation of the data structure from the traditional search process. More information about the BST data structure can be found in Optimal Binary Search Trees (Nagaraj, 1997). The A* approach creates a BST comprised of internal nodes that are connected to one another by the next statejof the proceeding node to branch into multiple search paths. The rst series of nodes that reaches the terminal state node from the initial state node is also the optimal path. A single node object is represented by symbolNand is expressed as Nik=Ni(xi[k]) (65) whereNikis a node at current state iand timek. The node represents multiple keys of information that are required to track the necessary variables of the EWH at that instant of the path. A single node tracks the state xi, the optimal total path cost J ik, the optimal next state j ikas a result of u= 0 and u=Pelec, the optimal upper node temperature T Uik, the optimal lower node temperature T Likand the optimal volume ratioV rik. Initial state node: The initial state node is the rst node initialised by the BST for the desired starting 14 statesat the rst time instant t= 0 and is de ned as follows: Ns=Ns(xs) (66) and is assigned the following optimal cost: J s= 0 (67) This node is assigned a cost of zero to indicate that it is the origin of all succeeding paths that will branch from it. Assignments are only made for the optimal next state j sfor each possible control input usif an admissible node is reached as a result of the control input during the algorithm execution. Terminal state node: A terminal state node is a unique modi cation of the prede ned node object. For the desired nal state fat the last time instant t=N, the terminal node state is de ned as follows: NfN=Nf(xf[N]) (68) This node is unique as it only needs to track the terminal state xfand the desired nal outlet temperature T UfN. Optimal path: The optimal state trajectory path is represented by xwhere subscript refers to the path that minimises the energy from the rst time instant t= 0 to the nal time instant t=N. It is obtained as follows: x=xs (69) wheresis always the starting state for the optimal path. This is the outcome of the algorithm. It can also be used to nd the corresponding optimal input sequence u. Priority queue: The order of iterations for the algorithm execution jumps to the time instant depending on which search path has a higher priority than all the other search paths. A priority queue is de ned as follows: P=fNik;:::g (70) where each entry is a node corresponding to any admissible search path ending. However, the queue organises the entries in ascending order based on each node's total path cost Jik. The queue guides the algorithm execution with the order of iterations for extending the search paths simultaneously. Each iteration begins by removing the rst, and highest priority, node from the queue and is expressed as P=Nik (71) This node with the highest priority is used to determine whether any admissible state is reachable for each possible control input, if that input is applied to the node's current state. This is the reason why the rst search path to reach the terminal state node is also the optimal path. 4. Results and discussion The EWH optimal control is determined and used for the simulation of each water heater (aided with the temperature feedback controller and water mixer). The results of including strati cation in our model are obtained by the simulator and evaluated in this section. In the rst half of this section, the proposed optimal energy EWH control techniques using a two-node EWH model that takes strati cation into account are compared to traditional thermostat control. First, 15 Table 2: Parameters used for simulations and optimisation. Symbol Description Value Unit EWH model parameters RTH Thermal resistance of EWH 0.4807Kday kWh c Speci c heat capacity of water 4184J kgK  Water density 1000kg m3 Tset Target temperature 68.5C Thyst Hysteresis (deadband) 1:5C Tamb Ambient temperature 20C Tinlet Inlet temperature of EWH 20C Vtank Tank volume of EWH 150 L Prated Power rating of element 3 kW Optimisation parameters Ttank(max) Maximum temperature of EWH 70C Ttank(min) Minimum temperature of EWH 20C Ttank(use) Minimum target usage temperature 40C Tstart Initial boundary condition of EWH 68.5C Tend Terminal boundary condition of EWH 68.5C ns Node state of EWH model 1 or 2 Water draw dataset D Duration 7 days t Sampling period 1 min Resolution 0.5 L Number of water heaters 77 Number of seasons 4 Average number of events per EWH per day 7.5 the simulation results for a single EWH are presented and discussed to illustrate the operation of the three optimal EWH control strategies (temperature matching, energy matching, and energy matching with Le- gionella prevention) compared to the operation of the traditional thermostat control. Then, the statistical performances of the three optimal EWH control strategies compared to traditional thermostat control are evaluated by analysing the distributions of some performance metrics over the simulation results for all of the EWHs . In the second half of this section, the bene t of taking strati cation into account when performing the optimal energy EWH control is evaluated by comparing the results for optimal energy EWH control using two- node planning to the results for optimal energy EWH control using one-node planning. First, the simulation results for a single EWH are presented and discussed to illustrate the operation of the optimal energy EWH control using two-node planning compared to the operation of the optimal energy EWH control using one- node planning. Finally, the statistical performance of optimal energy EWH control using two-node planning compared to one-node planning is evaluated by analysing the distributions of some performance metrics over the simulation results for all of the EWHs . 4.1. Simulation setup The hot water usage data, the software implementation in Jupyter Notebook, and the simulation output are available at https://bit.ly/StratifiedOptimisation . Table 2 lists the dataset properties, parameters and constants used in the optimisation and simulation. 16 (a) Thermostat control (b) Temperature matching (c) Energy matching (d) Energy matching with Legionella control Figure 3: Simulation results for TC, optimal TM, optimal EM, and optimal EML. Shown are EWH temperature, ow rate, and heating element state. The cold event threshold (40C) is shown with a dashed line in black. 4.2. Simulation results for a single EWH using two-node planning Figure 3 shows an example of simulation results for a single EWH. Figures 3 (a), (b), (c) and (d) compare the four control strategies, TC, TM, EM and EML. For each strategy, we plot the EWH temperatures, outlet ow rateQdraw, and heating element control signal Pelecas a function of time for a 24-hour period. The upper node thermostat control temperature TTC,U is repeated in the four gures as a reference. The results shown are for the same hot water draw pro le, with all simulations starting from the same initial EWH temperature of 68.5C for both upper and lower nodes. Because of the way the EM and EML strategies work, the draw patterns di er from those of the TC and TM strategies. Thermostat control (TC): This strategy maintains the temperature at a set point of 68.5C (allowing for 1.5C hysteresis). The temperature drops signi cantly during usage events, when the outlet ow rate is non-zero. When the rst usage occurs at time t= 3, the lower node temperature immediately drops to 20C and the upper node temperature rapidly decreases as a result of water strati cation caused by the increasing volume of cold water. The heating element switches on when the upper node temperature drops below 67C. As the element supplies power to the lower node, that node will need to reach the temperature of the upper node, returning the water in the tank to a one-node state, before the upper node temperature can increase to the set-point temperature. Optimal temperature matching (TM): This strategy matches the outlet temperature to the corresponding temperature for thermostat control, but only during usage events. Between usage events, TM allows the tem- perature to drop. Heating resumes shortly before the next usage event. Exponential responses are observed in the upper node and lower node temperatures when the heating element is o but the two nodes are still at di erent temperatures. These responses are due to the inter-node energy transfer striving to even out the 17 two temperatures. Optimal energy matching (EM): This strategy ensures that the outlet temperature remains above 40C during usage events and the outlet ow rate increases so that the thermal energy drawn is equivalent to that of TC. The rst usage of the day at t= 3 is large and the temperature of the tank is increased to 57C just be- fore the event starts to ensure that the upper node temperature will remain above 40C throughout the event. Optimal energy matching with Legionella sterilisation (EML): The results for this strategy look very similar to EM, except that the temperature of the whole tank is increased to the Legionella sterilisation threshold of 60C at timet= 3 and maintained at that level for 11 minutes. This increase is scheduled to be just before the largest usage event of the day, excluding cold events. The temperature during and between usage events is therefore almost the same for EM and EML, except for the 11-minute increase to sterilise Legionella . We therefore expect EML to use slightly more electrical energy. However, since the temperature is increased to sterilise Legionella just before the largest event, the required increase in temperature is minimal. 4.3. Metrics for evaluating the results of the EWH model with strati cation Water draw is represented as usage events . An event starts when the user opens a tap and ends when the user closes it; in other words, it starts when a zero volume sample is followed by a non-zero volume sample and it ends with a zero volume sample. This de nition is a convenient way to refer to sections of water usage patterns. It gives us a metric for counting the number of times a user experiences an undesired temperature. A cold event is de ned as one where the initial water temperature experienced by the user is lower than expected (i.e. less than 40C). Since there is a considerable length of pipe between the water heater outlet and the tap, the point of use, events with a volume of less than 2 L are unlikely to result in the hot water reaching the point of use. Taking, for example, a pipe of the normal 22 mm diameter and a pipe length of 5 m, we nd that a volume of 1 :9 L hot water is drawn into the piping before it reaches the point of use. We treat these events that draw hot water that does not reach the point of use as unintended events, for which our algorithm does not instruct the element to heat, and we exclude them from the number of cold events. Typical reasons for these events are using a mixer tap with a position between hot and cold, or mistakenly opening the hot tap when needing cold water. Table 3: Performance assessment metrics. Metric Description Unit Eelec Daily average electrical energy used for water heating. kWh Edraw Daily average thermal energy in hot water drawn from tank. In- dicates e ective energy used, but includes energy lost due to un- intentional use.kWh Eloss Daily average energy lost to environment through tank. kWh E(kwh) Reduction in electrical energy per day compared to thermostat control.kWh E(%) Reduction in electrical energy per day compared to thermostat control.% Tusage Average event temperature of the upper node (excludes uninten- tional use)C Cold events Number of events with any sample T <40C. Table 3 shows the metrics we use to see how the various strategies perform: electrical energy supplied by the element, energy in the drawn water and energy lost to the environment. To determine how much energy is saved, we use the change in electrical energy as a percentage reduction and as an absolute change. We also use the average event temperatures, including the number of cold events, to assess user comfort and convenience. Figures 4 (a) to (d) shows, for all 77 EWHs, the distributions of electrical energy usage per day, thermal energy usage per day, average outlet temperature during usage events, and thermal energy losses per day. 18 The average electrical energy an individual EWH uses per day is calculated using Pelecjh=PNh k=1Pelecjh(k)t DkWh/day (72) wherePelecjhis the average electrical energy used by heater hper day,Pelecjh(k) is the electrical power used by heaterhat time instant k, tis the sampling period, Nhis the total number of samples, and Dis the total number of days in the data set. The average thermal energy used per day Pdrawjhand the average thermal energy losses per day Plossjhfor an individual EWH are calculated using Pdrawjh=PNh k=1Pdrawjh(k)t DkWh/day (73) Plossjh=PNh k=1Plossjh(k)t DkWh/day (74) Figures 4 (e) and (f) show the distributions of the electrical energy savings per day for TM, EM and EML, expressed as a reduction in both kWh per day and percentage. To calculate the distribution of the energy savings for a particular strategy, we rst calculate the individual savings for each EWH and then plot the distribution of the individual savings over all 77 EWHs. For example, for each individual EWH we use the following formulas to compare the energy savings per day achieved by EM and the baseline TC strategy and then plot the distribution of savings for the EM strategy over all 77 EWHs: Pelecjh;EM(kWh/d) = Pelecjh;TCPelecjh;EMkWh/day (75) Pelecjh;EM(%) =Pelecjh;TCPelecjh;EM Pelecjh;TC100% (76) 4.4. Distribution of results over all EWHs using two-node planning Table 4: Energy, temperature, volume, and cold event results for the two-node model. TC TM EM EML Vhot (L=day) 119 119 231 192 Eelec (kWh=day) 7.3 6.7 5.6 6.0 Edraw (kWh=day) 4.7 4.7 4.7 4.7 Eloss (kWh=day) 2.5 2.3 1.1 1.5 Tusage (C) 69.3 68.9 40.2 54.5 Eelec(kWh) (kWh =day) { 0.3,0.6,1.0 1.4 ,1.6,1.7 0.9 ,1.2,1.4 Eelec(%) % { 3.7,6.3,14.7 13.3 ,21.9,33.2 11.9 ,16.2,18.8 Cold events0,0,0 0 ,0,0 0 ,0,0 0 ,0,0. Note: The distributions are reported as 25thpercentile, median ,75thpercentile Cold events are taken as the total cold events for the EWH that uses the 25thpercentile, median , and 75thpercentile of the total volume of water used. The e ects of the four control strategies on energy and temperature using a two-node model, i.e. taking strati cation into account, are summarised in Table 4 and shown in Figure 4. Figure 4 (a) shows TM, EM and EML used less electrical energy than TC. The reductions are shown as distributions in Figures 4 (e) and (f), as reductions in daily energy (kWh =day) and percentage, respectively. Temperature-matched optimisation The median electrical energy used for TM was 6 :7 kWh=day, which is 8 :2 % less than the 7 :3 kWh=day for TC. Figure 4 (c) shows that the outlet temperature during events was about the same for TM and TC, despite the reduction in electrical energy used, and the median temperature di erence was only 0 :4C. The 19 (a) (b) (c) (d) (e) (f) Figure 4: Energy and temperature results for the di erent control strategies represented as distributions for 77 water heaters using a two-node model. (a) shows electrical energy used per EWH per day, (b) shows thermal energy drawn per EWH per day, (c) shows outlet temperatures during usage events, (d) shows thermal losses per EWH per day, and (e) and (f) shows the savings achieved in electrical energy per EWH, respectively as a reduction in kWh per day and as a percentage of total used. 20 number of cold events did not increase for TM and stayed at 115 out of a total of 15 581 events. Of the 77 EWHs, only nine had at least one cold event and the EWH with the most cold events had 61 of the 521 events (11:7 %). The occurrence of these cold events was inevitable since they also occurred for TC. Looking at the electrical energy reduction in Figures 4 (e) and (f), we nd that the reduction for TM was [ 0.3,0.6, 1.0] kWh=day ([ 3.7,6.3,14.7] %). Figure 4 (d) shows that the median thermal losses decreased from the 2:5 kWh=day for TC to 2 :3 kWh=day. Energy-matched optimisation As expected, the average outlet temperature for EM and EML was considerably lower than for TC and TM. The median for EM was 40 :2C and for EML 54 :5C. These lower temperatures meant that the average volume draw increased from 119 L to 231 L for EM and 192 L for EML and the median standing losses decreased by 1 :4 kWh=day (56 %) for EM and 1 :0 kWh=day (40 %) for EML from 2 :5 kWh=day for TC. This explains why much less electrical energy was used for EM: a median of 5 :6 kWh=day. This was 1:7 kWh=day (23:3 %) less than the 7 :3 kWh=day median for TC and reduced the energy usage by [ 1.4,1.6, 1.7] kWh=day and [ 13.3,21.9,33.2] %. The number of cold events did not increase for EM. Energy-matched optimisation with Legionella sterilisation The median electrical energy used for EML was 6 :0 kWh=day, 17:8 % less than for TC. This reduced the energy usage by [ 11.9,16.2,18.8] % and [ 0.9,1.2,1.4] kWh=day. The number of cold events did not increase for EML. 4.5. E ects of accounting for strati cation in the planning for a single EWH The e ect of accounting for strati cation is determined by the using the optimal heating schedule planned for a one-node EWH model and simulating it with the corresponding hot water pro le on a two-node EWH model. This section compares the simulation results for the optimal plan designed for the one-node EWH model in Booysen et al. (2019) with those for the two-node EWH model described in this paper. We simulate both optimal strategies on a two-node EWH and compare the di erences in results. We expect to encounter model inaccuracies for the two-node EWH when simulating it with a one-node optimal heating strategy. The temperature feedback controller will come into e ect to ensure that the temperature of the EWH follows the one-node optimal temperature trajectory. The mixer and the user are also simulated, to ensure that the thermal energy drawn for events is equivalent to that of TC simulated for a two-node EWH model. Figure 5 compares simulation results for a two-node EWH with a heating schedule using dynamic pro- gramming for a one-node model (solid lines) and A* for a two-node model (dashed lines). Figures 5 (a) and (b) show the results for the TM and EM strategies respectively. The temperature trajectory shown in red is the optimal planned temperature for a one-node model. Figures 5 (a) and (b) show example simulation results for a single two-node EWH over a 24-hour time period for TM and EM control. For each gure, the simulated two-node EWH temperatures (solid and dashed lines show one- and two-node planning, respectively), outlet ow rate and heating element control signal are plotted for the one-node planning. The upper node TC temperature is also indicated. The red temperature trajectory shown is the optimal temperature planned for a one-node EWH model and is used as the reference signal for the temperature feedback controller. The one-node EWH assumes that the outlet temperature is equal to that of the whole tank. The feedback controller will therefore ensure that the temperature of the whole tank follows the optimal temperatures. Temperature matching (TM): Figure 5 (a) shows that the outlet temperature of the tank is equal to that of TC for both the one-node and two-node planning just before the water event at t= 6. After the event, the lower node temperature for the EWH that uses the one-node planning is immediately raised again to the optimal temperature trajectory to ensure that all the water in the tank is at the optimal temperature. This di ers from the temperature for the EWH that uses two-node planning, as it is only required that the outlet temperature, which is the upper node temperature, follows its optimal temperature plan. This means that the simulations of the one-node planning will generally have a slightly higher tank temperature and the di erence will be observable in the standing losses. The electrical energy usage for TM was 8 :2 kWh for the one-node planning and 8 :0 kWh for the two-node planning over the 24 hours. 21 (a) Temperature matching (b) Energy matching Figure 5: Simulation results for TM and EM. The plots show the optimal temperature trajectory planned for a one-node EWH (red line), the temperatures achieved by the two-node EWH simulation (solid lines), and the temperatures achieved with the A*-based method that uses the two-node model (dashed lines). TTC,U is repeated on all plots with a dashed black line. Tusage is indicated with a dashed black line at 40C. Energy matching (EM): Figure 5 (b) shows that the outlet temperature of the tank is raised notably higher for the one-node than the two-node planning just before the water event at t= 19. As with TM, the feedback controller ensures that all the water in the tank is at a temperature that will avoid a cold event for the one-node planning. This means that the energy usage is higher than for the two-node planning and the strategy only ensures that the upper node temperature avoids cold temperatures during usage. The electrical energy usage for EM was 9 :45 kWh for the one-node model and 8 :2 kWh for the two-node model over the 24 hours. 4.6. Metrics for comparing results of optimal planning without accounting for strati cation In Booysen et al. (2019) we planned the optimal heating schedule for a one-node EWH model using dynamic programming to determine how much energy could be saved for the control strategies TM, EM and EML. In the present paper we now explore the e ect of accounting for strati cation by using the optimal heating schedule planning for a one-node model and simulating it with the corresponding water schedule for a two- node model. The di erences in electrical energy used per day when the newly designed two-node model EWH uses the optimal heating schedule and temperature planning of the previously implemented one-node model are calculated using PNs elecjh=P2 elecjhP1 elecjhkWh/day (77) where the subscript  Nsrefers to the di erence between the types of planning and subscripts 1 and 2 refer to the results of simulations using one-node or two-node model planning respectively. We apply similar modi cations to calculate the di erence in thermal energy used per day, average outlet temperature during usage events, thermal energy losses per day and energy savings per day. The distributions are plotted in Figure 6. 4.7. E ects of accounting for strati cation in the planning for multiple EWHs In this section we discuss the statistical results from all of the simulations performed for all 77 EWHs. We de ne a metric to asses the performance of the EWH-speci c changes during the simulation. We compare the simulation results for TM, EM and EML obtained in Figure 4 for two-node EWHs with those obtained from the one-node planning. The EWH-speci c changes in the results are summarised in Figures 6 (a) to (f). These distributions indicate the e ect of accounting for strati cation when determining the optimal planning for an EWH. 22 (a) (b) (c) (d) (e) (f) Figure 6: Di erences in energy and temperature results of the di erent control strategies represented as distributions for all water heaters, where the results from the one-node EWH planning is subtracted from that of the corresponding two-node results using Equation 77. (a) shows the di erence in electrical energy used per EWH per day, (b) shows the di erence in thermal energy drawn per EWH per day, (c) shows the di erence in outlet temperatures during usage events, (d) shows the di erence in thermal losses per EWH per day, and (e) and (f) shows the di erence in savings achieved in electrical energy per EWH, respectively as a reduction in kWh per day and percentage points of total used. 23 Figure 6 (b) shows that the thermal energy usage did not change between the two types of planning, and they therefore used the same amount of thermal energy as TC. Figure 6 (a) shows that the electrical energy usage increased when strati cation was not accounted for, with median increases of 0 :08 kWh=day for TM, 0:4 kWh=day for EM, and 0 :4 kWh=day for EML. The increased energy usage for TM is shown in Figure 6 (d), where the slight median increase of 0 :03 kWh=day in standing losses is a result of the temperature feedback controller ensuring that the whole tank matches the outlet temperatures for the one-node optimal planning. For EM and EML, the electrical energy increase can be explained by a combination of the increased standing losses of 0:2 kWh=day for both control strategies and the sizeable outlet temperature increase during usage events shown in Figure 6 (c) of up to 8C for EM and 7 :6C for EML. As Figures 6 (e) and (f) show, the electrical energy reduction relative to the baseline TC was decreased by [0.01,0.08,0.17] kWh=day, or [ 0.1,1.2,2.5] percentage points for the TM control; [ 0.2,0.4,0.6] kWh=day, or [ 4.0,5.4,7.0] percentage points for the EM control; and [ 0.2,0.4,0.6] kWh=day, or [ 4.0, 5.5,7.0] percentage points for EML when strati cation was not accounted for in the optimal planning. The number of cold events did not change from one type of optimal planning to another for any of the control strategies. 4.8. E ects of accounting for strati cation in the planning Comparing the simulation results from the one- and two-node EWH model planning, we nd that the e ect of accounting for strati cation did not have a big impact on TM, where the electrical energy savings increased by 1.2 percentage points when strati cation was accounted for. However, the impact was much bigger for EM and EML. This is largely because the one-node planning increases the temperature before usage events to ensure that the whole tank does not fall below 40C. When strati cation is not accounted for in the planning, the electrical energy savings drop by 5.4 percentage points for EM and 5.5 for EML. The temperature feedback controller ensured that the number of cold events did not change from one type of optimal planning to another for any of the control strategies. 5. Conclusion Heating water contributes substantially to total domestic electrical energy usage. Users in some countries tend to pay a at fee per kWh, which makes them sensitive to how much energy they use rather than when they use it. A broader concern is greenhouse gases caused by fossil fuels used for electricity in these countries. To investigate how much electrical energy can be saved by an optimal temperature control schedule, we examined 77 energy-storing electric water heaters over four weeks, a week for each season. We compared our baseline strategy, thermostat control (always on), TC, with three optimal control strategies. Our three strategies, using the A* algorithm to ensure optimal heating, ensure comparable delivery at the outlet of the water heater. The rst strategy, TM, provides a temperature-matched output with equal volume, the second, EM, provides an energy-matched output with lower but still hot temperature and increased volume, and the third, EML, adapts the second strategy to ensure that Legionella is sterilised - despite the lower temperatures. The median savings were 6 :3 % for TM, without adversely a ecting the temperature at which water is delivered, 21 :9 % for EM, with a reduction in energy lost to the environment, and 16 :2 % for EML. For all three strategies, the number of cold events did not increase from the baseline strategy, TC. We also compared the results for the three heating strategies taking strati cation into account. This produced only a small increase of 1.2% in the median saving for TM, but a 5.4% increase for EM and 5.5% for EML. By using real world, not synthesised, hot water usage pro les, we determined the absolute best energy savings, with no additional cold events, that can be achieved with scheduling and temperature control while not inconveniencing to the consumer. Acknowledgements We thank the following organisations for funding: MTN (S003061) and the WRC (K1-7163), and Eskom (TESP-2019). 24