Title: Energy Footprinting and Human-Centric Building Co-Optimization with Multi-Task Deep Reinforcement Learning
Description: In the United States, commercial and residential buildings are responsible for 40% of total energy consumption, which provides an important opportunity for energy impact. As we spend the majority of our active moments during the day in transportation, commercial buildings, streets, and infrastructure, some of the greatest opportunities to reduce energy usage occur when we are outside of the home. A large percentage of energy consumption in the built environment directly or indirectly services humans; thus, there is a significant amount of untapped energy savings that can be achieved by involving humans in the optimization process. By including occupants in the building co-optimization process, we can gain a better understanding of individual energy responsibility and significantly improve energy consumption, thermal comfort and air quality over non human-in-the-loop systems and strategies. First, we present ePrints, a scalable energy footprinting system capable of providing personalized energy footprints in real-time. ePrints supports different apportionment policies, with microsecond-level footprint computation time and graceful scaling with the size of the building, frequency of energy updates, and rate of occupant location changes.

Finally, we present applications enabled by our system, such as mobile and wearable applications to provide users timely feedback on the energy impacts of their actions, as well as applications to provide energy saving suggestions and inform building-level policies. Next, we extend the idea of energy footprinting to the city-scale with CityEnergy a city-scale energy footprinting system that utilizes the city's digital twin to provide real-time energy footprints with a focus on 100% coverage. CityEnergy takes advantage of existing sensing infrastructure and data sources in urban cities to provide energy and population estimates at the building level, even in built environments that do not have existing or accessible energy or population data. CityEnergy takes advantage of LFTSys, a low frame-rate vehicle tracking and traffic flow system that we implement on New York City's traffic camera network, to aid in building population estimates. Evaluations comparing CityEnergy with building level energy footprints and city-wide data demonstrate the potential for CityEnergy to provide personal energy footprint estimates at the city-scale.

We then tackle the challenge of involving humans in the building energy optimization process by developing recEnergy, a recommender system for reducing energy consumption in commercial buildings with human-in-the-loop. recEnergy learns actions with high energy saving potential through deep reinforcement learning, actively distribute recommendations to occupants in a commercial building, and utilize feedback from the occupants to better learn four different types of energy saving recommendations. Over a four week user study, recEnergy improves building energy reduction from a baseline saving (passive-only strategy) of 19% to 26%. Finally, we extend the recommender system to co-optimize over energy consumption, occupant thermal comfort, and air quality. The recommender system utilizes a multi-task deep reinforcement learning architecture, and is trained using a simulation environment. The simulation environment is built using different models trained on data captured from a digital twin of a real deployment.

To measure occupant thermal comfort, the digital twin utilizes a real-time comfort estimation system that extracts and integrates facial temperature features with environmental sensing to provide personalized comfort estimates. We studied three different use cases in this deployment by varying the objective weights in the recommender system, and found that the system has the potential to further reduce energy consumption by 8% in energy focused optimization, improve all objectives by 5-10% in joint optimization, and improve thermal comfort by up to 21% in comfort and air quality focused optimization by incorporating move recommendations.