A Bio-sensing and Reinforcement Learning Control System for Personalized Thermal Comfort and Energy Efficiency
Creators: name: Chenlu Zhang
Publisher: name: Carnegie Mellon University
Year: 2019
Subjects: subject: 120202 Building Science and Techniques
subject: FOS: Civil engineering
subject: FOS: Civil engineering

description: A comfortable indoor thermal environment plays a crucial role in preserving occupant health and productivity. In most office building today, the indoor thermal environment is regulated by heating, cooling, and air-conditioning<br>(HVAC) systems with static schedule-based rules. While prevalent, this control strategy has resulted in low thermal satisfaction rates and energy waste. A growing number of researchers are focusing on occupant-centric building<br>controls and applying various advanced control methods to improve thermal comfort and energy efficiency. However, it is still challenging to integrate occupants’ personalized requirements into a control system with a capability<br>of learning from the environment. This thesis has developed a bio-sensing and reinforcement learning control system for continuously integrating occupants’ bio-signals into the operation of different heating, cooling, and ventilation<br>systems, learning through interaction to achieve personalized thermal comfort and energy savings.<br>A bio-sensing and reinforcement learning control (Bio-REAL) system is comprised of a bio-sensing network, multiple Bio-REAL agents, and a negotiator. The bio-sensing network uses smart wristbands to measure occupants wrist temperature in real-time. The Bio-REAL agent initiates the best control actions on behalf of each occupant in response to the wrist temperature, subjective feedback, and environmental conditions. The negotiator resolves<br>conflicts in the control actions initiated by different Bio-REAL agents to maximize collective thermal comfort and minimize energy consumption. A state-of-art reinforcement learning algorithm, double Q learning with experiment<br>replay and neural network approximation, is applied to train the Bio-REAL agents. This thesis evaluates the Bio-REAL systems using three types of experimental techniques: simulation experiments, preliminary field and simulation<br>experiments, and field experiments. The simulation experiment trains a Bio-REAL system with three virtual occupants and an office room with a variable air volume (VAV) system in a heating season. The three virtual occupants are simulated using classic thermal comfort models. The room of a small-sized office building is modeled by the EnergyPlus simulation tool. The preliminary<br>heating season field and simulation experiments gather data from six occupants, providing inputs to create the personalized occupant models. The experimental test space is a room with water-sourced radiators for heating and modeled by the EnergyPlus tool. The co-simulation with personalized occupant models and EnergyPlus model assesses the performance of the Bio-REAL system. The cooling season field experiment evaluates the real-world<br>performance of the Bio-REAL systems with fourteen occupants in a tropical climate, occupying a studio with ambient temperature controls and shared controls of ceiling fans. The three types of experiments each demonstrated that the Bio-REAL system has more advantages for improving thermal comfort and energy efficiency<br>compared to the conventional control systems based on thermal comfort models and static schedules. With the combinations of bio-sensing and learning capability, the Bio-REAL system was able to derive dynamic and adaptive control policies, mapping occupants’ personalized requests and the changes of indoor and outdoor environmental conditions to optimum control actions. The Bio-REAL system contributes an innovative approach for controlling<br>building conditioning systems, to deliver thermal comfort for each individual at the lowest energy possible, with benefits for occupant health and productivity, as well as sustainability. The Bio-REAL research addresses individual<br>differences in thermal comfort for multi-occupant spaces with limited individual controls. It also addresses a range of heating and cooling choices from ambient to task systems. The structure and learning process of the Bio-REAL system, the strategies for the simulations, and the real-world implementation offer creative solutions for building control systems, contributing to the application of the Internet of Things and artificial intelligence in buildings. <br>