Paper:
Data center cooling using model-predictive control
Despite the impressive recent advances in reinforcement learning (RL) algorithms, their deployment to real-world physical systems is often complicated by unexpected events, limited data, and the potential for expensive failures. In this paper, we describe an application of RL "in the wild" to the task of regulating temperatures and airflow inside a large-scale data center (DC). Adopting a data-driven, modelbased approach, we demonstrate that an RL agent with little prior knowledge is able to effectively and safely regulate conditions on a server floor after just a few hours of exploration, while improving operational efficiency relative to existing PID controllers.

Review:
This paper applies model predictive control (MPC) for data center cooling, i.e. regulating air temperature and air flow on server room/floor by controlling blower speeds and water flow at air handling units (AHUs). The dynamic model for the MPC is assumed to be a linear autoregressive model and its parameters are learned from randomized exploration over controls. The performance of the approach is evaluated by field experiments at a data center (DC), discussed and compared with the classical/traditional PID control.  Pros - This is a very practical topic which has great potentials for real-world applications. It is exciting to see the field experiments have been performed (I believe the field experiment is expensive) and the results are promising.   - This paper is clearly written, well-organized. The high presentation quality makes this paper an enjoyable read.  Cons - The theory/methodology contribution is limited. The paper is more of a demonstration that we can applying existing techniques to a DC for substantial improvements. - I don't like the comparison of computation time with cvxpy in Section 4.3, line 185. The computation time largely depends on programming language, computing hardware, and many details in implementing the optimization library. Besides, given the cooling is a slow process, I'm curious if we can increase our step to be more than 30s.  Other Questions/Comments  The proposed approach relies on little or no prior knowledge. For an operating DC which has historical recordings, will these historical data somehow help with the model learning, if we want to upgrade its cooling from PID to MPC? I was thinking this because I believe obtaining its historical data is not expensive/difficult.   Regarding the CAT/DP sensors. For "failed sensors", could you please add comments on the sensors' failing rates? For "measurement redundancy", how close are the measurement/readings from nearby sensors? I was thinking the adverse impacts of the failed sensor to the control and whether it is beneficial to build a model which contains every sensor's reading.  In Fig2, could you please add some explanations/descriptions for the model structure on shared parameters among local models. Is there any symmetry in the structure? I was thinking, (assuming the AHU are located on a 2*6 matrix, zero-based), why AHU_0_0's (top row, leftmost) red arrow points to the right, while AHU_0_5's (top row, rightmost) red arrow also points to the right? Why Column_4 is different from Columns 0/1/2/3 in that its red arrows point to the right?    In the exploration stage (Section 4.2), for the safety considerations, are you only limiting the size of control step? Are you also monitoring the temperatures at critical points?   In the experiments analysis, the entering water temperature (EWT) is quite different between Model3 and Model1/2. Could you please comment on how will EWT impact the cooling? Is EWT a big impact to cooling efficiency? How difficult is it to make EWT consistent for all experiments, thus that we have a fairer comparison?  ======= Phase 2 ========= I have read the response from authors. Thanks for the explanations!