Paper: Climate-sensitive Urban Planning through Optimization of Tree Placements (ICLR-2024)

Reviewer comment (Novelty):
No technical contribution in ML. Even the considered surrogate model is a simple U-Net. It is not used for anything other than approximation of Solweig. If it gives computational benefits, this aspect is never discussed, nor is the quality of approximation evaluated.

Author response:
We kindly refer the reviewer to Section 4.1 that thoroughly compares our method to SOLWEIG. Next, we provide further details about the computational speed-ups of our method: we report speed-ups of factors of 30, 17, 44000, and 400000 for the time periods: day, week, year, or decade, respectively, when comparing using our model that directly outputs the aggregated mean radiant temperature to first predicting the mean radiant temperature for each time step and then aggregating over all time steps. Note that using deep learning methods compared to physical modeling with SOLWEIG achieves another factor of ca. 22x [1]. Note that prior works exclusively used physical simulation models. To contextualize the amount of speed-up we gain, let us consider an example: consider that the optimization takes 1 minute with our method and optimizes the tree positions for the time period decade. Consequently, the estimated runtime for SOLWEIG would be 1 min\*400000\*22=ca. 16.73 years.

As mentioned by the reviewer, the main contribution of our work is the introduction of a “novel spatial optimization problem with a very relevant application”. Prior work mostly focused just on computational efficiency or improved modeling, i.e., improving the forward direction. In contrast, we consider the backward direction, i.e., making interventions in the input to optimize some physical quantity, i.e., in our case reducing the mean radiant temperature. Besides that we do *not* use it as a surrogate model but learn to directly predict aggregated results which goes beyond a simple replacement.

Reviewer comment (Experiments):
The optimization seems to be done one tree at a time, rather than jointly over an entire configuration. This need not be a weakness, but the approaches need to be compared.

Author response:
After greedily placing trees subsequently, the iterative search jointly optimizes all tree positions. We note that sequential placements would incur significantly more computational costs and, thus, we did not run such.

Reviewer comment (Experiments):
The problem is solved in rather artificial settings, without considering many realistic effects (as mentioned by the authors themselves, in the limitations section).

Author response:
We note that many limitations could be overcome in future work. The main issue for most limitations is the data bottleneck, i.e., the data often is not available, or limitations of physical modeling (refer to Appendix C for more details in the revised manuscript). Both could be overcome by future advances in easier access, e.g., by means of estimating them through satellite imagery, as well as advances in physical modeling.

Reviewer comment (Novelty):
The proposed approach seems to be too specific to one application to be of sufficient interest to the ICLR audience in general.

Author response:
We kindly refer the reviewer to Section 4.1 that thoroughly compares our method to SOLWEIG. Next, we provide further details about the computational speed-ups of our method: we report speed-ups of factors of 30, 17, 44000, and 400000 for the time periods: day, week, year, or decade, respectively, when comparing using our model that directly outputs the aggregated mean radiant temperature to first predicting the mean radiant temperature for each time step and then aggregating over all time steps. Note that using deep learning methods compared to physical modeling with SOLWEIG achieves another factor of ca. 22x [1]. Note that prior works exclusively used physical simulation models. To contextualize the amount of speed-up we gain, let us consider an example: consider that the optimization takes 1 minute with our method and optimizes the tree positions for the time period decade. Consequently, the estimated runtime for SOLWEIG would be 1 min\*400000\*22=ca. 16.73 years.

As mentioned by the reviewer, the main contribution of our work is the introduction of a “novel spatial optimization problem with a very relevant application”. Prior work mostly focused just on computational efficiency or improved modeling, i.e., improving the forward direction. In contrast, we consider the backward direction, i.e., making interventions in the input to optimize some physical quantity, i.e., in our case reducing the mean radiant temperature. Besides that we do *not* use it as a surrogate model but learn to directly predict aggregated results which goes beyond a simple replacement.

Concerning the venue, we believe that ICLR is a good platform for our work. Our aim is to prompt ML experts leveraging their skills across various scientific domains to reconsider the prevailing emphasis mostly exclusively on computational efficiency or general forward modeling. Instead, we would be happy if the takeaway of our paper for them would be to consider alternative ML applications in the scientific domain more often, such as optimization of some physical quantity.

Reviewer comment (Reproducibility):
Algorithm 1 suggests that the MRT seems to have been pre-computed for placing each tree separately. But in each iteration of the GA, the existing tree placements are partially perturbed. Do we not need to recompute the MRT once again for each perturbation?

Author response:
Thanks for pointing out the missing evaluation in the algorithm. We indeed need to evaluate the new candidate. We fixed the algorithm in the revised manuscript.

Reviewer comment (Experiments):
Is U-Net the best model for generating the spatial maps? Have different architectures been compared? I didn't see much discussion about this.

Author response:
Our approach is not specific to the model architecture. The goal of our work is not just to show good numbers but to outline a novel optimization problem that uses deep learning approaches for the optimization of physical quantities. Consequently, the model architecture is a non-important implementation detail in our opinion and future work could improve upon it.

Reviewer comment (Novelty):
Several limitations have been mentioned in the paper itself. Can those be overcome within the proposed framework itself, or do they need completely different approaches?

Author response:
We note that many limitations could be overcome in future work. The main issue for most limitations is the data bottleneck, i.e., the data often is not available, or limitations of physical modeling (refer to Appendix C for more details in the revised manuscript). Both could be overcome by future advances in easier access, e.g., by means of estimating them through satellite imagery, as well as advances in physical modeling.

Reviewer comment (Reproducibility):
Unclear what data has been used. Does the CityGML data come from [this](https://www.ogc.org/standard/CityGML/) homepage and is this all simulation data, or real observations? As far as I know, ERA5 data is only available on 30m resolution; how do you get to 1m resolution? Where do your digital elevation and surface models, your land cover, wall aspects and height, and sky view factor maps come from?

Author response:
We obtained the pre-processed data from Briegel et al [1]; refer for details to them. They were provided with spatial data and obtained past meteorological measurements from the internet. The input data is real data (from the point in time it was recorded) but mean radiant temperatures were simulated with SOLWEIG, as recording real measurements of mean radiant temperature is challenging (see below for a brief discussion on why).

Regarding ERA5 data: ERA5 Land data has a spatial resolution of ca. 9km x 9km, nor does it contain mean radiant temperatures. Please refer to [2] for further details. Note that all study areas are within the same ERA5 Land grid cell. This means, all study areas have the same global meteorological forcing data. Thus, we consider it as constant within our 500m x 500m areas.

Reviewer comment (Experiments):
Benchmarking traditional physical model would be highly appreciated to understand the performance of your U-Net in terms of accuracy. That is, does your method incur a trade-off between accuracy and efficiency, and, in particular, how accurate is your U-Net to approximate $T_{mrt}$?

Author response:
We compare our modeling with SOLWEIG in Section 4.1. We find that the U-Net to SOLWEIG error (i.e., 1.93K) is smaller  than the previously found errors of SOLWEIG to real sparse measurements (i.e., 4.4K; refer to Table 5 in [1]). Consequently, it indicates that U-Net is a well-suited computational shortcut and does not (or perhaps a negligible) trade-off between modeling performance and efficiency. Note that this is not typically the case when replacing a physical model with some data-driven approach.

Reviewer comment (Evaluation):
Is SOLVEIG a traditional model and do you benchmark this? Please help readers by clarifying this more thoroughly. What does an `L1 error of 1.93K` mean, is it computed between U-Net output and ground truth, or between U-Net output and SOLVEIG output?

Author response:
Yes, SOLWEIG is a popular physical simulation model for estimation of mean radiant temperature. We provide details on SOLWEIG in Appendix C.

Regarding the L1 error: it is computed between the U-Net’s and SOLWEIG’s estimation. Note that obtaining real measurements of mean radiant temperatures is difficult and monetarily expensive, as both short- and longwave radiation from six directions need to be measured. In addition, such measurements can only be made for single points, which in turn makes comprehensive evaluation over an entire spatial area challenging.

Reviewer comment (Experiments):
Given that the solar incoming radiation $I_g$ has the biggest impact, how does the tree position make a difference? An ablation comparing the effect on $T_{mrt}$ caused by number of trees vs. the positioning of the trees would be interesting. Also, seing Figure 4, I do not quite agree that the number of trees saturates (although it certainly will at some point). In short: Does the number of trees or their position contribute more to a better $T_{mrt}$? (I have seen your comment `Notably, the improvement by relocation of existing trees is significantly larger than the effect of 50 added trees [...]`, but I could not quite understand the quantitative effect of each treatment, i.e., #trees vs position).

Author response:
We added a comparison of the effect of planting more trees vs. optimizing or not optimizing their position in Appendix F.3. The results highlight the importance of our optimization, i.e., not optimizing would only harness ca. 50% of their potential in reducing mean radiant temperatures.

Reviewer comment (Presentation):
In Figure 1: What does the legend show, is it difference in $T_{mrt} [K]$ compared to a no-trees condition? Where are the trees placed (maybe indicate with opaque green circles)?

Author response:
It is the difference between the status quo to the status after planting an additional 50 trees. We added green crosses in the figures to indicate the tree positions.

Reviewer comment (Experiments):
Have you considered other tree positioning approaches, such as projecting error gradients in the U-Net's $T_{mrt}$ prediction on the positions of the trees? That is, somewhat similarly to [[1]](https://arxiv.org/abs/1904.09019), where optimal node-positioning was optimized via gradients in a GNN.

Author response:
Yes, but it yielded very poor results. We mainly attribute this to the fact the discretization step was causing issues. We believe modeling mean radiant temperature with a GNN may be fruitful for future work and would allow for such an optimization.

Reviewer comment (Experiments):
What type of trees have been investigated? To what kind of tree does $t_g$ used in your experiments relate?

Author response:
We note that current physical simulation models, such as SOLWEIG, are not capable of distinguishing types of tree. We added a list of limitations of the physical model SOLWEIG in Appendix C to make ML experts aware of such limitations that our approach consequently inherits. Our approach would benefit from advances in incorporating such in physical modeling with SOLWEIG.

Reviewer comment (Evaluation):
Why does your algorithm suggest to populate east-to-west passages more densely? Wouldn't this prevent wind to transport hot air out of the city in summer? And following up on this, what tree positions does your algorithm predict when exclusively optimizing for winter or summer months?

Author response:
Our study focuses on mean radiant temperature and, thus, does not consider other effects, such as wind, air temperature or humidity. Thus, we outlined this limitation in Section 5 and proposed it as a next step to integrate them. For example, one could extend our approach to UTCI, a popular thermal comfort index that also includes such factors.

We optimized tree positions for winter and summer months and provided the additional results in Appendix F.2. We find that trees are predominantly positioned in larger, open areas and clustered together during the summer, whereas they are more broadly spread over the neighborhood during the winter.

Reviewer comment (Novelty):
In my opinion, the methodological contributions of the paper are thin, and essentially boil down to 1) estimating the aggregated radiant temperatures directly instead of individually and 2) integrating this estimation in a standard local search procedure. These are both fairly straightforward. The "theoretical analysis" in Section 3.1 is extremely tenuous and, in my opinion, should not be branded as such.

Author response:
Our work aims to formulate a problem that can be solved by deep learning techniques. Compared to previous works in physical modeling with deep learning that (*passively*) typically just aimed at computational efficiency or improved modeling in the forward direction, we instead take an *active* position by optimizing the input s.t. it optimizes a physical measure to achieve a certain goal (backward direction).

Reviewer comment (Evaluation):
The evaluation does not include error bars and confidence intervals, which are a must for drawing reliable conclusions from the results, given the stochasticity of the methods (bar greedy search which is deterministic; but given the estimation itself is learned, one could consider an experimental design where the "seed" for the estimation varies in a paired fashion).

Author response:
As suggested by the reviewer, we have added error bars for the optimization of trees in our comparison of optimization strategies (Table 1). It clearly shows the superiority of our optimization method above common baselines for black-box optimization.

Reviewer comment (Theory):
There is a potential fundamental limitation in the considered objective, which does not include any measure of population density, traffic, or footfall for the areas.

Author response:
We also agree this is a limitation (as discussed in the limitations section). However, this is not a limitation of our method. Indeed, we can integrate such factors into the objective function if such data (e.g., the population density) is available. Consequently, it can be ensured that trees are planted in areas, in which they effectively improve the thermal comfort of individuals where they reside.

Reviewer comment (Evaluation):
The omission of population density, traffic, or footfall is potentially problematic as the optimization procedure may find solutions where trees are placed in places that yield good deltas in the temperature metric, and yet are comparatively less (or not at all) populated.

Author response:
We also agree this is a limitation (as discussed in the limitations section). However, this is not a limitation of our method. Indeed, we can integrate such factors into the objective function if such data (e.g., the population density) is available. Consequently, it can be ensured that trees are planted in areas, in which they effectively improve the thermal comfort of individuals where they reside.

Reviewer comment (Evaluation):
The placements may therefore not lead to decreases in thermal comfort experienced by individuals, given that an area is not actually frequented by people.

Author response:
We also agree this is a limitation (as discussed in the limitations section). However, this is not a limitation of our method. Indeed, we can integrate such factors into the objective function if such data (e.g., the population density) is available. Consequently, it can be ensured that trees are planted in areas, in which they effectively improve the thermal comfort of individuals where they reside.

Reviewer comment (Writing):
This limitation is potentially alluded to in the text (bottom of page 8) and acknowledged in Section 5, but it deserves to be addressed in more depth.

Author response:
We also agree this is a limitation (as discussed in the limitations section). However, this is not a limitation of our method. Indeed, we can integrate such factors into the objective function if such data (e.g., the population density) is available. Consequently, it can be ensured that trees are planted in areas, in which they effectively improve the thermal comfort of individuals where they reside.

Reviewer comment (Evaluation):
It may be that the density is sufficiently uniform in the considered grid that this is not an issue. Evidence that the approach does not "game" the objective would also counteract this point.

Author response:
We also agree this is a limitation (as discussed in the limitations section). However, this is not a limitation of our method. Indeed, we can integrate such factors into the objective function if such data (e.g., the population density) is available. Consequently, it can be ensured that trees are planted in areas, in which they effectively improve the thermal comfort of individuals where they reside.

Reviewer comment (Experiments):
Another means of strengthening the evaluation is to include other, more competitive baselines from the literature. Is there no such method? Given the improved scalability attained by your method, what is the point at which prior methods (e.g. that estimate the temperature in a more granular fashion) are too slow? Some more extensive benchmarking than the figures quoted in the text in 4.1 would help.

Author response:
Note that prior work only used areas of sizes of ca. 50m x 80m [1] or 30m x 80m [2], i.e., our study areas are roughly two orders of magnitude larger. Further, they optimized for a maximum number of 8 [1] or 14 [2] trees, whereas we optimized for up to 200 trees. This makes our approach hardly comparable to prior work.

Regarding the computational speed-ups of our method: we further report speed-ups of factors of 30, 17, 44000, and 400000 for the time periods: day, week, year, or decade, respectively, when comparing using our model that directly outputs the aggregated mean radiant temperature to first predicting the mean radiant temperature for each time step and then aggregating over all time steps. Note that using deep learning methods compared to physical modeling with SOLWEIG achieves another factor of ca. 22x [1]. Note that prior works exclusively used physical simulation models. To contextualize the amount of speed-up we gain, let us consider an example: consider that the optimization takes 1 minute with our method and optimizes the tree positions for the time period decade. Consequently, the estimated runtime for SOLWEIG would be 1 min\*400000\*22=ca. 16.73 years.

Reviewer comment (Writing):
Lack of technical description of the algorithm. It is not clear how the hill-climbing algorithm works. Methods use domain-specific terminology that have not been adequately explained for an ML conference reader.

Author response:
### Additional technical description

We revised the manuscript and, e.g., added missing explanations of the hill-climbing algorithm. We would provide further feedback if the reviewer sees other parts that may require improvement.

Reviewer comment (Novelty):
Application of standard ML methods, novelty is in a narrow application area.

Author response:
### Novelty in narrow application area

We would like to note that the majority of previous work on physical modeling focused on improving computational efficiency or improved modeling in the forward modeling. Our work instead focuses on finding interventions in the input, i.e., locations of trees, to optimize a physical quantity (backward direction). Our case study extends beyond the scope of the presented application area and may provide an interesting direction for future work in using deep learning approaches to support humans in challenging optimization problems.

Reviewer comment (Experiments):
500m x 500m is not a "large neighborhood". Prior state-of-the-art is not stated.

Author response:
### 500m x 500m is not a large neighborhood; what area is covered by previous work, and missing state-of-the-art

We kindly disagree. Note that prior work only used areas of sizes of ca. 50m x 80m [1] or 30m x 80m [2], i.e., our study areas are roughly two orders of magnitude larger. Further, they optimized for a maximum number of 8 [1] or 14 [2] trees, whereas we optimized for up to 200 trees. This makes our approach hardly comparable to prior work.

Reviewer comment (Experiments):
No details on the size of the training data, separation of train-test split details is provided.

Author response:
### Missing dataset details and HP tuning

Thanks for pointing this out! We added the missing dataset details in Appendix B. Note that we did not perform hyperparameter tuning. We believe future work could already improve performance by optimizing the hyperparameters.

Reviewer comment (Experiments):
It is unclear how the hyper-parameters are tuned. Is there a separate held-out dataset?

Author response:
### Missing dataset details and HP tuning

Thanks for pointing this out! We added the missing dataset details in Appendix B. Note that we did not perform hyperparameter tuning. We believe future work could already improve performance by optimizing the hyperparameters.

Reviewer comment (Experiments):
It would be good to see a cost comparison of planting a new tree vs transferring existing tree.

Author response:
### Cost comparison of planting new trees vs. transferring existing trees

We hope the reviewer understands that a cost comparison is out-of-scope for this work and may depend on many parts that are hard to control for. Nonetheless, we would like to point the reviewer to Section 4.4 that showcases that we could significantly better utilize the potential of urban trees by moving them to “better-suited” locations in terms of reduction in mean radiant temperature. This may be of interest in the consideration whether planting new trees or transferring existing ones is the most cost-efficient solution for a given city.

Reviewer comment (Evaluation):
What is the correlation value between mean radiant temperature and mortality?

Author response:
### Correlation value between mean radiant temperature and mortality

We kindly refer the reviewer to Table 1 and Figure 3 of Thorsson et al. [3] for details.

Reviewer comment (Novelty):
What is area covered by prior state-of-the-art in tree placement?

Author response:
### 500m x 500m is not a large neighborhood; what area is covered by previous work, and missing state-of-the-art

We kindly disagree. Note that prior work only used areas of sizes of ca. 50m x 80m [1] or 30m x 80m [2], i.e., our study areas are roughly two orders of magnitude larger. Further, they optimized for a maximum number of 8 [1] or 14 [2] trees, whereas we optimized for up to 200 trees. This makes our approach hardly comparable to prior work.

Reviewer comment (Evaluation):
I did not understand how the aggregated performance is worse than point prediction, the absolute number is lower.

Author response:
### I did not understand how the aggregated performance is worse than point prediction, the absolute number is lower

We compared the direct prediction of aggregated mean radiant temperature against making predictions for every time step and then aggregating the results. We did not compare with SOLWEIG’s results, as it is prohibitively expensive for our larger time periods (year & decade), and wanted to stay consistent within our comparisons. Below, we provide comparisons of the results of aggregated performance compared to SOLWEIG for both smaller time periods (day & week) that make clear why making point prediction and then aggregating is slightly better, though computationally significantly more expensive.

| type | day | week |
| :- | :-: | -: |
| nonaggregated         | 1.32        | 0.98         |
| aggregated         | 1.48         | 1.07         |

Reviewer comment (Evaluation):
Isn't it a good thing that the temperature increases during nights and winters when the environment is colder?

Author response:
### Isn't it a good thing that the temperature increases during nights and winters when the environment is colder?

Yes, this is indeed the case and it is beneficial. The effect can be seen in Figure 3. We did not delve into this deeper, as we chose to focus on mean radiant temperature reduction for combating heat stress.

Reviewer comment (Experiments):
How can you plant a big tree wherever you want in the city? Won't they need to be small and grow over time? The size of the trees considered is unrealistic.

Author response:
### How can you plant a big tree wherever you want in the city? Won't they need to be small and grow over time? The size of the trees considered is unrealistic. 

This is a simplifying assumption. As noted in our limitations sections, there are many other factors one has to consider that make the optimization problem substantially more challenging in practice. We note that we assumed a static urban environment (footnote 4), including static vegetation, i.e., trees are assumed to not grow over time. This is a common assumption due to the difficulty of modeling such dynamic processes. Note that the physical model SOLWEIG also makes this assumption and, as a consequence, our approach also inherits this limitation. We make this limitation stemming from the physical modeling more clear and added a list of limitations for SOLWEIG in Appendix C to make ML experts more aware of the limitations of physical modeling and, consequently, also our method. Finally, note that the tree size is a changeable parameter and depends on the user. We note that such trees are realistic in the considered study area; even though we admit they belong to the top 20-percentile.

Reviewer comment (Experiments):
How do you ensure the distribution of training data is sufficient? The optimization algorithm can potentially shift the distribution away from the training set.

Author response:
### How do you ensure the distribution of training data is sufficient? 

This is a great point! We trained the model on sufficient data (Appendix B). Further, we added an analysis in Appendix F.1 to confirm that the optimized tree locations actually lead to similar effects in the physical modeling with SOLWEIG and are not just mere artifacts of out-of-distribution changes. It confirms that it is the former, highlighting that the found solution is not a mere artifact of the deep learning method.

Reviewer comment (Evaluation):
Why is a reduction of 0.83K substantial? Please provide sufficient domain context for an average ML scientist can follow.

Author response:
### Why is a reduction of 0.83K substantial?; Did not understand how you got 20% reduction for 60C T_mrt.

The reduction is indeed substantial. We recognize that understanding the figure for non-domain experts may be challenging. To enhance the understanding, we also analyzed the change in peak instances, specifically the reduction in hours with heat stress, i.e., mean radiant temperature of above 60°C. To this end, we summed up all such hours of the entire study areas and calculated the reduction before and after the optimization. Our counterfactually-inspired optimization in Section 4.4 yielded a remarkable 20% decrease in hours during the hottest week in 2020. Note that  the number of trees stays the same in this experiment and no new trees are added.

To provide a deeper perspective, consider that only 6–8 hours a day (from late morning to late afternoon) typically have mean radiant temperature exceeding 60°C. During all other hours, including nighttime, mean radiant temperatures usually remain below this threshold. Consequently, the 20% reduction is confined to 8 hours a day, or a total of 56 hours a week. In practical terms, the 20% reduction translates to ca. 11.2 hours per week, or an average of 1.6 hours per day of reduced hours exceeding the threshold. This means an additional average of 90 minutes each day when individuals can enjoy a more comfortable outdoor experience in terms of thermal comfort.

Reviewer comment (Evaluation):
Did not understand how you got 20% reduction for 60C T_mrt. No details have been provided.

Author response:
### Why is a reduction of 0.83K substantial?; Did not understand how you got 20% reduction for 60C T_mrt.

The reduction is indeed substantial. We recognize that understanding the figure for non-domain experts may be challenging. To enhance the understanding, we also analyzed the change in peak instances, specifically the reduction in hours with heat stress, i.e., mean radiant temperature of above 60°C. To this end, we summed up all such hours of the entire study areas and calculated the reduction before and after the optimization. Our counterfactually-inspired optimization in Section 4.4 yielded a remarkable 20% decrease in hours during the hottest week in 2020. Note that  the number of trees stays the same in this experiment and no new trees are added.

To provide a deeper perspective, consider that only 6–8 hours a day (from late morning to late afternoon) typically have mean radiant temperature exceeding 60°C. During all other hours, including nighttime, mean radiant temperatures usually remain below this threshold. Consequently, the 20% reduction is confined to 8 hours a day, or a total of 56 hours a week. In practical terms, the 20% reduction translates to ca. 11.2 hours per week, or an average of 1.6 hours per day of reduced hours exceeding the threshold. This means an additional average of 90 minutes each day when individuals can enjoy a more comfortable outdoor experience in terms of thermal comfort.