# Heat Conduction Plate Layout Optimization Using Physics-Driven Convolutional Neural Networks
## 4 PD-CNN Model Training  
The CNN surrogate model, which is shown in Figure 4, is used for the optimization tasks described in the next section. The finite element method FEM model can provide an accurate temperature prediction for heat transfer. However, the optimization based on the FEM model could be infeasible, as discussed above. The FEM can only provide solutions with precise boundary conditions and layouts. Therefore, the result is not given in an explicit form. The PD-CNN model, in contrast, is an accurate mapping between the layouts and corresponding solutions and can predict the temperature distribution for any randomly given geometry. Therefore, a CNN surrogate model could be a suitable choice for layout optimization. Figure 4 shows the input and output of the CNN surrogate model in the case of four holes. Ten randomly generated masks are used as the input for the CNN surrogate model during every epoch. The masks also change with different epochs. As shown in Figure 5a, the Laplace residual value drops exponentially in the beginning; after approximately 20 iterative steps, the convergence of the Laplace residual exhibits minor differences. When the iterative training step reaches 500k, the value decreases to be sufficiently small. After then, the Laplace residual oscillates around 0.35, as shown in Figure 5b, which is considered converged. The whole converge curve is very similar to the curve using the combined data- and physics-driven the method in Ref. [41]. The training process with 500,000 epochs requires around 9 h using the following device: Intel Core i5-10300H CPU and NVIDIA GeForce RTX 2060. Figure 4: The temperature distribution prediction using the trained PD-CNN surrogate model. Figure 5: Convergence curve. (**a**) Iterative steps: 1–500 k; (**b**) Iterative steps: 499,500–500 k.  
The training process is shown in Figure 6. Here, 100,000 are set as the maximal epochs. In each generation, the GPU trains ten different cases as one batch, and the learning rate is specified as 0.001. The positions of masks in each epoch are randomly generated. The PD-CNN is used to obtain the solutions to multiple cases with a unique surrogate model. The input masks are varied in a moderate range in the training stage. The fixed network can generate the corresponding temperature field. The setup of the mask position is shown in Table 1, as discussed earlier. Training loss is relatively significant at the beginning of the training process, and there are many errors in local temperature fields. These inconsistency errors show a significant Laplace loss, which cannot predict the temperature field very well. As the epochs increase, the inconsistency of the temperature field decreases, and the results seem to be more reasonable, shown in Figure 6 The training loss decreases very fast at first and then keeps stable. Although the loss does not decrease, the results keep becoming better after 10,000 epochs. In the end, the model adapts to the random size of different masks, which completely covers the surrogate model's range. As shown in Figure 6, the mask changes in every epoch in the learning process, and the contours of the temperature field become smoother. As the advantage of physics-driven learning, the relation of adjacent points is constrained by the Laplace equation. For this reason, the values do not vary dramatically. Then in epoch 1000, a significant error spot appears, denoted as a peak in which the value is round 3, see Figure 6, and the residuals near the domain's boundaries are negative. The whole temperature field is much different from the proper solution. Finally, from epoch 15,000, the global structure is stable and very  
Figure 6: The training evolution of temperature field solution from epoch 0 to 500,000 for the single hole case. similar to the proper solution after the gradual disappearance of the peaks. However, there are still residuals near the boundaries of holes, which is a specific numerical solution to an unsteady heat conduction problem with a Dirichlet boundary condition [41].