# Cross-Feature Transfer Learning for Efficient Tensor Program Generation
## 4 Evaluation
### Evaluation of Heterogeneous Transfer Learning  
Various transformations can be implemented on a given computation graph, which comprises tensor operations along with input and output tensor shapes, thereby influencing their performance on the target hardware. For instance, consider the conv2D tensor operation, where the choice of tiling is contingent upon whether the target hardware is a GPU or CPU, given the constraints imposed by grid and block size in GPUs. A tiling size deemed appropriate for a CPU may be unsuitable for a GPU, and not all combinations yield optimal performance. To identify jointly optimized schedules for a kernel and hardware, we leveraged the TVM auto-scheduler. Subsequently, we applied these optimized schedules to similar untuned kernels using an attention mechanism. To streamline the process, we organized the kernels by their occurrences and total contribution to the FLOPs' count, ensuring efficiency. The tuning process focused on refining a select few significant tensor operations. We conducted an evaluation of our methodology using three architecturally distinct networks on both CPU and GPU. In contrast to the baseline approach, where tasks were randomly tuned, we specifically selected tasks that contribute more to the FLOPs' count. As outlined in Table 9, our approach achieved mean inference times comparable to the baseline, while significantly reducing tuning time. On CPU, we observed a reduction in time of 30% for ResNet_50, 70% for MobileNet_50, and 90% for Inception_v3. However, ResNet_50 experienced a performance regression due to a lack of matching kernel shapes in the trained dataset for the given hardware. On the GPU, we achieved a remarkable 80-90% reduction in tuned time across all networks. The greater reduction in tuning time for GPUs can be attributed to the utilization of hardware intrinsics, leveraging the inherent higher parallelism in GPUs compared to CPUs. The standard deviation associated with the reported mean inference time in Table 9 ranges from 5% to 7%. In this evaluation, the tuner was trained on features extracted from neural networks and hardware. We conducted a thorough comparison of tuners based on the convergence epochs, considering both the baseline and sampled datasets. Following the design principles of TVM's auto-scheduler, it is anticipated that tuners like XGB and MLP will converge after a substantial number of trials. To ensure an equitable evaluation, we assessed their convergence in terms of epochs. As depicted in Figure 3, the convergence patterns for XGB, MLP, and LGBM tuners exhibit minimal differences between the two datasets. In contrast, our attention-inspired tuner showcased remarkable performance by converging in a comparable number of epochs while achieving a twofold improvement in error loss. \begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Target Hardware**} & \multicolumn{2}{c}{**Baseline Dataset**} & \multicolumn{2}{c}{**Sampled Dataset**} \\ \cline{2-5}  & **W/o Transfer Tuning** & **W/ Transfer Tuning** & **W/o Transfer Tuning** & **W/ Transfer Tuning** \\ \hline A64 FK (CPU) & 66.81 & 149.5 & 58.7 & 112.43 \\ Xeon (CPU) & 91.34 & 282.2 & 85.22 & 189.25 \\ A40 (GPU) & 627 & 416 & 599 & 175 \\ A100 (GPU) & 578 & 391 & 585 & 400 \\ H100 (GPU) & 128.12 & 67.30 & 93.42 & 54.25 \\ RTX2080 (GPU) & 18.67 & 27.68 & 17.37 & 841.74 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Inference Time Comparison (Seconds). Specifically, the root mean square error (RMSE) for our optimized tuner is 0.04 after 200 epochs, outperforming the values of 0.08 and 0.09 for XGB and LGBM, respectively. It is important to note that we are actively addressing the offline training overhead as part of our ongoing research efforts. This represents an initial phase in our research, and we are concurrently investigating potential instabilities in the tuners.