# Bayesian Optimization Based Efficient Layer Sharing for Incremental Learning
## 4 Experiment Results
### Comparison of Experimental Results for the 'Clone and Branch'  
In this section, we divided CIFAR-100 into three sets to compare the proposed algorithm with the 'clone and branch' technique. The 60 classes (T0) out of the 100 classes were used for training a base network, and the remaining 30 classes (T1) and 10 classes (T2) were used for training each incremental branch network. In Table 2, we set the threshold accuracy of 'T0-T1' to 2.33% and that of 'T0-T2' to 1.6% for making fair comparisons with 'clone and branch' using similarity score [14]. In the case of 'T0-T1', the proposed method could achieve the same accuracy result as the 'clone and branch' in only four attempts. In case of 'T0-T2', we got the same result as the _'clone and branch'_ in nine attempts and since 'T2' had 10 classes, which was much smaller than 'Base', it needed more attempts to converge to reduce the tendency for combined classification accuracy. In other words, despite the same accuracy results of both algorithms, the 'clone-and-branch' computationally took a lot of time to obtain the optimal number of sharing layers, searching all cases by increasing the layer one by one for the branch network in the empirical searching  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{3}{*}{**Task**} & \multirow{3}{*}{**Classes**} & \multicolumn{3}{c}{**With Sharing Layers in Base Network**} & \multirow{3}{*}{**w/o**} \\ \cline{3-4} \cline{6-7}  & & \multicolumn{2}{c}{**‘Clone-and-Branch’**} & & \multicolumn{2}{c}{**The Proposed Method**} & \\ \cline{3-4} \cline{6-7}  & & \multicolumn{2}{c}{**Technique**} & & & \\ \cline{3-4} \cline{6-7}  & & \multicolumn{2}{c}{**Accuracy**} & \multicolumn{2}{c}{**Time**} & \multicolumn{2}{c}{**Accuracy**} & \multicolumn{2}{c}{**The Optimal**} & \multicolumn{2}{c}{**\# of**} \\ \cline{3-4} \cline{6-7}  & & \multicolumn{2}{c}{**(hours)**} & & \multicolumn{2}{c}{**Layer**} & \multicolumn{2}{c}{**Time (hours)**} & \multicolumn{2}{c}{**Accuracy**} \\ \hline T0 & 60(base) & - & - & - & - & - & 80.90\% \\ \hline T0–T1 & 60–30 & 66.73\% (45 th layers) & 63.23 & 66.73\% & 45 & 4/3.34 & 68.96\% \\ \hline T0–T2 & 60–10 & 68.74\% (46 th layers) & 68.74\% & 46 & 9/3.88 & 70.34\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of experimental results for ‘clone and branch’.  
Figure 9: Visualization results of the optimal sharing layer configuration through BayesOpt BayesOpt as threshold accuracy degradation is 8% less than the baseline accuracy. (EMNIST: 40 classes/seven classes) (**a**) the result of the selected optimal number of sharing layers. (**b**) The L2 distance of the consecutive observed points. (**c**) The value of the calculated best-selected sample for every iteration.  
step. In Table 2, we could also find out the comparison of computational time between the 'clone and branch' technique and the proposed algorithm. We could observe that under the same accuracy results, the computational time required to find the optimal number of sharing layers in our algorithm was reduced by 8.75 times than in the 'clone and branch' techniques. ('clone and branch' technique: 62.23 h, the proposed algorithm: 7.22 h) Moreover, our proposed algorithm could adaptively find the global optimal sharing configuration with target accuracy via adjusting the threshold accuracy parameters in the proposed loss function.