# An All-Batch Loss for Constructing Prediction Intervals
## 3 Experiments
### Model Comparisions  
In case study 1, after changing the batch size, the performance of the QD loss and the AB loss is shown in Figures 2 and 3. Compared with the PIs generated by QD loss, the PIs generated by AB loss are closer to the original curve on small batches.  
The median values of PICP, MPIW, CWC and MSE metric values of each experiment in case study 1 are shown in Table 3. In the case of small batches, the model trained with AB loss has a significant improvement in MSE. As for the CWC metric, using AB loss has an improvement of up to 20%, which is mainly because of the average situation which is considered by CWC metric. The results prove that the AB loss is more reliable than the QD loss in the case of small batches.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Parameter** & **Numerical Value** \\ \hline Common & \(\alpha\) & 0.05 \\  & \(N_{epoch}\) & 1000 \\ LUBE & \(\eta\) & 50 \\  & \(\mu\) & 0.95 \\ QD Loss & \(\lambda\) & 15 \\  & \(s\) & 160 \\  & \(\lambda\) & 0.03 \\ AB Loss & \(\beta\) & 1.2 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Parameters used in the experiments.  
Figure 3: Comparison of PI boundaries for the QD loss by changing the batch size. The upper and lower boundaries are the predicted 95% PIs. The predicted value is generated by the prediction network in the framework.  
Figure 2: Comparison of PI boundaries for the AB loss by changing the batch size. The upper and lower boundaries are the predicted 95% PIs. The predicted value is generated by the prediction network in the framework.  
Figure 4 shows the results of the MVE method and the AB loss under various types of noise. According to the results, the MVE outputs PIs close to the generated data for normal noise, but struggles with exponential noise, while the AB loss approximates both reasonably. Such results are consistent with the assumption of the variance distribution in the MVE method. They also verified that the AB loss may not be restricted by the variance distribution, which is more flexible than the MVE method.  
According to Table 4, the MPIW of the QD loss and the AB loss with GD is smaller than the LUBE method with PSO, which indicates that the QD loss and the AB loss can generate higher quality PIs. Due to the heuristic search algorithm, it is difficult to find the optimal result in a limited time when there is no restriction on the data. Therefore, the implemented LUBE method performed poorly in the experiment. According to the CWC metric values, in case study 3, 4 and 7, the performance of the QD loss is slightly better than the AB loss while in other cases, the AB loss gives a better performance. The result confirms that the AB loss can generate high-quality PIs in regression tasks.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Batch**} & \multicolumn{3}{c}{**QD Loss**} & \multicolumn{3}{c}{**AB Loss**} \\ \cline{2-9}  & **PICP (\%)** & **MPIW** & **CWC** & **MSE** & **PICP (\%)** & **MPIW** & **CWC** & **MSE** \\ \hline
16 & 96.24 & 1.21 & 625.40 & 0.29 & 95.20 & 0.94 & 503.70 & 0.14 \\
32 & 96.24 & 1.17 & 584.94 & 0.32 & 96.74 & 1.01 & 515.22 & 0.13 \\
64 & 95.49 & 0.95 & 488.79 & 0.20 & 96.99 & 1.03 & 557.50 & 0.08 \\
128 & 95.49 & 0.95 & 506.38 & 0.13 & 97.24 & 1.02 & 536.68 & 0.07 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Statistical characteristics of PICP, MPIW, CWC and MSE for the QD Loss and the AB loss with changing batch size.  
Figure 4: Comparison of PI boundaries for the MVE method and the AB loss under normal noise and exponential noise. The upper and lower boundaries are the predicted 95% PIs.