# Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
## 2 Materials and Methods
### Regression of Quantity and Metrics
errors more significantly than the errors toward the center of the data range is desired. Additionally, since both edges of the residual are of equal significance and the results are non-uniformly sampled across the observation space, a weighting that balances the weights into histogram bins will be used to normalize equal significance in the edges of the GoF measure. For simplicity, three bins will be used to indicate observations with lower, mid, and high data quantity relative to the log-linear fit. The weights are suggested as  
\[w^{\prime}(q_{x}[i]) = \begin{cases}|q_{x}|/|b_{low}|&q_{x}[i]\in b_{low}\\ |q_{x}|/|b_{mid}|&q_{x}[i]\in b_{mid}\\ |q_{x}|/|b_{hi}|&q_{x}[i]\in b_{hi}\end{cases}  \tag{12}\] \[w(q_{x}[i]) =\frac{w^{\prime}(q_{x}[i])}{\sum_{j=1}^{|q_{x}|}w^{\prime}(q_{x} [j])}\]  
where the middle bin has one-third the weight of the edges, which without it, would have all three regions equally weighted. The division edges between bins are taken as evenly spaced on the log scale between the minimum and maximum data quantities in the set, with \(|q_{x}|\) being the number of elements in the set, while \(\{|b_{low}|,|b_{mid}|,|b_{init}|\}\) are the numbers of observations within that bin. Those weights are then normalized such that their sum is unity. The GoF is then taken as the Normalized Root Weighted Mean Squared Error (NRWMSE). \[\text{gof}(\alpha_{x},q_{x},\hat{f}_{ll})=\sqrt{\frac{\sum_{i=1}^{|q_{x}[i]}w( q_{x}[i])\cdot(\hat{f}_{ll}(q_{x}[i])-\alpha_{x}[i])^{2}}{\text{Var}(\alpha_{x})}} \tag{13}\]  
where the quantities (\(q_{x}\)) and accuracies (\(\alpha_{x}\)) are used to derive the log-linear fit (\(\hat{f}_{ll}\)); however, the accuracies and fit can be swapped out for any other metric and matching fit. The GoF for the accuracy, NCE, LEEP, and LogME metrics are given in Table 4. A general conclusion is that all three metrics have the potential to provide a better prediction  
Figure 4: Relationship between the quantity of data used from each dataset (Top: \(\Omega_{C}\), Middle: \(\Omega_{S}\), Bottom: \(\Omega_{A}\)) and the accuracy achieved by networks trained on that amount of data. of data quantity needed to achieve high performance; however, considering the correlation presented in Table 3 in addition to these results suggests that NCE will be the most consistent estimate, with LEEP being a close second. LogME, by comparison, offers the most promise with regard to the augmented dataset but has the highest variability among the three metrics examined here. One more unique attribute about the linear regressions of the metrics is that accuracy, NCE, and LEEP all have residuals typically indicating that the true quantity of data that are needed will be underestimated, while LogME's residuals are inverted, suggesting that that LogME's regression will overestimate the number of data, giving soft bounds of the required quantity of data being between the estimates of NCE and LogME predictions. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \(\mathbf{\Phi}\) & \(\mathbf{\Omega}\) & \(\mathbf{\kappa}\) & **NCE** & **LEEP** & **LogME** \\ \hline \multirow{3}{*}{\(\Phi_{3}\)} & \(\Omega_{C}\) & 0.2478 & 0.2054 & **0.1885** & 0.2662 \\  & \(\Omega_{S}\) & 1.0196 & **0.9515** & 0.9521 & 0.9531 \\  & \(\Omega_{A}\) & 0.3120 & 0.2674 & 0.2636 & **0.1672** \\ \hline \multirow{3}{*}{\(\Phi_{5}\)} & \(\Omega_{C}\) & 0.2499 & 0.1552 & **0.1458** & 0.1987 \\  & \(\Omega_{S}\) & 0.9491 & **0.9367** & 0.9451 & 0.9652 \\  & \(\Omega_{A}\) & 0.3016 & 0.2163 & 0.2208 & **0.1433** \\ \hline \multirow{3}{*}{\(\Phi_{10}\)} & \(\Omega_{C}\) & 0.1514 & **0.1138** & 0.1173 & 0.1179 \\  & \(\Omega_{S}\) & 0.9853 & 0.9783 & 0.9731 & **0.9697** \\ \cline{1-1}  & \(\Omega_{A}\) & 0.2706 & 0.2652 & 0.2797 & **0.1102** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Goodness of fit (GoF) for a log-linear regression between dataset quantity available for training across datasets (\(\Omega\)) and waveform sets (\(\Phi\)) for Accuracy (\(\alpha\)) and NCE, LEEP, LogME. Perfect fit would have a value of \(0\).