# Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
## 2 Materials and Methods
### Regression of Quantity and Metrics  
With the confidence that the TL metrics discussed above have a positive and significant correlation with the performance of the system when performance increases with regard to the quantity of data used during training, the goal is to now derive the relationship between those metrics and data quantity, with preference being given to the metric that has a better goodness of fit (GoF) with a form of linear regression. In this case, a log-linear regression is used between the metrics and the data quantity. Starting with the accuracy of \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Set** & \(\Omega\) & **NCE** & **LEEP** & **LogME** \\ \hline \multirow{3}{*}{\(\Phi_{3}\)} & \(\Omega_{C}\) & **0.9774** & 0.9533 & 0.8033 \\  & \(\Omega_{S}\) & **0.8249** & 0.8144 & 0.7382 \\  & \(\Omega_{A}\) & **0.9666** & 0.9639 & 0.9377 \\ \hline \multirow{3}{*}{\(\Phi_{5}\)} & \(\Omega_{C}\) & 0.9438 & **0.9443** & 0.8788 \\  & \(\Omega_{S}\) & **0.6554** & 0.6553 & 0.6334 \\  & \(\Omega_{A}\) & **0.9794** & 0.9791 & 0.9582 \\ \hline \multirow{3}{*}{\(\Phi_{10}\)} & \(\Omega_{C}\) & **0.9794** & 0.9688 & 0.9609 \\  & \(\Omega_{S}\) & 0.5165 & 0.4262 & **0.5298** \\ \cline{1-1}  & \(\Omega_{A}\) & **0.9836** & 0.9808 & 0.9764 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: [PERSON]’s \(\tau\) weighted correlation across datasets (\(\Omega\)) and waveform sets for Accuracy and (NCE, LEEP, LogME). Strong correlations will have an absolute value near 1, while no discernible correlation will be around 0. **Bold** values represent the combination of the problem set and metric with the highest correlation with accuracy on the evaluation set. each network shown in Figure 4, the log-linear fit is able to provide a quality value in terms of the accuracy achievable for a given number of observations per class (OPCs) for the three datasets. Looking at the \(\Phi_{10}\) problem set shows the quality quantification as follows:  
* \(\Omega_{C}\to 81\%\) accuracy \(|\) 1M OPC
* \(\Omega_{S}\to 20\%\) accuracy \(|\) 1M OPC
* \(\Omega_{A}\to 76\%\) accuracy \(|\) 1M OPC,  
but the quality can just as easily be defined as the OPC needed in order to achieve a given accuracy given the linear fit can be inverted as  
* \(\Omega_{C}\to 5.25\)M OPC \(|\) 90% accuracy
* \(\Omega_{S}\to\infty\) OPC \(|\) 90% accuracy
* \(\Omega_{A}\to 7.04\)M OPC \(|\) 90% accuracy. However, the log-linear regression between data quantity and accuracy has an undesired effect between the data points and the linear fit, which is that at the ends of the available data, there is increased error relative to the center of the data points. Additionally, because the sign of error is the same at both ends, this suggests that the linear fit between data quantity and accuracy when there are minimal data will severely underestimate the data quantity needed to achieve high-performance systems. For a better look at this issue, Figure 5 examines the residuals for the \(\Phi_{10}\) waveform set across the three dataset types. This same sign of error at the ends of the available data suggests that a non-log-linear fit would be more appropriate for regressing the relationship between accuracy and data quantity, which is poorly suited to understanding the full relationship as available data become more limited to a narrow subset of the full data range. For example, just looking at a narrow portion of either end, with high or low data quantity, does not provide enough context to predict a good non-linear fit. Therefore, a GoF measure that weights the outer  
Figure 3: Visualization of the relationships between the three metrics (Left column: NCE, Middle column: LEEP, Right column: LogME) and the performance metric (Accuracy) of each network when measured on the results of the evaluation set \(\Omega_{TC}\), or the _target_ dataset in TL vernacular. Each dataset used for training is positioned along the rows (Top row: \(\Omega_{C}\), Middle row: \(\Omega_{S}\), Bottom row: \(\Omega_{A}\)).