# Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
## 4 Conclusions  
The problem of estimating the number of data needed in order to achieve a high-performing ML model for AMC problems is discussed within this work. Typical approaches in this type of estimate focus on reducing the loss function rather than performance metrics of interest, whereas this provides a direct linking of performance to the available dataset size required to achieve it before performing costly collections and data acquisition. While if large quantities of data are already available, \(>10^{9}\) observations per class in the problem presented in this work, a log-scale nonlinear regression between performance and data quantity can help to determine how many more data are needed, which is often not feasible for problems with only a small number of data on hand (i.e., \(<10^{5}\) observations per class) due to the potential for the asymptotic bends in performance not being visible, leading to the performance of power-law estimations to significantly underestimating the amount required. The metrics developed for TL in order to choose a model to help linearize, on a log scale, the relation between accuracy predictions and data quantity can be used by utilizing the metrics NCE, LEEP, and LogME. These metrics in turn can help bring a bound to the number of data that would be needed with a current approach and help determine whether a large-scale data collection should take place or if further refinement to the training procedure and model architecture is a better approach given the program's constraints. Given the tendency for NCE and LEEP metrics to underestimate the numbure of data needed (to a lesser extent than power-law approaches) and the tendency of LogME to overestimate the number of data, a midpoint approach is proposed, on a log scale, between NCE and LogME to offer a balanced estimate. Given that NCE and LEEP offer similar performance estimates and GoF measures on the collected dataset problems, and LogME is seen to offer a better GoF for the augmented dataset problems, and balancing the two offers a more reasonable measure under data-constrained conditions when performing an estimate.  
While this approach shows traction within the RFML problem space of AMC, additional research is still needed in order to understand if these techniques can be more widely applied to other classification problem spaces in ML in general.  
Conceptualization, W.H.C.IV and A.J.M.; methodology, W.H.C.IV; software, W.H.C.IV; validation, W.H.C.IV; formal analysis, W.H.C.IV; investigation, W.H.C.IV; resources, A.J.M.; data curation, W.H.C.IV; writing--original draft preparation, W.H.C.IV; writing--review and editing, W.H.C.IV and A.J.M.; visualization, W.H.C.IV; supervision, A.J.M.; All authors have read and agreed to the published version of the manuscript.  
This research is based upon work supported, in part, by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA). The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes, notwithstanding any copyright annotation therein.  
The datasets presented in this article are not readily available because the data are part of a corporate IRAD effort and the authors are unable to distribute.  
The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.