# Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
## 2 Materials and Methods  
The work performed in [PERSON] et al. [41] showed that, within the realm of AMC, the quantity of data has a functional relationship to the performance of a trained system given all other variables are constant. Additionally, the work showed that the performance could be found to have a log-linear, or power-law, relation to the quantity of data for lower performance regions, but a log-sigmoidal relationship is more appropriate as the performance reaches a maximum. The process of regressing the relationship between quantity and performance was then suggested as a quantification measure of dataset quality in [44], where different datasets could then be compared across different quantities with the expected accuracy (e.g., dataset \(A\) needs \(X\) observations, while dataset \(B\) needs \(2X\) observations to achieve an accuracy of 90%), or another metric of performance, taken as the quality (\(X|90\%\) or \(2X|90\%\) in the previous example) of the dataset. The inherent quality of any dataset can be described in three generalized terms: _Comprehensiveness_, _Correctness_, and _Variety_[25]. In this work, the datasets are already examined and confirmed to be _Comprehensive_, in that all the information being sought is included within the dataset, and _Correct_, in that the observations for each modulation are correctly identified and labeled. The main concept of quality being examined is then that of _Variety_ or rather that the distributions on the observations within the datasets match, approximate, or deviate from the distributions of the test set, and therefore, only the effect of quality in terms of _Variety_ can be examined in this work.  
While these works give an initial understanding of the data quantity and quality that fundamentally drive the process of an ML system, they provide minimal utility when trying to understand how many data are needed in order to achieve ideal performance and therefore reliably plan a data collection campaign. For example, in [44] fourth figure that is looking at the 10-class classification performance, the log-linear fit predicts a performance of 90% accuracy at roughly an order of magnitude less data than the corresponding log-sigmoidal fit, while both fits use the full range of trials available to regress the fit. The results discussed above all depend on some initial _good dataset_ to contrast with, and while this work does not alleviate that requirement, here, we answer the question of how to best use a limited _good dataset_ to forecast how many total data would be needed during training if neither the model nor training approach is modified.  
An ideal approach would be to use a metric that is both strongly correlated with the desired performance of the system, such as accuracy, in terms of [PERSON]'s \(\tau\) and has a relationship with data quantity that can be linearly derived from minimal data; however, a metric that reduces the error over that of performance directly regressed with quantity will be sufficient. For this reason, the metrics that have been developed to predict the transferability of a pretrained model onto a new _target_ dataset, discussed in Section 1.1.2, are repurposed to predict data quantity requirements and provide a new metric of quality for a model's training dataset with regard to the _target_ dataset, which is the evaluation dataset in this work, as shown in Figure 1c.