# Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
###### Abstract  
The data used during training in any given application space are directly tied to the performance of the system once deployed. While there are many other factors that are attributed to producing high-performance models based on the Neural Scaling Law within Machine Learning, there is no doubt that the data used to train a system provide the foundation from which to build. One of the underlying heuristics used within the Machine Learning space is that having more data leads to better models, but there is no easy answer to the question, \"How much data is needed to achieve the desired level of performance?\" This work examines a modulation classification problem in the Radio Frequency domain space, attempting to answer the question of how many training data are required to achieve a desired level of performance, but the procedure readily applies to classification problems across modalities. The ultimate goal is to determine an approach that requires the lowest amount of data collection to better inform a more thorough collection effort to achieve the desired performance metric. By focusing on forecasting the performance of the model rather than the loss value, this approach allows for a greater intuitive understanding of data volume requirements. While this approach will require an initial dataset, the goal is to allow for the initial data collection to be orders of magnitude smaller than what is required for delivering a system that achieves the desired performance. An additional benefit of the techniques presented here is that the quality of different datasets can be numerically evaluated and tied together with the quantity of data, and ultimately, the performance of the architecture in the problem domain. data analysis; data collection; Machine Learning; neural networks; pattern recognition; physical layer; RF signals; RFML; signal synthesis; software radio; wireless communication +
Footnote †: journal: _Telcom_  
that fall in the domain space of Radio Frequency Machine Learning (RFML). To better clarify this, RFML is a subset of ML that overlaps communications, radar, or any other application space that utilizes the RF spectrum in a statistically repeatable manner, where ML algorithms are applied as intelligently close to the digitized samples of the RF spectrum, or the physical layer in the Open Systems Interconnection (OSI) model, as possible [8]. For simplicity within this work, focus will be given to the field of Deep Learning (DL) as the particular subset within ML due to the well-suited nature of DL systems at extracting inference from raw data [9]. Two fundamental questions for developing a DL architecture for a given application are  
* What data should be or are available to be used in order to train the system;
* How many data are needed for the current approach to achieve the desired performance level? These two questions are systematically addressed within this work, providing a blueprint for how they can be answered in general. The two most common problems in regard to datasets with DL systems are having a large enough quantity of data, and having those data be of a high enough quality, in order to develop a well-generalized model [9]. In fact, answering the question of how many data are needed is a fundamental unknown that relies on the developers' experience and insight into the problem space. The end results of attempts at answering this question are the common rule of thumb answers of \"ten times the feature space\" or \"as many data as are available\", which is an issue across all domains [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]. The desire to understand how the final performance of a system is linked with the available training dataset size is in the domain of research looking at Neural Scaling Laws (NSLs) [20, 21, 22, 23, 24]. The NSLs tie the final loss of a network to the number of parameters in the architecture, training dataset size, and available computing budget [21].