# Training from Zero: Forecasting of Radio Frequency Machine Learning Data Quantity
### Machine Learning Concepts
#### 1.1.2 Transfer Learning  
TL is the practice of training a model on one dataset/domain (i.e., _source_), or otherwise taking a pretrained model, and training with a new dataset/domain (i.e., _target_) instead of starting from a random initialization [28]. Depending on assumptions between the _source_ and _target_, the TL application can be categorized as _homogeneous_, where differences exist in the distributions between _source_ and _target_, or _heterogeneous_, where the differences are in the feature space of the problem [28, 29]. A valuable discussion for understanding the concepts of _homogeneous_ and _heterogeneous_ within the RFML domain is provided by [PERSON] and [PERSON] [30]. By explaining the different datasets in use, this works as a change in the dataset's collected/generated domain. The feature space of the problem can be associated with the intended task the _source_ model is trained on and can be contrasted with the task of the _target_ problem. The two most common types of TL include retraining the classification head, where early layers are frozen during training, preserving feature extraction, or fine-tuning of the whole model [31]. In this work, TL is applied in a _homogeneous_ problem space where the underlying distributions of the data vary, but the generalized problem space is the same between datasets. The subset of TL is notated as _Domain Adaptation_ in [30], and more specifically, an _Environment Platform Co-Adaptation_. Additionally, wherever the retraining is carried out in this work, the fine-tuning approach is utilized, allowing for adjustments to the feature space, which might not be observable in the _source_ dataset. An important note here is that this work does not evaluate any aspect of TL on the problem  
Figure 2: DL NN image representing the CLDNN architecture used in this work as the DL approach for the 10-class waveform AMC problem space. space (i.e., the evaluation dataset \(\mathcal{X}\) does not change) but rather makes use of metrics developed for the purpose of TL. Understanding how TL is best used within RFML is beyond the scope of this work. The study of TL is complex and well explored [28, 29, 30, 31, 32, 33], and from the effort to understand how to choose an optimal pretrained model for a desired application, the metrics Negative Conditional Entropy (NCE) [32], Log Expected Empirical Prediction (LEEP) [31], and Logarithm of Maximum Evidence (LogME) [33] are repurposed to analyze the relationship between available data quantity during training and system performance for a given evaluation set. In this paper, the evaluation set has the same labels as the training set, but the distributions are not assumed to be equivalent. Therefore, the evaluation set, \(\{X\}_{1}^{N}\sim\mathcal{X}\), is drawn from an observation space, \(\mathcal{V}\), which is inherent to the generalized problem space, \(\mathcal{V}\subset\mathcal{S}\), visualized in Figure 1a,c. For clarity going forward, due to the shared labels between _source_ and _target_ in this work, the _source_ labels are found through a forward pass of the evaluation set through the network; therefore, the \(i^{\text{th}}\) observation's _source_ label is given by \(I_{y|xi}=\phi(x_{i})\in\mathbb{R}^{C}\), with the inference given as \(\hat{c}_{y|xi}=\operatorname*{argmax}(I_{y|xi})\in\mathbb{Z}^{1}\). The process of extracting the evaluation inference of a trained model is visualized in Figure 1e. By contrast, the _target_ label directly gives \(c_{x,i}\in\mathbb{Z}^{1}\) by the truth of the \(i^{\text{th}}\) observation and can be one-hot encoded to provide \(l_{xi}=\operatorname*{OH}(c_{xi},C)\in\mathbb{R}^{C}\). Given the above notation, NCE is given as  
\[ \begin{split}\text{NCE}&(\hat{c}_{y|x},c_{x})=\sum \limits_{j=1}^{C}\hat{P}(\hat{c}_{y|x}=j)\\ &\cdot\sum\limits_{k=1}^{C}\hat{P}(c_{x}=k|\hat{c}_{y|x}=j)\log( \hat{P}(c_{x}=k|\hat{c}_{y|x}=j)),\end{split}