# Signal Separation Based on Knowledge Representation
## 2 Theory
### Knowledge Representation
#### 2.2.3 Data-Driven Knowledge Representation  
In the realm of handwritten digit recognition, abundant resources are available, notably exemplified by the MNIST dataset [[http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)]([http://yann.lecun.com/exdb/mnist/](http://yann.lecun.com/exdb/mnist/)) (accessed on 11 March 2024), which comprises a vast array of handwritten numeral instances. Mixed digit handwriting can be the result of overlaying multiple different digit handwritings. It can be represented as  
\[\mathbf{X}=\sum_{i=1}^{N}\mathbf{S}_{n} \tag{9}\]  
where \(\mathbf{X}\) represents the observed mixed digit handwriting, and \(\mathbf{S}_{i}\) denotes the \(i\)-th source signal, which corresponds to a single handwritten digit component. The number of source signals is denoted by \(N\).  
When confronted with a scenario involving the amalgamation of disparate digits, such as the fusion of '\(3\)' and '\(8\)' in a single image, a pragmatic strategy involves sourcing extensive sets of handwritten samples that bear semblance to the respective shapes of '\(3\)' and '\(8\)'. It is pivotal to note that similarity in shape features suffices, obviating the necessity for exact matches. For instance, when delineating the numeral '\(1\)', a judicious selection encompasses samples from both the '\(1\)' class and data samples with shapes similar to a dash, owing to their perceptible similarities in shape. Here, a cosine similarity function \(L_{\cos}(\mathbf{\tilde{S}}_{i},\mathbf{\tilde{S}}_{j})\) is defined to measure the structural similarity between two signals.  
\[L_{\cos}(\mathbf{\tilde{S}}_{i},\mathbf{\tilde{S}}_{j})=1-\left|\frac{ \mathbf{\tilde{S}}_{i}\cdot\mathbf{\tilde{S}}_{j}}{\|\mathbf{\tilde{S}}_{i} \|_{F}\|\mathbf{\tilde{S}}_{j}\|_{F}}\right| \tag{10}\]  
where \(\|\cdot\|_{F}\) is the Frobenius norm. Let \(\mathbf{D}\) represent the entire dataset, which consists of samples divided into \(L\) classes. Thus, \(\mathbf{D}\) can be expressed as \(\mathbf{D}=\{\mathbf{D}_{1},\mathbf{D}_{2},\ldots,\mathbf{D}_{L}\}\), where each \(\mathbf{D}_{t}\) represents a subset of samples for the \(t\)-th class (\(1\leq t\leq L\)) and \(L\geq N\). The knowledge representation of the separated source signals, denoted as \(\mathbf{K}_{i}\subseteq\mathbf{D}_{t},1\leq i\leq N\), exhibits similarity with the separated target features and possesses distinguishable characteristics from each other. Thus, the representation of knowledge can be aptly formulated as follows:  
\[\begin{split}&\mathbf{K}=\left\{\mathbf{K}_{i}\subseteq\mathbf{D}_{t} \ \middle|\ L_{\cos}(\mathbf{K}_{i},\mathbf{S}_{i})<\epsilon,\quad L_{\cos}( \mathbf{K}_{i},\mathbf{K}_{j})>\delta,\forall j\
eq i\right\},\\ &\text{for }1\leq i\leq N,\quad 1\leq t\leq L.\end{split} \tag{11}\]  
where \(\epsilon_{i}\) can be considered as an empirical parameter given based on the observed signals. It represents a threshold, and if \(\epsilon_{i}\) is smaller, it indicates a stronger correlation between the target source signal and the knowledge representation result. Similarly, \(\delta\) is also an empirical parameter. The larger it is, the greater the characteristic difference between the knowledge representation results of different target signals.