# Open-Set Signal Recognition Based on Transformer and Wasserstein Distance
## 3 Preliminaries  
Before introducing our proposed open-set signal recognition algorithm, some preliminaries are discussed as follows, including the settings of the fundamental model of the OSSR problem and the popular benchmark algorithm APRL [26]. In this paper, we denote the training and testing datasets as \(\mathcal{D}_{train}\) and \(\mathcal{D}_{test}\), respectively. The training dataset is expressed as:  
\[\mathcal{D}_{train}:=\{(\mathbf{x}_{1},\mathbf{y}_{1}),\cdots,(\mathbf{x}_{i },\mathbf{y}_{i}),\cdots,(\mathbf{x}_{n},\mathbf{y}_{n})\},\]With \(n\) as the number of training samples, \(\mathbf{x}_{i}\in\mathbb{R}^{d}\) and \(\mathbf{y}_{i}\) are the features and labels of the \(i\)-th sample, respectively. We assume that there are \(\ell\) known classes in the training dataset labeled as \(\mathbf{y}_{i}\in\{1,\cdots,\ell\}\). On the other hand, the testing dataset \(\mathcal{D}_{test}\) contains \(u\) samples and it can be expressed as:  
\[\mathcal{D}_{test}:=\left\{\hat{\mathbf{x}}_{1},\cdots,\hat{\mathbf{x}}_{j}, \cdots,\hat{\mathbf{x}}_{u}\right\},\]  
where the label space of the testing procedure is \(\{1,\cdots,\ell\}\cup\{\ell+1,\cdots,\ell+\ell_{u}\}\), i.e., there are \(\ell_{u}\) unknown classes. Following the work in [26], the deep embedding space of category \(k\) is denoted by \(\mathcal{S}_{k}\) and its corresponding open space is denoted by \(\mathcal{O}_{k}\). In order to formalize and manage the risk of the open space effectively, \(\mathcal{O}_{k}\) is separated into two subspaces, namely the positive open space from other known classes \(\mathcal{O}_{k}^{positive}\) and the remaining infinite unknown space as the negative open space \(\mathcal{O}_{k}^{negative}\) with  
\[\mathcal{O}_{k}=\mathcal{O}_{k}^{positive}\cup\mathcal{O}_{k}^{negative}.\]  
We can define \(\mathcal{D}_{train}^{k}\in\mathcal{S}_{k}\) from category \(k\) as the positive training data samples, \(\mathcal{D}_{train}^{\
eq k}\in\mathcal{O}_{k}^{positive}\) from other known classes except category \(k\) as the negative training data samples, and \(\mathcal{D}_{\mathcal{U}}\in\mathcal{O}_{k}^{negative}\) from \(\mathbb{R}^{d}\) as the potential unknown data samples. The function \(h:\mathbb{R}^{d}\rightarrow\mathbb{N}\) denotes the prediction function from embedding \(x\) to label \(y\). The overall open-set classification problem can be formulated as:  
\[\min_{f}\ \underbrace{\sum_{k=1}^{\ell}\mathcal{L}_{known}\Big{(}f,\mathcal{S}_{ k}\cup\mathcal{O}_{k}^{positive}\Big{)}}_{\mathcal{L}_{known}(f,\mathcal{D}_{ train})}+\alpha\underbrace{\sum_{k=1}^{\ell}\mathcal{L}_{o}\Big{(}f, \mathcal{O}_{k}^{negative}\Big{)}}_{\mathcal{L}_{o}(f,\mathcal{D}_{\mathcal{U} })}, \tag{1}\]  
where \(\mathcal{L}_{known}\) denotes the empirical classification risk on the known data, and \(\mathcal{L}_{o}\) denotes the open space risk considered in the measurement of the uncertainty of labeling the unknown samples under the known or unknown class. To emphasise, the objective function can be considered a combination of known classes and unknown classes, while the model needed to solve Problem (1) can minimize the combination of the empirical classification risk on the labeled known data and the open space risk on the potential unknown data simultaneously, over the space of the allowable recognition functions. This makes the embedding function more distinguishable between the known and unknown spaces. The adversarial reciprocal point learning algorithm (ARPL) [26] is one of the typical methods for open set recognition, which also follows the scheme to minimize the overlapping of the known and unknown distributions without any loss in the known classification accuracy. In this paper, our proposed algorithm follows the basic structure of ARPL, while a series of improvements were introduced, not only to increase its recognition efficiency but also to improve its adaptability to signal data.