# Generative Adversarial Networks for Zero-Shot Remote Sensing Scene Classification
## 2 Methods
### Feature Generation Module  
Since there are no samples of unseen categories in the training data, it is difficult to train a classifier for unseen categories. Our method synthesizes unseen classes' image visual features through class semantic information and noise vectors. We trained a classifier that can classify unseen class images based on the synthesized image features and their corresponding unseen class labels. Our approach was inspired by [14]. CSPWGAN is composed of a generative adversarial network. The generative adversarial network model converts the zero-shot RSSC problem into a traditional classification problem. We used the conditional generative adversarial net [32] as our baseline model. The network is composed of a generative model \(G\) and a discriminative model \(D\) that compete in a two-player minimax game. \(G\) generates the visual image representation \(\hat{x}\) by its semantic feature \(e(y)\) and a noise vector sampled from a normal distribution \(z\sim N(0,I)\). The generation process can be represented as: \(y\times z\to x\). We used the semantic feature \(e(y)\) and a noise vector \(z\) as inputs of \(G\), and \(\hat{x}\) of the class label \(y\) as outputs of \(G\). \(\theta_{G}\) denotes the parameters of \(G\). The discriminative model \(D\) can be represented as: \(e\times x\rightarrow[0,1]\), \(\theta_{D}\) denotes the parameters of \(D\). \(D\) aims to accurately distinguish the real image visual feature \(x\) from the generated visual features \(\hat{x}\). \(\theta_{D}\) denotes the parameters of the discriminative model \(D\). \(G\) aims to cheat the discriminator \(D\) by generating images that can be mistaken for the real ones. This is the training process for estimating the parameters of \(\theta_{D}\) and \(\theta_{G}\). Our model only uses the seen class data for training, but it can also generate image features of unseen classes. Generating models are usually hard to train and not easy to stabilize. We used a stable training methods called WGAN [28], the loss function of WGAN is as follows:  
\[L_{WGAN}=E[D(x,e(y)|\theta_{D})]-E[D(\hat{x},e(y)|\theta_{D})]-\lambda E[( \lVert\bigtriangledown_{\hat{x}}D(\hat{x},e(y))\rVert_{2}-1)^{2}] \tag{1}\]  
where \(E[. ]\) denotes the expectation function. \(\hat{x}=G(z,e(y)|\theta_{G})\), \(\hat{x}=\beta x+(1-\beta)\hat{x}\) with \(\beta\sim U(0,1)\). Both \(G\) and \(D\) are multilayer networks. \(\lambda\) represents the penalty factor. We set \(\lambda=10\) in this study. To achieve good optimization, we applied classification loss, semantic regression module and class-prototype loss to train the network. **Classification Loss. ** Because of the complexity of remote sensing images, the WGAN model cannot ensure that the generated samples are discriminative enough. Generating inaccurate samples results in the bad performance of the classifier. To alleviate this issue, we added a classifier to identify whether the generated image features can be classified into the correct class. By doing this, the generated features are more discriminative. We added a simple classifier that uses the negative log-likelihood:  
\[L_{C}=-E_{\hat{x}\sim p_{\hat{x}}}[logP(y|\hat{x};\theta)] \tag{2}\]  
where \(\hat{x}=G(z,e(y)|\theta_{G})\), the class label of \(\hat{x}\) is \(y\), and \(P(y|\hat{x};\theta)\) represents the probability that the class label of \(\hat{x}\) predicted by the classifier is its true label \(y\). \(\theta\) is pre-trained on the seen classes dataset. It can be represented as:  
\[P(y|\hat{x};\theta)=\frac{\exp((\theta)^{T}\hat{x})}{\sum_{c\in y}\exp(( \theta)^{T}\hat{x})} \tag{3}\]  
**Semantic Regression Module. ** We used class semantic features to guide the generator to generate image features for the corresponding class. The generator is trained using seen class semantic features and visual features in the training phase, and using unseen class semantic features synthesizes unseen visual features in the testing phase. Finally, we converted the zero-shot classification problem into a traditional classification problem.