## Appendix A  
This appendix contains the key details of the datasets, hyperparameter settings, and code implementation snippets referenced in the main text. ```
1 classQnet(nn.Module):
2 '''FFNwithlayernormalization,nodropout'''
3 def__init__(self,state_dim,hidden_dims,action_dim,dropout_rate -0.1):
4 super()__init__()
5 self.fc1=torch.nn.Linear(state_dim,hidden_dims[0])
6 self.ln1=nn.LayerNorm(hidden_dims[0])#LayerNormafterfirst hiddendlayer
7 self.fc2=torch.nn.Linear(hidden_dims[0],hidden_dims[1])
8 self.fc3=torch.nn.Linear(hidden_dims[1],action_dim)
9
10 defforward(self,x):
11x=F.relu(self.ln1(self.fc1(x)))#Dimension:hidden_dims[0]
12x=F.relu(self.fc2(x))#Dimension:hidden_dims[2]
13x=self.fc3(x)#Dimension:action_dim
14 returnx
15
15 classDON:
17\"\"\"DQNclass\"\"\"
18 defupdate(self,transition_dict):
19#transition_dictisadictofminibatchdata
20 states=torch.tensor(transition_dict['states'],dtype=torch.float ).to(self.device)
21 actions=torch.tensor(transition_dict['actions'],dtype=torch.int64).view(-1,1).to(self.device)
22 rewards=torch.tensor(transition_dict['rewards'],dtype=torch.float).view(-1,1).to(self.device)
23 next_states=torch.tensor(transition_dict['next_states'],dtype=torch.float).to(self.device)
24 dones=torch.tensor(transition_dict['dones'],dtype=torch.float).view(-1,1).to(self.device)
25#computetargetq
26q_values=self.q_net(states).gather(1,actions)
27 max_action=self.q_net(next_states).max(1)[1].view(-1,1)#Note :doubleDON,useq_nettoobtainnextaction
28 max_next_q_values=self.target_q_net(next_states).gather(1,max_action)#usetarget_q_nettoobtainqvalues
29q_targets=rewards+self.gamma*max_next_q_values
30#computetloss
31 loss=F.smooth_11_loss(q_values,q_targets)#Huberlossformorerobusttraining
32 self.optimizer.zero_grad()
33 loss.backward()
34 self.optimizer.step()
35#targetnetworkupdate
36 ifself.count%self.target_update==0:
37 self.target_q_net.load_state_dict(self.q_net.state_dict())
38 self.count+=1
```  
Listing 14: Code for DQN model of Signal-RL. ```
1 defperturb_sentence(sentence):
2 words=sentence.split()#cleandata
3 foriinrange(len(words)):
4 ifrandom.random()<p:#p:datmissinglevel
5 original_word=words[i]
6 words[i]=tokenizer.unk_token#Replacewith[UNK]
7 ifimputation_method=='insert_noise':#artificial'insert_noise': imputation
8 ifrandom.random()<q:#qisimputationnoiselevel
9 words[i]=random.choice(whole_words)#Replacewithrandomwholeword

## References

* [PERSON] and [PERSON] (2019) [PERSON]; [PERSON] _Statistical Analysis with Missing Data_, 3 rd ed. ; Wiley: Hoboken, NJ, USA, 2019. * [PERSON] et al. (2021) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] A Survey on Missing Data in Machine Learning. _J. Big Data_**2021**, \(8\), 1-37. [CrossRef] [PubMed]
* [PERSON] (2020) [PERSON] Language Models Are Few-shot Learners. _arXiv_**2020**, arXiv:2005.14165. * [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON]; [PERSON] Curiosity-driven Exploration by Self-supervised Prediction. In Proceedings of the International Conference on Machine Learning, Sydney, Australia, 6-11 August 2017; pp. 2778-2787. * [PERSON] (1976) [PERSON] Inference and Missing Data. _Biometrika_**1976**, _63_, 581-592. [CrossRef]
* [PERSON] (2006) [PERSON]; [PERSON] _Pattern Recognition and Machine Learning_; Springer: New York, NY, USA, 2006. * [PERSON] et al. (2014) [PERSON]; [PERSON]; [PERSON] Explaining and Harnessing Adversarial Examples. _arXiv_**2014**, arXiv:1412.6572. * [PERSON] and [PERSON] (2000) [PERSON]; [PERSON], [PERSON] _Mathematical Methods and Algorithms for Signal Processing_; Prentice Hall: Upper Saddle River, NJ, USA, 2000; ISBN 0-201-36186-8. * [PERSON] et al. (2022) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]-G. Learning from Noisy Labels with Deep Neural Networks: A Survey. _IEEE Trans. Neural Netw. Learn. Syst._**2022**, _34_, 8135-8153. [CrossRef] [PubMed]
* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] SpanBERT: Improving Pre-training by Representing and Predicting Spans. _Trans. Assoc. Comput. Linguist._**2020**, \(8\), 64-77.