## 1 Introduction
In [37], the authors studied the problem of resource assigning in a single carrier communication system in the presence of a jammer, a Bayesian jamming game framework was proposed and the Nash strategy was also compared to the Stackelberg strategy to verify its sensibility. In [38], the authors studied a new anti-jamming problem of unknown nodes in a peer-to-peer network attacked by a random jammer or an intelligent one. In [39], considering the idealized case and potential energy constraint, the authors investigated the jamming attacks optimization problem that the jammer could control the probability of jamming and transmission range to cause maximal damage to wireless network. The existing works [34; 35; 36; 37; 38; 39] investigated the anti-jamming problems from the perspectives of adaptive defense strategies, potential energy constraint, jamming modes, ideal and non-ideal cases in different communication scenarios to perfect the study. Game theoretical approaches, such as stochastic game and Stackelberg game, are just methods which can help us analyze the anti-jamming problems effectively. The main difference between this paper and the above anti-jamming works [29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39] is that, in the presence of an intelligent jammer, we investigate the relay selection problem under a power-energy constraint with imperfect information. Moreover, considering the existing power optimization methods in anti-jamming domain, there are still some problems needing to be solved. Firstly, it is hard for the user and jammer to get the math utilities and channel state information. Secondly, there exists error when a user measures the communication quality or a jammer measures the jamming effect. Thirdly, the optimizations of power strategy and relay selection need to be considered simultaneously. In this paper, we investigate the joint relay selection and power control optimization problem in an anti-jamming relay communication scenario. Considering the competitive relationship between the user and jammer, we formulate the anti-jamming problem as a Stackelberg game, which is an effective tool to model the hierarchical optimization problem. In the game frame, the user acts as leader to select relay and power strategy firstly, and the jammer acts as follower then to choose power strategy. Moreover, the proof of existence of SE is given. Based on Q-learning algorithm and multi-armed bandit (MAB) method, a hierarchical joint optimization algorithm is proposed. Simulation results show the user's strategy selection probability and the jammer's regret. We give utility comparison under the proposed algorithm and random selection algorithm with feedback error and eavesdropping error. In the last, we summarize the main contributions of this paper in the following:  
* In relay communication scenario, considering the competitive relationship between the user and jammer, we formulate the anti-jamming joint optimization problem as a Stackelberg game, in which the user acts as leader and jammer acts as follower. * We prove the existence of SE and propose a hierarchical joint optimization algorithm based on Q-learning and MAB method via Stackelberg game frame. * Simulation results show the user's strategy selection probability and jammer's regret. Utility under the proposed algorithm is compared with random selection algorithm to verify the algorithm's superiority. Moreover, the influence of feedback error and eavesdropping error on utility is analyzed. The main differences from our previous work [36] are summarized as: (i) There exist multiple relays in the anti-jamming system which needs to realize the optimization of relay selection and power control simultaneously. (ii) The feedback error and eavesdropping error are introduced because it is hard for the the user and jammer to get accurate feedback information. (iii) The user and jammer do not need to know the power and channel fading information which is ignored in [36].