## 1 Introduction  
Mobile applications have grown explosively in recent years, leading to an increased interest in mobile communication systems [1; 2; 3]. Relay-assisted mobile cooperative communication is an inevitable trend for future mobile networks, especially with regard to reliability [4; 5; 6]. In a study by the authors of [4], a multi-antenna decode-and-forward relay was used to assist a two-user non-orthogonal multiple access system and ensure secure transmission. For trusted and untrusted scenarios, secure relaying was considered in non-orthogonal multiple access [5]. A beamforming design was investigated for cooperative secure transmission in cognitive two-way relay networks [6].  
As a promising technology, multiple-input-multiple-output (MIMO) can improve the performance of mobile cooperative communication. Massive MIMO was combined with general orthogonal precoding for high mobility scenarios in [7]. A minimum mean square error (MMSE) detector was used for channel estimation of massive MIMO systems [8]. In [9], the pairwise error probability (PEP) of the massive multiuser MMSE-MIMO systems was derived. However, the hardware complexity of MIMO also increased. Transmit antenna selection (TAS) is a practical option to reduce hardware complexity. It has also been attracting the attention of academicians [10; 11; 12]. Based on the asymptotic theory on order statistics, the authors of Ref. [13] derived the asymptotic upper capacity bounds of massive MIMO system with TAS over Rayleigh fading channels. A TAS strategy was investigated for full-duplex distributed antenna systems in [14]. Based on Monte Carlo Tree Search (MCTS), a self-supervisedlearning method was proposed to solve the antenna selection problem for a massive MIMO system [15]. TAS and extreme value theory were used to investigate the asymptotic behavior of spectrum-sharing systems in [16].  
To data, cooperative communication and TAS technologies only consider Rayleigh and Nakagami fading channels. In reality, due to the complex and variable environments, the fading channels for mobile cooperative communication are more complicated than Rayleigh and Nakagami fading channels [17; 18]. The \(N\)-Nakagami model is adopted in mobile cooperative communication [19; 20; 21]. The outage probability (OP) performance is essential for the design and evaluation of mobile cooperative communication networks over \(N\)-Nakagami channels. Therefore, predicting OP performance on time and then providing corresponding evaluation can effectively guarantee communication quality. However, performance prediction is the foremost task in the evaluation of the mobile cooperative communication networks. There is a lack of research on OP performance prediction of mobile cooperative communication networks with TAS.  
Motivated by the above discussion, we investigate OP performance prediction with variable-gain amplify-and-forward (AF) relaying and TAS. The main contributions of this paper are as follows:  
1. For the TAS scheme, we derive the novel exact closed-form mathematical OP expressions.
2. Based on the derived OP expressions, a back-propagation (BP) neural network-based OP performance prediction algorithm is proposed. We use the OP theoretical results to generate training data. We test extreme learning machine (ELM), locally weighted linear regression (LWLR), support vector machine (SVM), and BP neural network methods.
3. Through Monte-Carlo simulations, we verify the derived OP expressions. Compared with ELM, LWLR, and SVM methods, the Monte-Carlo results verify that our method can consistently achieve higher prediction results.  
This paper is organized as follows: Section 2 provides the related work. The system model is presented in Section 3. The OP performance of the TAS scheme is investigated in Section 4. Based on the BP neural network, we propose an OP performance prediction algorithm in Section 5. OP performance is evaluated in Section 6. Section 7 offers concluding remarks.