## 6 Results and Discussion
### Transfer Learning Actor-Critic Solution for Energy-Efficient Data Protection Scheme  
This section verifies the performance of the proposed actor-critic framework in comparison to the myopic solution and the POMDP-based solution from earlier work. With regard to the myopic scheme, if the PU is found absent from the channel, the SU will sacrifice its energy to maximize data security [41]. Previous work proposed an optimal decision policy for a CRN to maximize the security level based on a POMDP framework, which requires complex numerical computations as well as prior information about the arrival of harvested energy [20]. The complexity of the problem depends on the required amount of the computation _space_ (e.g., the sizes of the input states, actions, transition probabilities, and observations). In a POMDP, an agent controls the process by choosing the action at each time  
Figure 11: Probabilities of detection and false alarm according to average SNRs when the number of sensing samples for each SU changes. Figure 12: Sensing error according to average SNRs when the number of sensing samples for each SU, \(N\), changes. step based on the observation history to maximize the expected long-term reward. The optimal policy for the agent to choose an action can be found by solving the Bellman's equation using value iteration-based dynamic programming. Each iteration requires \(O(|\mathbb{A}||\mathbb{S}|^{2})\) operations to compute all the probabilities of transitioning from one state, \(s\in\mathbb{S}\), to another state, \(s^{\prime}\in\mathbb{S}\), after taking an action, \(a\in\mathbb{A}\). The actor-critic method, on the other hand, does not require the agent to compute all the occurrence probabilities to find the solution in advance. In addition to that, the agent learns the optimal policy from actual experienced transitions by directly interacting with the stochastic environment. The basic simulation parameters for this exercise are shown in Table 3. For analytic convenience, we fixed the SNR value of the sensed channel at \(-10\) dB, and thus the probabilities of detection and false alarm are approximated as \(P_{d}\approx 0.9\) and \(P_{f}\approx 0.1\), respectively (based on the results of the proposed CBCSS method). We assume that the SU transmits a packet of 16-byte data in every time slot, which is equivalent to the minimum encryption block length in the AES cryptography; and the transmission channel gain is unchanged during a time slot. It is worth noting that one packet of energy is equivalent to 25 \(\upmu\)J, and each simulation was run over a thousand of time slots for several iterations to obtain average values. We first examined the convergence speed of the TLAC algorithm during the training process by calculating the average reward received after every 1000 time slots. The average harvested energy was fixed at \(e_{h,avg}=4\) packets. As can be seen from Figure 13, there is a significant rise in the convergence rate of the algorithm during the first 10,000 time slots of the training process; after that, the reward keeps increasing, but at a slower speed. Finally, the algorithm converges to an optimal policy for the SU to determine operation mode after 20,000 time slots when the reward is about 0.91.