## 1 Introduction
While the actor performed the actions without the need for optimized value function, the critic criticized the actions taken by the actor and kept updating the value function [16]. DDPG adopted a one-step update framework instead of updating the strategy after the whole episode, which was more efficient than PG. However, the computational complexity of DDPG was sufficiently high, since it had to train two independent neural networks at the same time. These DRL frameworks have been used for the cross-layer routing design in intelligent networks. In [17], an intelligent routing protocol based on DQN was presented. It allowed choosing the best data transmission paths according to the best criteria and based on the network status. The author in [18] imposed the DDPG to optimize the routing in software-defined networks with the DDPG Routing Optimization Mechanism (DROM). It simplified network operation and maintenance by enhancing the performance with black-box optimization in continuous time. [17; 18] utilized the DRL approach to overcome the uncertainty of network topology, but they failed to consider energy efficiency in the network. The author in [19] developed a RL-based joint spectrum access and clustering scheme to improve idle channel sensing and reduce system energy expenditure. In [20], Prioritized Memories Deep Q-Network (PM-DQN) was developed to solve the route selection and channel access. It introduced a concept called responsibility rating to compress the huge action space and realize power adaptation for lower power consumption and calculation complexity. Nevertheless, the responsibility rating merely allowed single-level transition of the transmit power, which brought about inefficiency of power allocation. Furthermore, a single source node for generating data streams was considered in [20]. If multiple source nodes exist in the network, every node has to run the DRL for routing. In addition, all of these works adopted a self-learning scheme that required many interactions with the environment through trial and error. Since each interaction means the establishment of a communication link, this resulted in large communication overhead, extensive energy consumption and long exploration time. Some applications in CRN cannot afford the extensive latency and energy expenditure. If there exists any node that has learned a stable routing strategy, it is unnecessary for the newly joined node to learn from scratch. By using DQfD, interactions with the real environment decrease significantly with the guidance of empirical data from the expert node. This accelerates the learning process and reduces the trial-and-error cost, which is more compatible with applications of CRN. Since most SUs rely on batteries for power supply, energy constraints have become an urgent problem to be resolved [21]. Therefore, improvement of energy efficiency is significant for the practical application of CR technology. Three methods are adopted to reduce the energy consumption of the network: Firstly, dynamic adjustment rating is introduced for efficient power adaptation. Secondly, the trial-and-error cost is reduced with expert's demonstration data, which leads to lower energy expenditure. Thirdly, energy utilization ratio is included as a part of utility function in the DRL. In this paper, we apply an apprenticeship learning scheme called DQfD to the cross-layer routing design in CRN. Initially, the data streams emerge from a source node and it achieves a stable routing strategy through self-learning. After that, new data streams emerge from another source node, and it learns the routing protocol from the empirical data of the previous source node, instead of learning from scratch by itself. The source node to be learned from is defined as the expert source node, and the source node that needs guidance is called the apprentice source node. Our characteristic work can be featured by the following:  
1. As far as we know, this is the first work that applies DQfD to the large-scale CRN with a single-agent learning framework.