## 1 Introduction
Since the satellite is powered by solar panels, its power consumption cannot be neglected. Even though we could adopt RSMA to increase the capacity of ISTN, it is also crucial to complete the task with the lowest power possible. As for the energy efficiency optimization for RSMA, some of the literature proposes SCA-based algorithms for this problem [15; 16]. However, these approaches are either too complex or only approximate to optimal solutions. Deep reinforcement learning, on the other hand, is a model-free method that can find the optimal solution with no need of prior knowledge. Additionally, the ability to learn is considered a fundamental requirement in the next-generation communication systems. Naturally, DRL is a hopeful candidate solution. Using DRL for energy efficiency optimization in communications and other systems has been researched over the years. These studies fall into two categories: one with \"discrete DRL\" and one with \"continuous DRL\". The former could only deal with discrete action/state space problems, e.g., deep Q-network (DQN) and state action reward state action (SARSA). However, most of the time, energy efficiency optimization is a continuous problem. Therefore, to apply \"discrete DRL\" to it, the action/state space should be discretized first. For example, Ref. [17] proposes to combine centralized DQN with multi-agent DQN in a 5G cognitive heterogeneous network, Ref. [18] utilizes a SARSA-based algorithm for ultra-dense networks, a multi-armed bandit RL is used for the energy-efficient trajectory design of unmanned aerial vehicles (UAVs). The second kind of RL could tackle continuous action/state space tasks, like deep deterministic policy gradient (DDPG) and proximal policy optimization (PPO). So there is no discretization requirement in these schemes. Ref. [19] applies DDPG to jointly optimize computing and caching; however, the constraints in it are simplified and not shown in the algorithm design. To allocate resources for 5G radio access network (RAN), asynchronous advantage actor-critic (A3C) is exploited. To satisfy the constraints, Ref. [20] uses many modules, which increases the complexity. Besides DRL, other approaches--model-based or model-free--are also widely used for energy efficiency optimization in various systems. In [21], the authors adopt supervised machine learning to forecast traffic load and evaluate energy efficiency in Beyond 5G (B5G) networks. In wireless sensor networks (WSNs), Ref. [22] proposes a reliable clustering and routing algorithm considering different factors to save energy. The authors of [23] designed a cluster-based hierarchical routing protocol forming minimum spanning trees among communicating nodes, reducing energy wastage and prolonging the network lifetime in WSNs. Energy efficiency optimization problems in communication networks are always limited by some conditions. Therefore, DRL, which could meet the constraints and tackle continuous tasks, is needed. Fortunately, constrained DRL (CDRL) has been proposed for this kind of problems. Generally, there are three main classes of CDRL:  
1. Primal-dual approach incorporates Lagrangian relaxation to transform the original constrained problem to an unconstrained dual problem [24; 25];
2. Policy optimization approach directly uses a more trackable function to replace original objective or constraint [26; 27];
3. Penalty function approach designs an unconstrained problem by adding penalty in the objective [28]. CDRL has been applied for network slicing recently. In [29], interior-point policy optimization (IPO) is adopted while [30; 31] utilize the soft actor-critic (SAC) method. Another reference [32] gives an example of optimal power flow using constrained DDPG. Nonetheless, there is no prior research on the energy efficiency optimization of RSMA adopting CDRL. So we propose to use SAC with a Lagrangian relaxation technique to improve the energy efficiency with the QoS constraint in ISTN.