## 1 Introduction
### Contributions of the Research  
In contrast with existing studies, which quantized the continuous set into discrete space [16; 19; 20], we propose utilizing the parameterized deep Q-network (P-DQN) to handle the problem with a _hybrid_ action space composed of _discrete_ user association and _continuous_ power allocation [31]. This overcomes the difficulty of traditional DQN which can cope with RL problems having discrete action spaces, either with intrinsically discrete actions or with discrete actions quantized from continuous action space. In this work, we provide a joint solution for power allocation and user association with the objective of maximizing downlink energy efficiency under backhaul link constraint and QoS guarantee using P-DQN. A flexible reward function is devised to meet each user equipment's QoS demands in different traffic scenarios, and a penalty mechanism is introduced when the backhaul link constraint is violated. Simulation results demonstrate that the P-DQN outperforms other approaches in terms of overall energy efficiency while satisfying QoS requirements and backhaul constraints. The main contributions of this paper are summarized as follows:  
* We provide a joint solution for the power allocation and user association with the objective of maximizing downlink energy efficiency under backhaul link constraint and QoS guarantee. We employ the novel model-free parameterized deep Q-network (P-DQN) framework that is capable of updating policies in a hybrid discrete-continuous action space (i.e., discrete BS-UE association and continuous power allocation).
* To the best of our knowledge, most DRL-based research about power allocation do not consider the wireless backhaul capacity constraint and user QoS. We design the flexible reward function to meet the QoS demands at different traffic scenarios and introduce a penalty mechanism when the backhaul link constraint is violated. We verify by simulations that the proposed P-DQN framework outperforms other proposed  
\begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Ref.** & **Objective** & \begin{tabular}{l} **QoS** \\ **Constraint** \\ \end{tabular} & \begin{tabular}{l} **Backhaul** \\ **Constraint** \\ \end{tabular} & \begin{tabular}{l} **User** \\ **Association** \\ \end{tabular} & \begin{tabular}{l} **Power** \\ **Allocation** \\ \end{tabular} & **Method** \\ \hline \hline [11] & Data Rate & \(\times\) & \(\times\) & \(\times\) & Continuous & DDPG \\ \hline \hline [16] & Data Rate & \(\times\) & \(\times\) & \(\times\) & Discrete & Distributed DQN \\ \hline \hline [18] & \begin{tabular}{l} Power \\ Efficiency \\ \end{tabular} & \(\times\) & \(\times\) & \(\times\) & Discrete & DQN \\ \hline \hline [19] & Data Rate & ✓ & \(\times\) & \(\times\) & Discrete & DQN \\ \hline \hline [20] & Data Rate & \(\times\) & \(\times\) & \(\times\) & Discrete & DQN \\ \hline \hline [21] & \begin{tabular}{l} Power \\ Efficiency \\ \end{tabular} & ✓ & \(\times\) & \(\times\) & Continuous & DRL-DPT \\ \hline \hline [23] & \begin{tabular}{l} Energy \\ Efficiency \\ \end{tabular} & \(\times\) & \(\times\) & \begin{tabular}{l} Not \\ Exactly \\ \end{tabular} & Continuous & Actor-Critic \\ \hline \hline [24] & \begin{tabular}{l} Energy \\ Efficiency \\ \end{tabular} & \(\times\) & \(\times\) & ✓ & Discrete & DQN \\ \hline \hline \
\begin{tabular}{l} This \\ work \\ \end{tabular} &
\begin{tabular}{l} Energy \\ Efficiency \\ \end{tabular} & ✓ & ✓ & ✓ & Continuous & P-DQN \\ \hline \hline \end{tabular}
\end{table}
Table 1: Summary of Related Work in Power Allocation and User Association. A check mark “✓” means the issue in the column is considered in the corresponding reference, whereas a cross mark “\(\times\)” means otherwise.  
approaches in terms of overall energy efficiency while satisfying QoS requirements and backhaul constraints.