  Grant-free non-orthogonal multiple access (GF-NOMA) is a potential multiple
access framework for short-packet internet-of-things (IoT) networks to enhance
connectivity. However, the resource allocation problem in GF-NOMA is
challenging due to the absence of closed-loop power control. We design a
prototype of transmit power pool (PP) to provide open-loop power control. IoT
users acquire their transmit power in advance from this prototype PP solely
according to their communication distances. Firstly, a multi-agent deep
Q-network (DQN) aided GF-NOMA algorithm is proposed to determine the optimal
transmit power levels for the prototype PP. More specifically, each IoT user
acts as an agent and learns a policy by interacting with the wireless
environment that guides them to select optimal actions. Secondly, to prevent
the Q-learning model overestimation problem, double DQN based GF-NOMA algorithm
is proposed. Numerical results confirm that the double DQN based algorithm
finds out the optimal transmit power levels that form the PP. Comparing with
the conventional online learning approach, the proposed algorithm with the
prototype PP converges faster under changing environments due to limiting the
action space based on previous learning. The considered GF-NOMA system
outperforms the networks with fixed transmission power, namely all the users
have the same transmit power and the traditional GF with orthogonal multiple
access techniques, in terms of throughput.
