  In this paper, we investigate the random access problem for a
delay-constrained heterogeneous wireless network. As a first attempt to study
this new problem, we consider a network with two users who deliver
delay-constrained traffic to an access point (AP) via a common unreliable
collision wireless channel. We assume that one user (called user 1) adopts
ALOHA and we optimize the random access scheme of the other user (called user
2). The most intriguing part of this problem is that user 2 does not know the
information of user 1 but needs to maximize the system timely throughput. Such
a paradigm of collaboratively sharing spectrum is envisioned by DARPA to better
dynamically match the supply and demand in the future [1], [2]. We first
propose a Markov Decision Process (MDP) formulation to derive a modelbased
upper bound, which can quantify the performance gap of any designed schemes. We
then utilize reinforcement learning (RL) to design an R-learning-based [3]-[5]
random access scheme, called TSRA. We finally carry out extensive simulations
to show that TSRA achieves close-to-upper-bound performance and better
performance than the existing baseline DLMA [6], which is our counterpart
scheme for delay-unconstrained heterogeneous wireless network. All source code
is publicly available in https://github.com/DanzhouWu/TSRA.
