Reinforcement Learning Based Scheduling Algorithm for Optimizing Age of Information in Ultra Reliable Low Latency Networks Age of Information (AoI) measures the freshness of the information at a remote location. AoI reflects the time that is elapsed since the generation of the packet by a transmitter. In this paper, we consider a remote monitoring problem (e.g., remote factory) in which a number of sensor nodes are transmitting time sensitive measurements to a remote monitoring site. We consider minimizing a metric that maintains a tradeoff between minimizing the sum of the expected AoI of all sensors and minimizing an Ultra Reliable Low Latency Communication (URLLC) term. The URLLC term is considered to ensure that the probability the AoI of each sensor exceeds a predefined threshold is minimized. Moreover, we assume that sensors tolerate different threshold values and generate packets at different sizes. Motivated by the success of machine learning in solving large networking problems at low complexity, we develop a low complexity reinforcement learning based algorithm to solve the proposed formulation. We trained our algorithm using the stateoftheart actorcritic algorithm over a set of public bandwidth traces. Simulation results show that the proposed algorithm outperforms the considered baselines in terms of minimizing the expected AoI and the threshold violation of each sensor.

Minimizing Age of Information for RealTime Monitoring in ResourceConstrained Industrial IoT Networks This paper considers an Industrial Internet of Thing (IIoT) system with a source monitoring a dynamic process with randomly generated status updates. The status updates are sent to an designated destination in a realtime manner over an unreliable link. The source is subject to a practical constraint of limited average transmission power. Thus, the system should carefully schedule when to transmit a fresh status update or retransmit the stale one. To characterize the performance of timely status update, we adopt a recent concept, Age of Information (AoI), as the performance metric. We aim to minimize the longterm average AoI under the limited average transmission power at the source, by formulating a constrained Markov Decision Process (CMDP) problem. To address the formulated CMDP, we recast it into an unconstrained Markov Decision Process (MDP) through Lagrangian relaxation. We prove the existence of optimal stationary policy of the original CMDP, which is a randomized mixture of two deterministic stationary policies of the unconstrained MDP. We also explore the characteristics of the problem to reduce the action space of each state to significantly reduce the computation complexity. We further prove the threshold structure of the optimal deterministic policy for the unconstrained MDP. Simulation results show the proposed optimal policy achieves lower average AoI compared with random policy, especially when the system suffers from stricter resource constraint. Besides, the influence of status generation probability and transmission failure rate on optimal policy and the resultant average AoI as well as the impact of average transmission power on the minimal average AoI are unveiled.