  In this paper, a novel framework for guaranteeing ultra-reliable millimeter
wave (mmW) communications using multiple artificial intelligence (AI)-enabled
reconfigurable intelligent surfaces (RISs) is proposed. The use of multiple
AI-powered RISs allows changing the propagation direction of the signals
transmitted from a mmW access point (AP) thereby improving coverage
particularly for non-line-of-sight (NLoS) areas. However, due to the
possibility of highly stochastic blockage over mmW links, designing an
intelligent controller to jointly optimize the mmW AP beam and RIS phase shifts
is a daunting task. In this regard, first, a parametric risk-sensitive episodic
return is proposed to maximize the expected bit rate and mitigate the risk of
mmW link blockage. Then, a closed-form approximation of the policy gradient of
the risk-sensitive episodic return is analytically derived. Next, the problem
of joint beamforming for mmW AP and phase shift control for mmW RISs is modeled
as an identical payoff stochastic game within a cooperative multi-agent
environment, in which the agents are the mmW AP and the RISs. Two centralized
and distributed controllers are proposed to control the policies of the mmW AP
and RISs. To directly find an optimal solution, the parametric functional-form
policies for these controllers are modeled using deep recurrent neural networks
(RNNs). Simulation results show that the error between policies of the optimal
and the RNN-based controllers is less than 1.5%. Moreover, the variance of the
achievable rates resulting from the deep RNN-based controllers is 60% less than
the variance of the risk-averse baseline.
