  In this paper, we propose a model-free deep reinforcement learning (DRL)-
based proactive failure recovery (PFR) framework called zero-touch PFR (ZT-PFR)
for the embedded stateful virtual network functions (VNFs) in network function
virtualization (NFV) enabled networks. To realize the ZT-PFR concept,
sequential decision-making based on network status is necessary. To this end,
we formulate an optimization problem for efficient resource usage by minimizing
the defined network cost function including resource cost and wrong decision
penalty. Inspired by ETSI and ITU, we propose a novel impending failure model
where each VNF state transition follows a Markov process. As a solution, we
propose state-of-the-art DRL-based methods such as soft actor-critic and
proximal policy optimization. Moreover, to keep network state monitoring
information at an acceptable level of freshness in order to make appropriate
decisions, we apply the concept of the age of information (AoI) to strike a
balance between the event and scheduling-based monitoring. Several simulation
scenarios are considered to show the effectiveness of our algorithm and provide
a fair comparison with baselines. Several key systems and DRL algorithm design
insights for PFR are drawn from our analysis and simulation results. For
example we use a hybrid neural network, consisting of long short time memory
(LSTM) layers in the DRL agent structure, to capture impending failure time
dependency.
