## 2 Methodologies
### Random-Delay-Corrected DRL Framework
It implements three interfaces to collect network-domain and service-domain data of NFs and control the resource utilization (e.g., CPU, memory); therefore, we can check if the Python service is alive to judge if the NF instance is available. _getNFStatus_ is used to obtain the running-time process identity (PID), CPU, memory, traffic load information, and/or static capacity of NF

Figure 2: Kubernetes-based cloud environment. instances. _getNFServiceStatus_ is used to obtain the signaling status of UE and/or running-time downlink/uplink traffic data. _getPodNetworkStatus_ is used to obtain the delay and packet loss rate information of the NF instances. On top of the basic environment, we implement the _envAdapter_ to provide unified APIs for data collection and network control. Three Python services are implemented to reach this target, including _controlPod_, _discoveryAllPod_, and _PodPoolServer_. The _controlPod_ service exposes the capabilities of horizontal or vertical scaling of NF instances (e.g., adding a new Pod instance). The _discoveryAllPod_ service is used to obtain the IP addresses of all available network function instances to support network-domain service discovery procedures. The _PodPoolServer_ service integrates network data from different domains (e.g., network-domain, resource-domain, and service-domain) together, facilitating graph-based association and then exposing them to the AIE. ### The Artificial Intelligence Engine Deploying DRL Algorithms

In order to provide AI-native functionalities for the agent to learn from the collected state of the underlying networks, we consider that there will be a scalable and effective engine running multiple and customized DRL algorithms in real-world systems. For this purpose, we develop an AIE referring to NVIDIA Cloud GPU solutions and implement a procedure for customized AI algorithms uploading, parsing, and running, which is illustrated in Figure 3. As is shown, the procedure includes \"Apps Uploading\", \"Apps Configuration\", and \"Apps Running\" in order. The model designer first uploads the application package (app.tar) that consists of the AI models/algorithms and requirements for running environment/network data. The application package will be stored in the \"apps\" database, waiting to be parsed and allocated with one or more NVIDIA Docker instances. During \"Apps Configuration\", the application package is decompressed with four files that are \"api.yaml\", \"env.yaml\", \"model.pb\", and \"procedure.py\", respectively. The \"api.yaml\" indicates the required network data for training the uploaded AI model and the network control interfaces for executing generated actions (or decisions). The AIE maintains all APIs for network data retrieval and network control and opens them all to ensure their availability for model designers. Moreover, different AI models/algorithms may be written with a different version of Python and neural networks so that the running environment requirements should be indicated specifically in \"env.yaml\". For example, one AI model/algorithm may run with Python 3.6 and Tensorflow 2.0 supporting GPU, while another one may run with Python 3.5 and Pytorch. The model designer may upload a pre-trained model optionally that is in the format of \"model.pb\". The main logic of the AI model/algorithm, including data retrieval, model training, model inference, and

Figure 3: The procedure for uploading and running customized AI models/algorithms. decision-making, is written in \"procedure.py\". With the four files, NWDAF then assigns the AI model/algorithm with one or more compliant NVIDIA Docker instances for running. Afterward, the AI model/algorithm running in NVIDIA Docker obtains the network data from the \"data\" database following the configured data APIs. The generated results can be used for network control following the configured network control APIs; therefore, the MAPE closed-loop architecture is implemented in the AIE, which facilitates the online network automation for a cloud-native 5G core network.