# Reinforcement Learning Based Resource Management for Network Slicing
## 3 System Model  
### Overall System Architecture  
In this section, we first describe the overall system architecture depicted in Figure 1. Table 1 summarizes the notation adopted in this work. Consider a network slicing infrastructure consisting of a set of network slices denoted as \(\mathbb{N}=\{1,2,\cdots,N\}\), where \(N\) is the number of slices. These multiple slices are built upon a physical network comprised of a radio access network and core network. There are two different types of resources spread across the entire physical network: virtual network function (VNF) resources and transmission resources. Network flows in a slice \(n\in\mathbb{N}\), denoted as \(f\), traverse a sequence of data centers executing a specific VNF and a number of physical transmission links and nodes before reaching the destination. The flows consume computing resources when traversing network servers, and occupy transmission resources on physical links and nodes for transmission. We treat the computing resources for network services, analytics and security in the data center as a single type of integrated VNF resources. Similarly, integrated transmission resources indicate wired and wireless bandwidth. We simply consider that such types of resources are proportional to the data rate. We assume that time takes discrete values denoted as \(t=1,2,3,\cdots\), and the achievable rate for a network slice at a time slot \(t\) is denoted as \(R_{n,t}\). In this paper, bandwidth is used as an example of resources unless otherwise stated. Figure 1: Business entities and shared resources in network slicing infrastructure. In our system, three stakeholders, namely the slice provider, slice tenant, and end-user, interact to realize the end-to-end communication service. As the slice provider and a number of slice tenants are independent business entities, they act independently to generate their own profits. Each slice tenant makes requests for the creation of new network slice instances and manages own network slice. From the business point of view, the tenants try to purchase resources for slice creation from the provider upon receiving requirements of the end-users in their slices. The provider possesses a physical resource pool and can sell resources to the tenants on demand. The total amount of the resource pool is \(R^{phy}\). The initial resource \(R^{ini}\) is first bundled into slices and then allocated to the slices. The tenants monitor the resource demand and utilization at every discrete time slot \(t\). The resource amount allocated to each slice \(n\) at time slot \(t\) through resource trading is \(R^{buy}_{n,t}\). The total resource amount allocated to all slices cannot exceed the amount of the available physical resource, (i.e., \(R_{n,t}=\sum_{n\in\mathbb{N}}R^{buy}_{n,t}\leq R^{phy}\)). Since there may exist unnecessary resources waste or resource scarcity due to demand fluctuations of end-users, the tenants should be able to trade unused resources and acquire additional resources from the provider. Therefore, each tenant should determine whether to increase or decrease the amount of resource being bought from the provider at each trading interval \(h\). However, this adjustment may cause an overhead, especially for short periods of trading. Because an excessive resource adjustment rate can be a burden on a resource controller, the operator should set the trading interval \(h\) appropriately depending on the dynamics of the demand change. Let \(p^{buy}\) and \(p^{sell}\) denote a unit buying price and a unit selling price of the bandwidth, respectively. Therefore, the tenants pay a fee of \(p^{buy}\cdot R^{buy}_{n,t}\) for the purchased resources to the provider every time slot \(t\). If the total resource demand of the slices and the amount of resource allocated to the slices