# Intelligent Caching with Graph Neural Network-Based Deep Reinforcement Learning on SDN-Based ICN
## 3 System Model
### System Architecture  
We propose an intelligent caching strategy in an SDN-based ICN (SDN-ICN) architecture, as depicted in Figure 1. The SDN-ICN architecture separates the control plane from the data plane, and the OpenFlow protocol facilitates the data transfer between them. We introduce the GNN-DDQN [14; 40] agent responsible for making caching decisions in the control plane. The data plane comprises network nodes that perform caching actions.  
Figure 1 illustrates the control plane, consisting of two modules: (i) the GNN-DDQN agent module, which plays a crucial role in caching decisions for ICN nodes in the data plane; and (ii) the content caching management module, which handles the content caching of each ICN node. We assume that the data plane's network topology consists of \(N\) ICN nodes, denoted as \(\mathcal{N}=\{n_{1},n_{2},\ldots,n_{N}\}\).  
The data plane encompasses ICN nodes, each fulfilling specific roles: (i) source nodes responsible for content publication without caching capabilities, (ii) receiver nodes accountable for sending requests to source nodes, also without caching capabilities, and (iii) router nodes responsible for forwarding requests and data packets across the network. Instead of assuming that all router nodes have the caching capability, we consider only some of them to have this. These router nodes equipped with caching capabilities have a cache capacity of \(z\), defined as the number of content items.  
The network contains \(C\) distinct content items, represented by the set \(\mathcal{C}=\{c_{1},\ldots,c_{C}\}\). We assume that an experimental time round can be divided into \(T\) slots of equal duration, denoted by \(\mathcal{T}=\{t_{0},t_{1},\ldots,t_{T}\}\). To indicate whether a node \(n_{k}\) caches content \(c_{i}\) at time step \(t_{l}\), we employ a binary variable \(\{b_{l_{l}}^{c,n_{k}}\}\), where \(t_{l}\in\mathcal{T}\), \(c_{i}\in\mathcal{C}\), and \(n_{k}\in\mathcal{N}\). Specifically, \(b_{l}^{c_{i},n_{k}}=1\) if and only if node \(n_{k}\) caches content \(c_{i}\) at time step \(t_{l}\), implying that content \(c_{i}\) is available at node \(n_{k}\) during the time interval between \(t_{l}\) and \(t_{l+1}\).  
In the SDN-ICN architecture, the controller can see the network's traffic. Therefore, at each time step \(t_{l}\), the GNN-DDQN agent observes the network state \(\mathcal{S}_{t_{l}}\), which encompasses information about the network's status during the time interval between \(t_{l-1}\) and \(t_{l}\). Subsequently, the GNN-DDQN agent performs content caching predictions at the node level, denoted as \(\mathcal{A}_{t_{l}}\), and communicates the recommended content to be cached to each router node with caching capabilities. When a router node with caching capabilities receives a request packet for content \(c_{i}\) within this period, it can fulfill the request directly if the requested content is cached or forward the request to the source node. At the subsequent time step \(t_{l+1}\), a set of rewards \(\mathcal{R}_{t_{l}}\) is sent to the GNN-DDQN agent. Specifically, \(\mathcal{R}_{t_{l}}=r_{t_{l}}^{n_{1}},r_{t_{l}}^{n_{2}},\ldots,r_{t_{l}}^{n_{N}}\) represents the rewards of all nodes at time step \(t_{l}\). Furthermore, \(r_{t_{l}}^{n_{k}}=r_{t_{l}}^{c_{1},n_{k}},r_{t_{l}}^{c_{2},n_{k}},\ldots,r_{t_{ l}}^{c_{C},n_{k}}\) represents the set of rewards for each content item received by node \(n_{k}\) at time step \(t_{l}\).  
Figure 1: The SDN-ICN architecture. In the controller, the GNN-DDQN agent receives a network state \(S_{t_{l}}\) and generates an action \(A_{t_{l}}\) at each time step \(t_{l}\). Subsequently, it receives a reward \(R_{t_{l}}\) at the next time step \(t_{l+1}\).