# Interest Forwarding in Named Data Networking Using Reinforcement Learning
## 1 Introduction  
Information-centric networking (ICN) has become one of the leading topics in the next-generation networking and future Internet architecture research. The fundamental concept in ICN is to make content directly addressable, a significant departure from the host-centric communication model of the current Internet. To achieve this, ICN names and addresses data units at the network level, so that in the place of a host address, request packets are routed and satisfied based on the name or identification of the desired content. This approach is motivated by the increasing domination of Internet traffic by multimedia, effectively making the Internet a content distribution network [1]. Despite the success of the current solutions for efficient information delivery like the content delivery networks (CDNs) and peer-to-peer (P2P) networking, they are limited and complicated by the underlying host-to-host communication [2]. By treating data as fundamental to the architecture, ICN seeks to match the current demands being placed on the Internet and improve on these patch solutions. The separation of content from location enables ICN to support naturally desirable features like in-network caching and multicast communication. In addition to the efficient delivery of content, ICN, using data-centric methods, also addresses significant issues including mobility, security, and quality of service. Various projects explore the ICN concept, differing mainly in their formats for naming data, caching policies, routing of data objects and requests, and in how they handle mobility and security [3; 4]. Of particular interest to us in this work, is the Named Data Networking architecture [5; 6] which incorporates a stateful and intelligent forwarding plane in addition to a routing layer. The forwarding plane, also called the _strategy module_, is expected to leverage on both the information provided by the routing layer and the observed packet delivery measurements to make adaptive decisions when forwarding requests. In the general case, the strategy module is defined for each reachable name announced by the routing layer. Furthermore, such an intelligent forwarding plane could relax the stringent convergence and correctness demands commonly placed on the routing layer as explored by the work in [7]. For scalability and practicality reasons, the name-based routing protocol in NDN can only announce and monitor paths leading to the actual publishers and designated repositories, thus leaving the responsibility of exploiting the in-network caching capability, which is vital for delivering the design goals of NDN, entirely to the forwarding strategy. However, most of the proposals for the strategy layer, including the default algorithm adopted by the NDN project, follow a \"monitor-the-routing-paths\" approach, whereby their forwarding and probing actions are restricted to the interfaces suggested by the routing layer cite [7; 8; 9; 10; 11]. This approach limits the forwarding strategy from exploiting local caches closer but outside the routing paths. In this paper, we develop a dynamic self-aware [12] strategy layer for NDN architectures to offer fast content delivery using local content stores, and we also keep the existing capabilities of the routing layer. The NDN forwarding strategy that we implement exploits the Random Neural Network (RNN) [13] with Reinforcement Learning, similar to the Cognitive Packet Network (CPN) [14; 15]. An overlay network with a similar scheme is described in [16]. The rest of the paper is organised as follows. In the remainder of this section we briefly describe the NDN forwarding engine. In Section 2, we review related work on forwarding in NDN, while Section 3 presents a reinforcement learning algorithm for the RNN [13; 17]. The _NDN forwarding strategy using reinforcement learning with the RNN_ (NDNFS-RLRNN) is detailed in Section 4. In Section 5, we describe the system we evaluate in our simulations, and we discuss our results in Section 6.