# Multi-Decision Dynamic Intelligent Routing Protocol for Delay-Tolerant Networks
## 3 Proposed Routing Protocol
### Multi-Decision Dynamic Intelligent Routing Protocol  
This section combines the double Q-learning algorithm with node relationships and message attributes to propose a multi-decision dynamic intelligent routing protocol. In this protocol, Q-values and node relationship values serve as the primary decision criteria for nodes to determine whether to forward messages. Additionally, it takes into account changes in message states to ensure effective message transmission. Within the double Q-learning framework, reward computation considers the network's overall average delay and average hops, effectively controlling the number of message duplicates and reducing message delivery latency during the intelligent learning process. Figure 3 shows the general workflow of the proposed MDDI routing protocol. The MDDI protocol consists of two main parts: a single decision based on double Q-values  
(SDDQ) and a dynamic decision based on node relationship (DDNR). SDDQ mainly uses the Q-value in the Q-learning algorithm to make decisions, and the discount factor changes dynamically and is determined by the state of the message in the node. While the actual reward value is updated after the message reaches the destination node. When the connecting node is not the destination node of the message, SDDQ protocol is applied first; if the message cannot be forwarded according to SDDQ protocol, then DDNR is further used. DDNR uses node relationships and message attributes to make a dynamic decision, and the node relationship is derived according to the interaction information between nodes and then integrated with the state of the message at this point in time to make a decision on whether to forward or not. Finally, the Q value and relationship value are updated. The following section provides a detailed description of the application process of this routing algorithm within the network. In this multi-decision dynamic intelligent routing model, each node in the network stores and updates four tables: two Q-tables, a dynamic node attribute table for participating in routing decisions, and a dynamic message attribute table. The updating of the Q-tables is based on the description in Section 2.2. The dynamic node attribute table contains global information related to the node. It stores encounter information about all the connected nodes for this node, including the number of historical connections within time interval \(T\), average connection duration, average message forwarding count, and average encounter interval. The Dynamic Message Attribute Table stores message and node-related attributes, including the time and number of hops that a message is forwarded from that node to the next-hop node until it reaches the destination node, and the discount value used by that node in calculating the Q-value of the node that determines the next-hop node for that message. As an example, let's consider a general node \(a\) within the network. As shown in the tables below, In Table 3, \(b\), \(c\), \(d\) denote the nodes that have established a connection with node \(a\). The table stores their historical number of encounters with node \(a\) during the period \(T\), the average encounter duration, the average number of forwarded messages, and the average encounter interval during the period \(T^{\prime}\). In Table 4, \(m^{e}\), \(m^{f}\), \(m^{g}\) are the messages stored in node \(a\), and their corresponding dynamic attributes are \(\gamma(m)\), \(Hop^{m}\), \(t^{m}\), where \(\gamma(m)\) is the discount factor corresponding to different messages, and \(Hop^{m}\) and \(t^{m}\) correspond to the number of hops and time to reach the destination node after the message is forwarded to different nodes  
Figure 3: Workflow of Proposed MDDI Protocol.