# Enhanced Routing Algorithm Based on Reinforcement Machine Learning--A Case of VoIP Service
## 3 The Proposed e-RLRP Algorithm
### Algorithm Used in the e-RLRP to Reduce the Overhead  
To send a packet, the node needs to know what neighbors nodes are directly connected. Therefore, a neighborhood discovery procedure is required. In RLRP, this procedure occurs through the broadcasting of messages called Hello. By default, Hello messages are sent every 2 s, thus, the information about neighbors is updated in the same period of time. This update interval parameter is called Broadcast Interval (BI). The RLRP has 10 types of headers, two of them are Reward Header and Hello Header. The structure of data fields of the Reward Header and Hello Header are shown in Tables 3 and 4, respectively. The Reward Header is 8 bytes. The Type field defines what the header is, the ID field is the unique identifier of the message service. The Neg Reward Flag field is a test flag that checks whether the reward is negative or positive, the Reward Value is a value of reward, and finally, the Msg Hash is the identifier of the packet to which the reward belongs. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Field Name** & **Size (Bits)** & **Description** \\ \hline TYPE & 4 & Type ID of the header \\ IPV4\_COUNT & 1 & Number of IPv4 addresses \\ IPV6\_COUNT & 2 & Number of IPv6 addresses \\ TX\_COUNT & 24 & Number of frame re-broadcasts \\ GW\_MODE & 1 & Indicates GateWay Mode \\ IPV4\_ADDRESS & 32 & IPv4 address (if exists) \\ IPV6\_ADDRESS\_1 & 128 & IPv6 address 1 (if exists) \\ IPV6\_ADDRESS\_2 & 128 & IPv6 address 2 (if exists) \\ IPV6\_ADDRESS\_3 & 128 & IPv6 address 3 (if exists) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Hello Header Format. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Field Name** & **Size (Bits)** & **Description** \\ \hline TYPE & 4 & Type ID of the header \\ ID & 20 & ID of the service message \\ NEG\_REWARD\_FLAG & 1 & Check reward \\ REWARD\_VALUE & 7 & Value of the reward \\ MSG\_HASH & 32 & ID of the data packet \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Reward Header Format. The Hello Header size ranges from 4 to 56 bytes, this variation depends on the node address that can be IPV4 or IPV6. The Type field defines what the header is, the field IPv4 Count defines the number of assigned IPv4 addresses, limited to one. The IPv6 Count is number of assigned IPv6 addresses, limited to three. Tx Count is the number of re-broadcasts, GW Mode define that a node is a Gateway in the network, the IPv4 and IPv6 address define the address of the node. As can be observed in Table 3, the Reward header used in RLRP is 8 bytes long and generates an additional overhead, which corresponds to the use of RL technique. In this context, the present research implemented an algorithm to reduce the overhead generated by the Hello message, specifically to reduce the frequency of sending Hello messages in order to compensate the additional overhead generated by the Reward header. It is clear that increasing the time interval for sending Hello messages, defined by the \(BI\) parameter, will decrease the frequency of sending messages and consequently, the overhead is also decreased. However, a high value also impacts the time of updating information about the neighborhood, and the routing can be negatively affected. Thus, the proposed algorithm implemented in the e-RLRP is capable of dynamically adjusts the frequency of sending Hello messages. This adjustment in the parameter \(BI\) is made according to the mobility present in the network. If the network is static, that is, no neighbors enter or leave the coverage range, it is not necessary to send Hello messages with a high frequency. Otherwise, if the network presents a high mobility, to send messages more frequently is necessary. A general high representation of the proposed algorithm is introduced in Figure 2.  
The sending of Hello messages starts together with the e-RLRP daemon. Next, the algorithm checks the mobility of the network.