## 6 Investigating Several RL-Based Routing Methods
### [PERSON] et al. [64] have presented the greedy routing with link stability (GLS) to the software-defined vehicular networks. This method divides the vehicular network into several sub-areas so that each intersection is located at the center of each sub-area. The controller performs the area selection process (AS) using a proactive manner. In GLS, the central controller checks traffic conditions in each area and initializes the entries of the routing table using a fuzzy system. This fuzzy system consists of three input parameters, namely mixed distribution (MD), one-way connectivity (OC), and valid distance (VD). Note that two scales, namely the density of vehicles in a road segment and the vehicle distribution in that area are used for calculating the MD parameter. The OC parameter is also determined by the density and movement direction of vehicles and indicates the successful packet delivery rate from an area to the adjacent area. Finally, the VD parameter guarantees that packages are sent to areas toward the destination and are close to it. According to this fuzzy system, a transfer priority is determined for each road segment and is recorded into the routing table. Then, a reinforcement learning scheme is used to select an area. This helps GLS to better adapt to the dynamics of the network environment. In this learning model, the controller expresses the agent, and each area in the network indicates a state. Moreover, the set of actions is to select adjacent areas for transferring data packets. Figure 19 shows the learning structure in GLS. This routing scheme presents a new technique for updating Q-table (also known as the routing table). In the multi-hop data transmission process, each packet stores all areas passed between the source node and the desired area. After ending this process, a report message about this route is uploaded into the controller. Then, it uses this information for updating Q-table. Upon receiving this message, the controller uses a tree-based positive updating mechanism to update successful paths in the routing table and improve their corresponding Q-values by giving a positive reward. This positive reward is calculated based on MD, OC, and VD parameters. On the other hand, if the routes are broken when sending the data, the controller uses a loss-aware negative updating mechanism to penalize the paths that have increased packet loss by giving a negative reward. This negative reward is calculated based on MD, OC and VD, and the size of the buffer capacity. After determining the connections between the adjacent areas using fuzzy logic and reinforcement learning, the central controller is responsible for deciding on the route based on the routing table. When the source node needs a new path between itself and the destination node, it transmits a route request message to the controller. After receiving this request, the AS operation is started by searching the routing table to determine the suitable route for transferring data (a sequence of intersections with maximum Q-value). After determining the optimal path, the controller sends this path to the requesting node through a message. The source node records this path in the packet header and sends it to the intermediate node, which is selected using the relay vehicle selection (RS) process. The intermediate node extracts the route information from the packet header and determines the next area for sending the data. In the relay vehicle selection  
Figure 18: Learning structure in RLRC.  
(RS) process, each vehicle uses a greedy forwarding technique to choose a relay vehicle with the maximum link stability from its neighboring vehicles close to the destination. For this purpose, the transmitter vehicle uses a fuzzy system to calculate the link stability for each neighboring vehicle. This fuzzy mechanism involves three inputs, including distances, movement direction, and speed. This information is obtained by broadcasting hello message in the network.