## 4 Proposed Work
### Personalized Recommendation and Route Planning  
In parallel to the message dissemination, each vehicle of the driver is precisely handled by Edge Node for provisioning of personalized assistance. This precise assistance is recommended by the past, present, and future behavior of the vehicles. We can also send one or more assistance provided according to the current state by using a tri-agent-based soft actor critic (TA-SAC) algorithm. Each agent purpose can be as follows,  
* Agent 1: It is used to disseminate the warning messages to nearby vehicles and edges
* Agent 2: It is used to determine personalized assistance for behaviors according to the current state
* Agent 3: It is used for route planning according to the drivers' preferences  
This process is constructed as a Markov decision process (MDP) with corresponding state (\(\hat{\mathcal{H}}\)), action (\(\hat{\mathrm{A}}\)), and reward (\(Rw\)). Let the policy of process be denoted as \(\pi_{\phi}(\hat{\mathrm{A}}_{t}\big{|}\hat{s}_{t})\). Then, the soft Q function and state value function can be denoted as \(Q_{\theta}\big{(}\hat{s}_{t},\hat{\mathrm{A}}_{t}\big{)}\) and \(V_{\psi}(\hat{s}_{t})\), respectively. The state represents the current state of the driver and the action represents the dissemination of warning message, providing personalized assistance and route planning which is performed by three different agents.  
The squared residual error is minimized by introducing a separate function approximation. The training of soft value function can be expressed as,  
\[K_{v}(\psi)=E_{\mathfrak{H}_{t}\sim D}\left[\frac{1}{2}\Big{(}V_{\psi}(\mathfrak{H }_{t})-E_{\Omega_{t}\sim\pi_{\phi}}\Big{[}Q_{\theta}\Big{(}\mathfrak{H}_{t}, \dot{\mathbb{A}}_{t}\Big{)}-\log\pi_{\phi}(\dot{\mathbb{A}}_{t}\Big{|}\mathfrak{H }_{t})\Big{]}\Big{)}^{2}\right] \tag{13}\]  
The estimation of gradient function can be formulated as,  
\[\hat{\
abla}_{\psi}K_{v}(\psi)=\
abla_{\psi}\ V_{\psi}(\mathfrak{H}_{t})\Big{(} V_{\psi}(\mathfrak{H}_{t})-Q_{\theta}\Big{(}\mathfrak{H}_{t},\dot{\mathbb{A}}_{t }\Big{)}+\log\pi_{\phi}(\dot{\mathbb{A}}_{t}\Big{|}\mathfrak{H}_{t})\Big{)} \tag{14}\]  
The current policy is considered to perform sampling of actions. The training of soft Q function is optimized by using a stochastic gradient, which can be formulated as,  
\[\hat{\
abla}_{\theta}K_{Q}(\theta)=\
abla_{\theta}Q_{\theta}\Big{(}\dot{ \mathbb{A}}_{t},\mathfrak{H}_{t}\Big{)}\big{(}Q_{\theta}\Big{(}\mathfrak{H}_{ t},\dot{\mathbb{A}}_{t}\Big{)}-Rw\Big{(}\mathfrak{H}_{t},\dot{\mathbb{A}}_{t} \Big{)}-\delta\ V_{\psi}(\mathfrak{H}_{t+1}) \tag{15}\]  
The learning of parameter policy is carried out to achieve the optimal policy, which can be formulated as,  
\[K_{\pi}(\phi)=E_{\mathfrak{H}_{t}\sim D,\varepsilon_{t}\sim N}\big{[}\log\pi_ {\phi}\big{(}f_{\phi}(\varepsilon_{t};\mathfrak{H}_{t})\big{|}\mathfrak{H}_{ t}\big{)}-Q_{\theta}\big{(}\mathfrak{H}_{t},f_{\phi}(\varepsilon_{t}; \mathfrak{H}_{t})\big{)}\big{]} \tag{16}\]  
where \(f_{\phi}(\varepsilon_{t};\mathfrak{H}_{t})\) denotes the action \(\dot{\mathbb{A}}_{t}\) and by doing so, the optimal policy is achieved for the three agents. For instance, if the current state of the driver is detected as distracted and drowsy, then the assistance recommended to the driver is to take a rest and drive defensively. The current location of the vehicles is used to provide assistance to driver such as \"There is a coffee shop in 100 m ahead, take a break and drive safely\". The audio system of the vehicle is utilized to provide these recommendations. Figure 5 presents the architecture of the TA-SAC algorithm in providing precise recommendation, warning message dissemination, and route planning. The Pseudocode 3 of our proposed TA-SAC algorithm is provided below. Figure 6 presents the overall flow chart of MODAL-IoCV approach.  
Figure 5: working of TA-SAC algorithm.  
Figure 6: Overall Flowchart.