## 3 System Design and the Proposed SEAC-Based Resource Allocation Methodology
### SEAC-Based Resource Allocation Algorithm  
In this section, we detail the SEAC methodology for efficient resource allocation in 5G-NR-V2X Mode 2 networks. The SEAC approach leverages a multi-agent setup, as depicted in Figure 4, where each vehicular agent learns to optimize its resource allocation strategy while sharing experiences with other agents to enhance the overall system performance. The SEAC model efficiently allocates communication resources among vehicles by employing both actor and critic networks. The actor network is responsible for selecting actions based on the current state \(s_{t}\) provided by the 5G-NR-V2X Mode 2 environment. The actor network generates an action \(a_{t}\) using its online policy network \(\mu\left(s_{t},\theta^{\mu}\right)\), which is subsequently executed in the environment. This action influences the environment, leading to a new state \(s_{t+1}\) and a corresponding reward \(r_{t}\)[21; 22]. The critic network evaluates the selected actions by computing the value function, providing feedback to the actor network. It consists of two \(Q\)-values: the online network and the target value network. The online value network calculates the \(Q\)-value \(Q\left(s,\ a,\ \theta^{Q}\right)\), while the target value network computes the target \(Q\)-value \(Q^{\prime}\left(s_{t+1},\ a^{\prime},\ \theta^{Q^{\prime}}\right)\), where \(a^{\prime}=\mu^{\prime}(s_{t+1})\) helps in refining the value function  
Figure 4 illustrates the Shared Experience Actor-Critic (SEAC) mechanism, where vehicles collaborate to improve resource allocation through shared learning in a 5G-NR-V2X environment. The process begins with each vehicle observing its environment state and using the actor network to select a resource allocation action. The critic network then evaluates this action by calculating a \(Q\)-value that reflects the effectiveness of the resource allocation. The outcome (state-action-reward tuple) is stored in a local experience pool  
Figure 4: Network structure and flow of the SEAC algorithm. for future training. To further enhance learning, vehicles share their experiences via V2V communication, which are aggregated into a shared experience pool, enabling each vehicle to learn from both its own and others' experiences. This shared learning improves the overall decision-making and resource optimization process. The actor and critic networks are updated through a process involving gradient calculations (policy gradient for the actor and Q-value gradient for the critic) based on the critic's feedback. Soft updates are applied to ensure stable learning, adjusting the target policy and value networks gradually. The temporal difference (TD) error is computed to refine the critic's evaluations, ensuring accurate learning over time. This iterative process allows vehicles to continuously refine their policies, leading to more efficient resource allocation and improved network performance by leveraging both local and shared experiences. The TD error is crucial in evaluating the difference between the expected value of the current state-action pair and the value after observing the outcome. This error helps in refining the value function. The TD error can be expressed as  
\[\delta_{t}=r_{t}+\gamma Q^{\prime}\Big{(}s_{t+1},\ a^{\prime},\theta^{Q^{ \prime}}\Big{)}-Q\ \Big{(}s_{t},\ a_{t},\ \theta^{Q}\Big{)} \tag{5}\]  
where  
* \(r_{t}\) is the immediate reward at time \(t\);
* \(\gamma\) is the discount factor;
* \(Q^{\prime}\Big{(}s_{t+1},\ a^{\prime},\theta^{Q^{\prime}}\Big{)}\) is the target value for the next state-action pair;
* \(Q\ \big{(}s_{t},\ a_{t},\ \theta^{Q}\big{)}\) is the current estimate of the state-action value. The critic's value function \(Q\ \big{(}s,\ a,\ \theta^{Q}\big{)}\) is updated based on the TD error, and the actor's policy is refined using the policy gradient calculated by the critic.