# Decentralized LSTM-Enhanced MADDPG Framework for Vehicular Edge Computing

Romain Dulout, Leo Mendiboure, Yannis Pousset, Virginie Deniau, Tidiane Sylla

► To cite this version:

Romain Dulout, Leo Mendiboure, Yannis Pousset, Virginie Deniau, Tidiane Sylla. Decentralized LSTM-Enhanced MADDPG Framework for Vehicular Edge Computing. IEEE 36th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC 2025), Sep 2025, Istanbul, Turkey. 10.1109/PIMRC62392.2025.11274952. hal-05535046

HAL Id: hal-05535046

https://hal.science/hal-05535046v1

Submitted on 3 Mar 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Decentralized LSTM-Enhanced MADDPG Framework for Vehicular Edge Computing

Romain Dulout<sup>1,2</sup>, Leo Mendiboure<sup>2</sup> (IEEE Member), Yannis Pousset<sup>1</sup>, Virginie Deniau<sup>3</sup>, Tidiane Sylla<sup>2,4</sup>

<sup>1</sup>XLIM (UMR CNRS 7252), Univ de Poitiers, F-86000 Poitiers, France

<sup>2</sup>COSYS-ERENA, Univ. Gustave Eiffel, F-77454 Marne-la-Vallee, France

<sup>3</sup>COSYS-LEOST, Univ. Gustave Eiffel, F-59666 Villeneuve d'Ascq Cedex, France

<sup>4</sup>LaMIA, Univ. Sciences, Techniques and Technologies of Bamako, Bamako, Mali

**Abstract**—In today's intelligent transportation systems, emerging vehicular applications, such as multi-vehicle data fusion, platoon management, and high-definition video processing, demand a new generation of communication architectures to meet stringent processing and latency requirements. NOMA-based Vehicular Edge Computing (VEC) architectures have gained prominence as a solution, offering efficient spectrum reuse and reduced processing delays by leveraging distributed edge resources. While decentralized management of these architectures promises real-time, adaptive task offloading, existing approaches are limited by their reliance on local observations and insufficient integration of comprehensive global network state information. To address these challenges, we propose ROLAM, a novel framework that fuses LSTM-based state estimation with Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to optimize resource allocation under dynamic conditions. Our evaluation demonstrates that ROLAM significantly improves energy efficiency and overall system performance, underscoring its potential to robustly meet the evolving needs of next-generation vehicular applications.

**Index Terms**—NOMA, MEC, AI, MADRL, Energy Efficiency

## I. INTRODUCTION

The development of automated and connected vehicles, coupled with advanced vehicular communication networks, is emerging as a solution to road safety challenges and to reduce the energy footprint of transportation [1]. These technologies require real-time applications that demand significant computing resources, such as combining data from multiple vehicles, remotely driving vehicles and managing platoons. Traditional Cloud architectures struggle to meet these demands, prompting a shift toward distributed architectures with computing resources at the network edge, known as Vehicular Edge Computing (VEC) [2], to reduce latency.

However, VEC alone does not fully address these applications' challenges, which require high bandwidth to transmit large data volumes (e.g., videos). To remedy this, a solution is to integrate VEC with Non-Orthogonal Multiple Access (NOMA), a technique that enables multiple users to share the same frequency resources simultaneously by exploiting power differences [3]. The resulting NOMA-VEC paradigm reduces energy consumption and processing delays, offering a promising approach for real-time vehicular services [4].

Recent studies such as [5], [6] have shown that decentralized NOMA-VEC approaches, where vehicles autonomously determine their offloading strategies, can significantly reduce

energy consumption compared to centralized methods by allowing real-time adaptation to variable channel conditions and stochastic task arrivals. However, these methods typically rely on limited local observations, with each vehicle monitoring only its own queue length or instantaneous channel quality, thus missing critical global network insights, such as the computational resources available at the MEC server level. Consequently, this fragmented perspective hinders coordination and load balancing at the system level, leading to suboptimal task offloading and MEC resource overload.

Furthermore, these studies adopt unrealistic modeling assumptions by presuming that vehicles can directly measure uplink SINR through channel reciprocity, which is unrealistic in mobile environments. They also exclude the consideration of the limited computational capacity of MEC servers, which requires consideration of resource constraints at the processing level. More realistic approaches are required.

To address these shortcomings, we propose ROLAM (Resilient Offloading using LSTM-Assisted MADDPG), a resilient, decentralized offloading solution for NOMA-VEC. ROLAM integrates an LSTM-based estimator within a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) architecture to compensate for unreliable base station (BS) broadcasts by effectively handling packet loss and outdated information, while also taking into account the server load. In our approach, each vehicle continuously receives comprehensive, cross-layer global-state information via BS broadcasts, enabling real-time, cooperative decision-making.

Our contributions are threefold: a) we introduce ROLAM, a decentralized offloading framework for NOMA-VEC that provides vehicles with a real-time view of network conditions; b) within this framework, we combine LSTM with MADDPG to overcome the limitations of local observation and optimize resource allocation under dynamic conditions, including limited MEC computational resources; and c) we conduct extensive evaluations under varying user densities to validate ROLAM effectiveness compared to existing solutions.

The remainder of this paper is organized as follows. Section II presents the system model. Section III describes the proposed LSTM-MADRL solution. Results and a discussion are presented in Section IV, and Section V concludes this paper.
<figure><img src="image_2.png" /><figcaption>Fig. 1: NOMA-VEC Architecture</figcaption></figure>

## II. SYSTEM MODEL AND PROBLEM FORMULATION

We consider a NOMA-VEC system comprising a set $\mathcal{I} = \{1, 2, \dots, I\}$ of single antenna vehicles and a BS with $M$ antennas ($M = I$) and an integrated MEC server. The proposed system supports real-time vehicular applications by jointly leveraging local computation and offloading to the MEC server. To address strict latency (time from task arrival until processing is completed) and energy constraints, our framework employs a Time Division Duplexing (TDD) scheme with dedicated uplink and downlink phases.

### A. Task Generation and Time Dynamics

Each vehicle executes critical tasks, such as real-time sensor data processing or collision avoidance that are represented as bit streams stored in an on-board FIFO task buffer. New tasks arrive in each discrete time slot $t \in \mathcal{T} = \{1, 2, \dots, T\}$. With limited local processing, vehicles must balance computation with offloading to the MEC server, managed via resource allocation: $p_i^{\text{loc}}(t)$ is the fraction of local CPU power for processing, while $p_i^{\text{off}}(t)$ is the fraction of transmit power for offloading. These parameters directly determine the rate at which data is processed locally versus offloaded.

Each 10 ms time slot is partitioned as $\tau = \tau^{\text{ul}} + \tau^{\text{dl}}$, with $\tau^{\text{ul}} = 8$ms for uplink offloading and $\tau^{\text{dl}} = 2$ms for downlink feedback and reception of processed results. Allocating 80% to downlink and 20% to uplink is a potential configuration in 5G TDD systems. To meet high offloading demands, we invert this ratio (80% uplink). By combining cross-layer local information with global BS observations, vehicles can make cooperative offloading decisions in real time for a more efficient NOMA-VEC environment.

The task buffer state for vehicle $i$ at time $t$, $B_i^{\text{car}}(t)$, is a key performance indicator; according to Little's Theorem [7], a smaller buffer implies lower latency. The buffer evolves as:

$$B_i^{\text{car}}(t) = \max \left\{ 0, B_i^{\text{car}}(t-1) - (b_i^{\text{loc}}(t-1) + b_i^{\text{mec}}(t-1)) \right\} + \lambda_i(t-1) \quad (1)$$

where $b_i^{\text{loc}}(t)$ and $b_i^{\text{mec}}(t)$ denote the number of bits processed locally and on the MEC server, respectively, and $\lambda_i(t)$ represents new tasks arriving during slot $t$. In this work, arrivals are modeled as a Poisson process where the task arrival rate $\mu_{\lambda}$ is set at 5Mbps, a common assumption for independent, memoryless patterns. Each arriving task is assumed independent, so there are no data dependencies requiring synchronized processing of local and offloaded bits.

### B. Vehicular Mobility Modeling

We model vehicle mobility using the SUMO simulator, which provides realistic traffic dynamics. At time $t$, we extract three key metrics for network modeling: the distance to the BS $d_i(t)$, vehicle speed $v_i(t)$, and the azimuth angle relative to the BS $\Phi_i(t)$. We chose to model a residential neighborhood in Bordeaux, France, where the speed limit is 50 km/h, using tools such as OpenStreetMap, Netconvert, Netedit, and SUMO's Car-Following model [8] which ensure coherent, realistic vehicular movement and supplies the necessary metrics.

### C. NOMA-Based Offloading Model

We adopt the NOMA communication model as presented in [5]. The uplink channel matrix at time $t$ is defined as:

$$\mathbf{H}^{\text{ul}}(t) = [\mathbf{h}_1^{\text{ul}}(t), \mathbf{h}_2^{\text{ul}}(t), \dots, \mathbf{h}_I^{\text{ul}}(t)] \in \mathbb{C}^{M \times I},$$

where $\mathbf{h}_i^{\text{ul}}(t)$ is the channel vector between vehicle $i$ and the BS antennas. The channel is modeled as the product of large-scale fading and small-scale fading effects:

$$\mathbf{h}_i(t) = \sqrt{h_r/d_i(t)^{\eta}} \mathbf{h}_i^{\text{small}}(t), \quad (2)$$

where $h_r = -30$ dB is the attenuation at 1 m from the BS, and $\eta = 2$ is the path loss exponent.

The small-scale fading component $\mathbf{h}_i^{\text{small}}(t)$ is modeled as an autoregressive (AR) process to capture the correlation between consecutive time slots:

$$\mathbf{h}_i^{\text{small}}(t) = \rho_i(t) \mathbf{h}_i^{\text{small}}(t-1) + \Delta(t) \sqrt{1 - \rho_i(t)^2}, \quad (3)$$

where $\rho_i(t)$ is the normalized channel correlation coefficient between consecutive time slots and $\Delta(t)$ is a complex Gaussian error vector. The correlation coefficient is computed using Jake's fading spectrum model: $\rho_i(t) = J_0(2\pi f_i^d(t)\tau^{\text{ul}})$, with $J_0(\cdot)$ denoting the zeroth-order Bessel function of the first kind. The Doppler frequency is given by: $f_i^d(t) = (v_i(t)/\lambda^{\text{wave}}) \cos \Phi_i(t)$,

where $v_i(t)$ is the speed of vehicle $i$ at time $t$, $\lambda^{\text{wave}}$ is the wavelength with $\lambda^{\text{wave}} = 0.1$ m, and $\Phi_i(t)$ is the azimuth angle between vehicle $i$ and the BS.

The received uplink signal at the BS is expressed as

$$\mathbf{y}(t) = \sum_{i=1}^{I} \mathbf{h}_{i}^{\text{ul}}(t) \sqrt{p_{i}^{\text{off}}(t)} s_{i}(t) + \mathbf{n}(t), \quad (4)$$

where $s_i(t)$ denotes the complex symbol transmitted by user $i$ and $\mathbf{n}(t) \sim \mathcal{CN}(0, \sigma_{\text{noise}}^2 \mathbf{I}_M)$, with $\mathbf{I}_M$ being the $M \times M$ identity matrix, and $\sigma_{\text{noise}}^2 = 10^{-9}$ W.

To decode and extract the individual user signals, the BS employs a Zero Forcing detector computed as:

$$\mathbf{H}^{\text{ZF}}(t) = [(\mathbf{H}^{\text{ul}}(t))^H \times \mathbf{H}^{\text{ul}}(t)]^{-1} (\mathbf{H}^{\text{ul}}(t))^H. \quad (5)$$

where $(\cdot)^H$ denotes the Hermitian (conjugate) transpose.

Let $\mathbf{h}_i^{\text{ZF}}(t)$ denote the $i^{\text{th}}$ row of $\mathbf{H}^{\text{ZF}}(t)$. According to [9], we have $\mathbf{h}_i^{\text{ZF}}(t) \mathbf{h}_k^{\text{ul}}(t) = \delta_{i,k}$, where $\delta_{i,k} = 1$ if $i = k$, and 0 otherwise. The signal corresponding to user $i$ is extracted as:

$$y_i(t) = \mathbf{h}_i^{\text{ZF}}(t) \mathbf{y}(t) = \sqrt{p_i^{\text{off}}(t)} s_i(t) + \mathbf{h}_i^{\text{ZF}}(t) \mathbf{n}(t). \quad (6)$$

Consequently, the SINR for user $i$ is given by:

$$\chi_i(t) = [\|\mathbf{h}_i^{\text{ZF}}(t)\|^2 \times \sigma_{\text{noise}}^2]^{-1}, \quad (7)$$
where $\chi_i(t)$ is the normalized SINR reflecting the channel conditions without the impact of the NOMA power allocation, and thus $\text{SINR}_i(t) = p_i^{\text{off}}(t) \times \chi_i(t)$. Finally, the number of bits offloaded by user $i$ to the BS is:

$$b_i^{\text{off}}(t) = \tau^{\text{ul}} W^{\text{ul}} \log_2 (1 + \text{SINR}_i(t)), \quad (8)$$

where $W^{\text{ul}} = 1$ MHz denotes the available uplink bandwidth.

### D. Computation and Task Processing Model

During the uplink phase of time slot $t$, the MEC server processes the tasks that were offloaded during the previous time slot ($t-1$). Note that during the following downlink phase at $t-1$, the BS completes the demodulation of the signals received during the uplink phase and forwards the resulting bits to the MEC server. Consequently, the MEC queues for each user $i$, denoted by $B_i^{\text{mec}}(t)$, contain exactly the bits demodulated in the downlink phase of $t-1$. To meet the stringent real-time requirements of our system, any bits that remain unprocessed within the designated processing interval are intentionally discarded, thereby preventing outdated data from affecting subsequent operations. The available processing capacity at the MEC server at time $t$ is given by:

$$R^{\text{mec}}(t) = L_m^{-1} \sum_{l=1}^{L} (1 - \varphi_l^{\text{load}}(t)) f^{\text{mec}} \tau^{\text{ul}}, \quad (9)$$

where $L_m = 500$ is the number of CPU cycles per bit, $L$ is the number of CPUs (with $L=I$), $f^{\text{mec}} = 5$ GHz is the maximum CPU frequency, and $\varphi_l^{\text{load}}(t)$ is the load on the $l^{th}$ CPU at time $t$. CPU load information is obtained using an emulator described in [10]. It allows us to sample the workload of each CPU of an edge server at $\tau^{\text{ul}}$, subject to different types of service. The resulting dataset corresponds to CPU loads between 75% and 95%, which in our model are non-vehicular services using edge resources. Remaining capacity $(1 - \varphi_l^{\text{load}}(t))$ is allocated to process the vehicular tasks.

A scheduling algorithm (e.g. Round-Robin) is then employed to distribute these resources among the users, yielding the number of bits processed for user $i$, denoted by $b_i^{\text{mec}}(t)$.

In parallel with MEC processing, each vehicle processes tasks locally using its onboard computing resources. The local CPU frequency for vehicle $i$ at time $t$ is determined by

$$f_i^{\text{loc}}(t) = \sqrt[3]{p_i^{\text{loc}}(t)/\kappa}, \quad (10)$$

where $\kappa = 10^{-29}$ is the effective switched capacitance. Accordingly, the number of bits processed locally by vehicle $i$ during time slot $t$ is given by

$$b_i^{\text{loc}}(t) = \tau L_m^{-1} f_i^{\text{loc}}(t). \quad (11)$$

### E. Downlink Communication and Global State Feedback

During the downlink phase, the BS delivers two types of information: processed results from the MEC server and global state feedback. The processed results payload is small compared to offloaded data (cf. the analysis proposed by the authors of [11]), allowing each vehicle to clear $b_i^{\text{mec}}(t)$ from its buffer. Additionally, the BS broadcasts global state information, specifically the uplink SINR and the MEC server

load, which are not available at the vehicles level. This global state allows each vehicle to go beyond their local vision for better offloading decision-making.

The downlink uses an OFDM-based transmission scheme. Let $\mathcal{N} = \{1, 2, \dots, N\}$ denote the set of available OFDM subcarriers ($N=120$). The downlink channel is: $\mathbf{H}^{\text{dl}}(t) = [\mathbf{h}_1^{\text{dl}}(t), \mathbf{h}_2^{\text{dl}}(t), \dots, \mathbf{h}_I^{\text{dl}}(t)] \in \mathbb{C}^{N \times I}$, with each $\mathbf{h}_i^{\text{dl}}(t)$ computed analogously to the uplink channel in Eq. (2).

The BS assigns contiguous groups of 12 subcarriers (one Resource Block, RB) to each vehicle. Denote by $S_i^{\text{dl}} \subset \mathcal{N}$ the set of subcarriers allocated to vehicle $i$, such that $\bigcup_{i=1}^I S_i^{\text{dl}} = \mathcal{N}$ and, ideally, $|S_i^{\text{dl}}| = N/(12I)$ for each vehicle. Based on the allocated subcarriers, the number of bits transmitted to vehicle $i$ on the downlink is given by:

$$b_i^{\text{dl}}(t) = \tau^{\text{dl}} \sum_{n \in S_i^{\text{dl}}} W^{\text{dl}} \log_2 \left( 1 + p_{\text{max}}^{\text{off}} |\mathbf{h}_{i,n}^{\text{dl}}(t)|^2 / \sigma_{\text{noise}}^2 \right), \quad (12)$$

where $W^{\text{dl}} = 1.8$ MHz is the downlink bandwidth. Then, the Communication Error Ratio (CER) is defined to quantify the error ratio in the received downlink frame for each vehicle: $CER_i^{\text{dl}}(t) = 1 - \min(1, I \times b_i^{\text{dl}}(t)/b_{\text{max}}^{\text{dl}})$, where $b_{\text{max}}^{\text{dl}}$ denotes the maximum number of bits that can be transmitted over the OFDM grid, defined by the number of RBs on the two subslots ($\tau^{\text{dl}}$), and the number of bits per RB.

For state feedback, each vehicle $i$ is allocated a set of RBs dedicated to carrying state information. Let $K^{\text{rb}}$ denote the number of RBs available per vehicle and $K^{\text{bits}}$ the number of bits per RB (assuming, for instance, 8-PSK modulation, 14 OFDM symbols per subframe, and 12 subcarriers per RB, thus $K^{\text{bits}} = 504$ bits). We form the set $\mathcal{K}_i = \{Y_{i,1}, Y_{i,2}, \dots, Y_{i,K^{\text{rb}}}\}$, where each RB is defined as: $Y_{i,k} = \{X_{i,k,1}, X_{i,k,2}, \dots, X_{i,k,K^{\text{bits}}}\}$, and each bit $X_{i,k,k'}$ is modeled as a Bernoulli random variable with parameter $CER_i^{\text{dl}}(t)$, used to generate bits with and without errors. Each RB is encoded using LDPC codes with a coding rate of $1/3$, which theoretically enables the correction of up to 10% of errors within the codeword [12].

For each vehicle, one RB is randomly selected from $\mathcal{K}_i$ according to a uniform distribution, representing the RB that carries the BS global states. This configuration is adequate, as the BS transmits only two states, encoded in two floating-point numbers (64 bits each), which is smaller than the useful capacity of an RB under this coding scheme. Let $\psi_i(t)$ indicate whether the states were correctly received by the vehicles. $\psi_i(t) = 1$ is defined as the event where the number of corrupted bits in the RB is strictly less than 10% of $K^{\text{bits}}$ indicating that the states have been correctly received, $\psi_i(t) = 0$ otherwise.

### F. Problem Formulation

The main objective is to determine the optimal allocation of computational and radio resources, specifically $p_i^{\text{loc}}(t)$ and $p_i^{\text{off}}(t)$, for each vehicle $i$ so as to minimize the cumulative processing cost under stochastic task arrivals $\lambda_i(t)$, uncertain channel conditions, and fluctuating MEC server capacity
$R^{\text{mec}}(t)$. This strategy ensures both rapid processing and low energy consumption. To quantify this trade-off, we define the computational cost for vehicle $i$ at time $t$ as:

$$C_i(t) = c^{\text{pow}}(p_i^{\text{loc}}(t) + p_i^{\text{off}}(t)) + c^{\text{buf}} B_i^{\text{car}}(t),$$

where $c^{\text{pow}}$ and $c^{\text{buf}}$ are weighting coefficients computed to ensure that energy consumption and latency are equally prioritized (ie 50-50). These coefficients are derived such that the contributions of local processing/offloading power and task buffer occupancy to the overall cost are balanced. The long-term average optimization problem can then be formulated as:

$$(P1): \quad \begin{array}{ll} \min_{p_i^{\text{loc}}(t), p_i^{\text{off}}(t)} & \lim_{T \to \infty} \frac{1}{T} \sum_{t=0}^{T} \sum_{i=1}^{I} C_i(t) \\ \text{subject to} & p_i^{\text{loc}}(t) \in [0, p_{\text{max}}^{\text{loc}}], \quad \forall t \in T, \\ & p_i^{\text{off}}(t) \in [0, p_{\text{max}}^{\text{off}}], \quad \forall t \in T. \end{array}$$

This formulation captures the joint resource allocation and task scheduling problem in a NOMA-VEC setting, with the aim of minimizing the overall computational cost while satisfying stringent latency and energy constraints.

### III. ROLAM: A DECENTRALIZED MADRL FRAMEWORK FOR COOPERATIVE OFFLOADING

This section presents our decentralized and cooperative solution, combining LSTM with a MADDPG framework, to solve the long-term problem (P1): ROLAM.

#### A. MDP Formulation

The NOMA-VEC model is formulated as an MDP, defined by the tuple {$S, A, P, \mathcal{R}, \gamma$}, with each time slot representing a transition. At the start of each slot, each vehicle (agent) selects an action based on its current observation, which then leads to a new state and a reward quantifying the chosen action's effectiveness. The MDP components are as follows:

1) **State Space:** Previous decentralized approaches [5], [6] have shown limitations due to the narrow local information available to vehicles in dynamic environments, which yields suboptimal strategies. Moreover, local SINR observations are unreliable in high-mobility scenarios because channel reciprocity no longer holds. Therefore, the BS acts as a central collector, aggregating a global view of the NOMA-VEC environment (including normalized uplink SINR) and broadcasting this information via downlink before the end of the slot. Exploiting the multi-level architecture of the NOMA-VEC infrastructure (access network, core network, MEC server), we adopt a cross-layer approach that combines PHY, network (extracted from CAM messages in the C-ITS stack), and application metrics. Consequently, each agent forms an observation vector $s_i(t) \in S$, such as $s_i(t) = [\chi_i(t-1), d_i(t), B_i^{\text{car}}(t), R^{\text{mec}}(t-1)]$;

2) **Action Space:** Based on $s_i(t)$, each agent is able to determine its set of actions $a_i(t) \in A$, which correspond to

its offloading strategy, through the allocation of its computational (CPU power) and radio (NOMA transmission power) resources, denoted by $a_i(t) = [p_i^{\text{loc}}(t), p_i^{\text{off}}(t)]$;

3) **Reward:** To minimize processing time and energy consumption, each agent's performance is measured by a cost function, with the reward defined as $r_i(t) = -C_i(t)$;

4) **Transition Tuple:** The overall interaction is captured by the transition tuple {$s_t, a_t, r_t, s'_t$}, which is formed by concatenating the individual agents' states, actions, and rewards, with $s'_t$ representing the next states.

#### B. MADDPG Algorithm

To solve the formulated MDP, ROLAM employs MADDPG, a multi-agent extension of the DDPG algorithm:

1) **Main components:** This approach employs an actor-critic architecture. The actor network (Deep Neural Network -DNN-, denoted by $\pi$) selects an optimal action based on the current observation to maximize long-term rewards, while the critic network (denoted by $Q$) evaluates the quality of that action through the Q-value function. Both networks are parameterized (with the actor's parameters as $\theta_i$ and the critic's as $\phi_i$) and are complemented by corresponding target networks (with parameters $\theta'_i$ and $\phi'_i$) that are updated more slowly to ensure training stability. Training is based on data stored in a common replay memory $\mathcal{B}$, which captures past interactions between the agents and the environment. Additionally, a centralized critic is used to exploit the entire state and action space, thereby fostering improved cooperation among vehicles;

2) **MADDPG Functioning:** The MADDPG operates in two stages, with a training phase which allows the DNN actor of each agent to learn estimating optimal policies with all the above components, followed by a testing phase, during which only these DNNs actors are used to choose actions. Although centralized training with decentralized execution is common in distributed systems [6], it assumes ideal communication conditions. Nevertheless, continuous, reliable uplink and downlink operation is impractical in real vehicular networks. Hence, to guarantee its effectiveness, we perform training in a simulated environment under ideal network conditions, and then use an enhanced simulator during testing to incorporate realistic communication constraints for accurate state feedback, thereby better reflecting real-world environments.

3) **Training Process:** First, the actor-critic DNN parameters $\theta_i$ and $\phi_i$ are randomly initialized, and the target network parameters are set equal to them, i.e., $\theta'_i \leftarrow \theta_i$ and $\phi'_i \leftarrow \phi_i$. Each agent then interacts with the environment for $T_{\text{step}}^{\text{max}} = 200$ steps per episode over $T_{\text{max}}^{\text{ep}} = 2000$ episodes to explore diverse scenarios and facilitate reward convergence.

At each step, an action $a_i(t)$ is generated by each agent such as $a_i(t) = \pi_{\theta_i}(s_i(t)) + \epsilon(t)$, where $\epsilon(t)$ is an Ornstein-Uhlenbeck exploration noise, with the decay rate and variation as 0.15 and 0.02 respectively. The actions are then executed in the simulator, which returns $r_t$ and the new state $s'_t$. This transition {$s_t, a_t, r_t, s'_t$} is then stored in $\mathcal{B}$, with $|\mathcal{B}| = 2.5e^5$. Subsequently, each agent samples a mini-batch of size $M_b = 128$ from $\mathcal{B}$ to update its different networks.
First, $\mathbf{a}'_t$ is calculated, which is related to the concatenation of each agent next actions obtained with target actor network and based on next state. Then, they compute the target Q-value such as $Q_i^{\text{target}} = r_i(t) + \gamma \times Q_{\phi'_i}(\mathbf{s}'_t, \mathbf{a}'_t)$, where $\gamma = 0.95$ corresponds to the discount factor balancing the importance of future rewards against the immediate reward. Main DNNs critic are then updated by minimizing the loss $L(\phi_i) = \mathbb{E}[(Q_i^{\text{target}} - Q_{\phi'_i}(\mathbf{s}_t, \mathbf{a}_t))^2]$, by using gradient descent such as $\phi_i \leftarrow \phi_i - \eta^{\text{critic}} \nabla_{\phi_i} L(\phi_i)$, with $\eta^{\text{critic}} = 0.001$.

Let $J(\theta_i) = \mathbb{E}[Q_{\phi_i}(\mathbf{s}_t, \mathbf{a}_t)]_{a_i=\pi_{\phi_i}(s_i)}$ as the policy objective function which measures the performance of policy $\pi_{\theta_i}$, the actors' parameters can then be updated as $\theta_i \leftarrow \theta_i - \eta^{\text{actor}} \nabla_{\theta_i} J(\theta_i)$, with $\eta^{\text{actor}} = 0.0001$. Finally, the parameters of the target networks are updated by a soft update to ensure stable convergence such as, $\phi'_i \leftarrow \eta^{\text{soft}} \phi_i + (1 - \eta^{\text{soft}}) \phi'_i$ and $\theta'_i \leftarrow \eta^{\text{soft}} \theta_i + (1 - \eta^{\text{soft}}) \theta'_i$, with $\eta^{\text{soft}} = 0.001$.

### C. LSTM-Enhanced Resilient State Estimation

In our test environment, downlink failures may prevent the BS from transmitting the global states $R^{\text{mec}}$ and $\chi_i$ to the vehicles, thereby converting the MDP into a Partial Observable MDP (POMDP) and reducing the MADDPG effectiveness. To alleviate the absence of these global states without using outdated informations, ROLAM introduces a real time estimation approach using two LSTM networks, each trained on a sliding window of past observations to reconstruct the respective global states. LSTMs are particularly well suited to this task, as they are designed to capture the long-term dependencies and temporal patterns inherent in time series data. Thus, whenever an agent does not receive the global states, i.e. $\psi_i(t) = 0$, then its LSTMs generate the estimates $\hat{R}^{\text{mec}}$ and $\hat{\chi}_i$ from a past observation window, and then submit the estimated observation vector $\hat{s}_i(t) = [\hat{\chi}_i(t-1), d_i(t), B_i^{\text{car}}(t), \hat{R}^{\text{mec}}(t-1)]$ to actor's DNN to obtain actions.

## IV. SIMULATION RESULTS AND DISCUSSION

### A. State-of-the-Art Techniques

In this section, we aim to evaluate the performance of the ROLAM framework and we selected three competitors:

1) **A State-of-the-Art algorithm (CMADRL):** consists in the solution proposed in [6], based on a MADDPG with PHY metrics states. We adapt it to guarantee its functioning in case of downlink failure, by injecting the last known states.

2) **A Heuristic Greedy Local Strategy (HGLS):** This strategy prioritizes local task processing, adjusting the transmission power to offload any surplus load.

3) **A Heuristic Greedy Offload Strategy (HGOS):** Here, tasks are predominantly offloaded at maximum power, while the local processing power is tuned to handle remaining tasks.

### B. Simulation Setup

Our experimental framework, common to both training and evaluation, integrates all simulation parameters and the associated values directly into the system model such as task arrival rates, TDD slot durations, CPU capabilities, channel models, power budgets, and noise levels (see Section II).

<figure><img src="image_3.png" /><figcaption>Fig. 2: MADDPG Training Results</figcaption></figure>

We employ the modular simulator NedgeSimRL [13], which has been adapted to incorporate our environment model, and leverage the AgileRL framework to automatically explore and optimize multi-agent configurations. All simulations are executed on IDRIS's Jean-Zay servers to ensure access to high-performance computing resources.

Each MADDPG network is implemented as a fully connected DNN with four layers, including two hidden layers containing 400 and 300 neurons, respectively, and utilizing ReLU activation functions. The actor output, comprising two values (for $p^{\text{loc}}$ and $p^{\text{off}}$), is constrained via a sigmoid function to the interval $[0, 1]$ and then scaled by $p^{\text{loc}}_{\text{max}} = p^{\text{off}}_{\text{max}} = 2$ W. This consistent experimental setup is maintained across scenarios ranging from 3 to 8 vehicles.

### C. Training Results

**MADDPG Training:** Figure 2 shows the reward progression during the MADDPG training phase over 2000 episodes. ROLAM (blue curve), converges faster and achieves a reward that is 34.8% higher than that of the CMADRL baseline (green curve). This improvement stems from the fact that the CMADRL algorithm, lacking server load information in its observation space, optimizes offloading based solely on SINR. In contrast, our method integrates server load, enabling it to balance offloading decisions more effectively and achieve a more efficient overall policy. Similar trends were observed across selected scenarios with different numbers of vehicles (ranging from 3 to 8).

<figure><img src="image_4.png" /><figcaption>Fig. 3: LSTM Training Results</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Fig. 4: Comparing ROLAN, CMADRL, HGLS and HGOS</figcaption></figure>

**LSTM Training:** In a second phase, two LSTM networks are trained to estimate metrics $R^{\text{mec}}$ and $\chi_i$ separately. Once the learning is complete, a copy of each model is distributed to all vehicles. The training is based on a dataset of 400,000 samples generated from equ. (7) and equ. (9). After testing several configurations, the best performance was obtained with a window size of 10, a learning rate of 0.0001, a hidden layer of 32, a structure of two LSTM cells and a training over 20 epochs. The model achieve an average error of 2.7 dB for SINR estimation (ranging from -8.3 dB to 33.8 dB) and approximately 3000 bits for $R^{\text{mec}}$ prediction (Fig. 3).

## D. Performance Evaluation

Figure 4 summarizes a) the average computational cost (cf. Sec. II-F), b) the total power consumption (in Watts), and c) the mean latency (in ms) for ROLAM, CMADRL, HGLS, and HGOS as the number of vehicles increases from 3 to 8. Overall, the heuristic methods exhibit very low latency (≤ 2 ms) due to their constant high power usage; however, this comes with substantially higher power consumption—nearly twice that of CMADRL and four times that of ROLAM. Notably, HGLS uses slightly less energy than HGOS by relying more on local processing, which mitigates the adverse effects of channel instability even at full power.

Focusing on the five-vehicle scenario, a representative cluster size for MADDPG-based control, the results reveal that ROLAM reduces the computational cost by 23% and cuts total power consumption by 41% compared to the CMADRL baseline. Although this gain is accompanied by a 10% increase in mean latency (approximately 2.19 ms), the trade-off is acceptable for applications that prioritize energy savings over

ultra-low delay. Importantly, these performance improvements are achieved in conjunction with an effective downlink mechanism, with global state updates successfully reaching vehicles in 73.3% of cases on average.

These findings underscore the robustness and scalability of ROLAM in moderate-scale NOMA-VEC scenarios, where balanced resource utilization and adaptability to server load constraints are paramount. The results further suggest that, although heuristic methods can attain minimal latency, their excessive power consumption may render them less viable for energy-sensitive real-time vehicular applications.

## V. CONCLUSION

In this paper, we propose ROLAM, a resilient offloading solution for NOMA-VEC that leverages an LSTM-assisted MADDPG approach to jointly optimize resource allocation and system performance. By integrating global state information into decentralized offloading decisions, ROLAM effectively balances energy consumption and processing delay in dynamic vehicular environments. Our simulation results demonstrate that ROLAM achieves superior performance compared to existing methods. This research underscores the potential of resilient, AI-driven offloading strategies for future vehicular networks and paves the way for further exploration of adaptive resource management or real-time applications.

## REFERENCES

[1] J. Gallego-Madrid, R. Sanchez-Iborra, J. Ortiz, and J. Santa, "The role of vehicular applications in the design of future 6G infrastructures," *ICT Express*, vol. 9, no. 4, pp. 556-570, 2023.

[2] L. Liu, C. Chen, Q. Pei, S. Maharjan, and Y. Zhang, "Vehicular edge computing and networking: A survey," *Mobile networks and applications*, vol. 26, pp. 1145-1168, 2021.

[3] B. Di, L. Song, Y. Li, and Z. Han, "V2X meets NOMA: Non-orthogonal multiple access for 5g-enabled vehicular networks," *IEEE Wireless Communications*, vol. 24, no. 6, pp. 14-21, 2017.

[4] R. Dulout, L. Mendiboure, Y. Pousset, V. Deniau, and F. Launay, "Non-orthogonal multiple access for offloading in multi-access edge computing: A survey," *IEEE Access*, vol. 11, pp. 118 983-119 016, 2023.

[5] H. Zhu, Q. Wu, X.-J. Wu, Q. Fan, P. Fan, and J. Wang, "Decentralized Power Allocation for MIMO-NOMA Vehicular Edge Computing Based on Deep Reinforcement Learning," *IEEE IoT Journal*, vol. 9, 2022.

[6] Z. Chen, L. Zhang, Y. Pei, C. Jiang, and L. Yin, "NOMA-Based Multi-User Mobile Edge Computation Offloading via Cooperative Multi-Agent Deep Reinforcement Learning," *IEEE Transactions on Cognitive Communications and Networking*, vol. 8, pp. 350-364, Mar. 2022.

[7] J. F. Shortle, J. M. Thompson, D. Gross, and C. M. Harris, *Fundamentals of Queueing Theory*. John Wiley & Sons, Apr. 2018.

[8] K. Grover, A. Lim, and Q. Yang, "Jamming and anti-jamming techniques in wireless networks: a survey," *International Journal of Ad Hoc and Ubiquitous Computing*, vol. 17, p. 197, 2014.

[9] T. L. Marzetta, "Noncooperative Cellular Wireless with Unlimited Numbers of Base Station Antennas," *IEEE Transactions on Wireless Communications*, vol. 9, pp. 3590-3600, Nov. 2010.

[10] T. Sylla, L. Mendiboure, M. A. Chalouf, L. Alouache, H. Aniss, and F. Krief, "User-centric iot service placement in shared edge computing infrastructures," *SN Computer Science*, 2025.

[11] K. Toczé, N. Schmitt, U. Kargén, A. Aral, and I. Brandić, "Edge workload trace gathering and analysis for benchmarking," in *2022 IEEE 6th Int. Conf. on Fog and Edge Computing (ICFEC)*. IEEE, 2022.

[12] S. A. Alabady, M. F. Mohd Salleh, and a., "LCPC error correction code for IoT applications," *Sustainable Cities and Society*, Oct. 2018.

[13] R. Dulout, Y. Ariouich, L. Mendiboure, Y. Pousset, and V. Deniau, "NedgeSIM-RL: Simulating Smart Offloading in NOMA-VEC Environments," in *2024 20th Int. Conf. on Wireless and Mobile Computing, Networking and Communications (WiMob)*. IEEE, 2024.