## 2 Related Studies and the Current Contribution
### Current Contribution  
This paper introduces a hybrid outer loop and inner loop framework designed to enhance the resource allocation efficiency in RIS-assisted mobile ad-hoc networks (MANET). Specifically, within the outer loop, a multi-player multi-armed bandit (MPMAB) algorithm is devised to determine the optimal selection of both the RIS and resource block (RB) for different device-to-device pairs in the MANET. Drawing inspiration from the classic multi-armed bandit (MAB) problem with a single player, as discussed in [18; 19], we formulate a novel type of decentralized multi-player multi-armed bandit problem. In this scenario, each player represents a device-to-device pair, independently selecting the RB and accessing the RIS-assisted channel without coordination among users. The combination of RB and RIS selection is treated as an 'arm' for each player. By addressing the decentralized multi-player multi-armed bandit (Dec-MPMAB) problem, each device-to-device pair in the MANET can significantly enhance the spectrum and energy efficiency. The developed framework represents a novel approach to optimizing resource allocation in RIS-assisted MANETs, offering a promising avenue towards improving the overall performance of wireless networks. Assuming that all arms exhibit rewards with independent and identically distributed properties across all users, the Upper Confidence Bound algorithm (UCB) becomes relevant. The UCB algorithm [20] strategically balances the exploration of new actions and the exploitation of previously discovered actions. This equilibrium is achieved by assigning confidence intervals to each potential action, derived from observed data. At each step, the algorithm selects the action with the highest upper confidence bound, optimizing the trade-off between exploration and exploitation. As time progresses, the algorithm dynamically adjusts its confidence intervals, thereby enhancing the overall performance. Nevertheless, conflicts may arise when two or more players simultaneously learn the optimal RIS and RB selections. To mitigate the communication costs and fully unleash the potential of RIS phase shifting, a departure from [21] is introduced. Following the players' RIS selections, communication sections inform the RIS about the number of users connected to it. Subsequently, the RIS divides its elements evenly. In the inner loop optimization process, each user optimizes only the portion assigned to them. The inner loop aims to fully harness the capabilities of the chosen RIS, and reinforcement learning (RL) [22] algorithms have gained prominence as an effective approach to adaptively controlling RIS elements. Building on this, recent research endeavors have explored innovative applications of RL algorithms to optimize wireless communications in RIS contexts. In this study, a Twin Delayed Deep Deterministic Policy Gradient (TD3) framework is employed for inner-loop resource allocation to determine the optimal energy efficiency for each device-to-device (D2D) pair. This is achieved through controlling the phase shifting of the RIS and power allocation of the D2D pair's transmitter, considering RB selection, RIS selection, and the provided partition information. The primary contributions of this paper are outlined as follows. * **It formulates a time-varying and uncertain wireless communication environment to address dynamic resource allocation in RIS-assisted mobile ad-hoc networks (MANET). ** A model has been constructed to depict the dynamic resource allocation system within a multi-mobile RIS-assisted ad-hoc wireless network. * **The optimization problems encompassing RIS selection, spectrum allocation, phase shifting control, and power allocation in both the inner and outer networks have been formulated to maximize the network capacity while ensuring the quality of service (QoS) requirements of mobile devices. ** As solving mixed-integer and non-convex optimization problems poses challenges, we reframe the issue as a multi-agent reinforcement learning problem.