## 1 Introduction  
With the increasing data traffic in wireless and vehicular networks, it has become difficult to meet data traffic requirements. To this end, cellular network architectures have been researched as a potential solution. In traditional architectures, one base station (BS) is deployed in one cell to provide voice and data services. To improve the bandwidth for 5G cellular networks, many architectures have been studied including heterogeneous network (HetNet), cloud radio access network (C-RAN), and heterogeneous cloud radio access network (H-CRAN).  
In HetNet, one macro-cell and multi-small cells are overlapped to satisfy the rapidly growing data traffic requirements. The small cells cover a smaller range than the macro cell and are deployed close to users to provide a higher data rate. However, small cells also consume more system energy at BSs than traditional network architectures. To minimize the system energy consumption, the BS structure is divided into baseband unit (BBU) and remote radio head (RRH) in C-RAN. The BBUs that process signals are centralized in a BBU pool and RRHs that serve as radio-frequency transceivers are distributed in the cells. However, it also causes serious interference because of the dense deployment. The H-CRAN has been developed to utilize the advantage of both HetNet and C-RAN [1]. In one cell of H-CRAN, multiple small RRHs are connected to the BBU pool through fronthaul links and a macro BS is connected to the BBU pool through the backhaul link. The macro BS functions only as a voice service, and RRHs function only as wireless data services. Hence, the BBU pool can manage and allocate communication resources efficiently.  
Device-to-device (D2D) communication has also been developed to satisfy the high data rate requirements. With D2D communication, nearby wireless devices can communicate directly without RRHs, reducing the system energy consumption. However, the rapidly varying channel condition owing to high mobility in the vehicular environment makes it difficult to collect instantaneous channel state information (CSI) at RRH. Unlike common wireless networks, data traffic in vehicular networks can have regularities in the spatial--temporal dimension as a result of the periodicity in urban traffic flow [2]. Hence, considerable research has been conducted on vehicle-to-everything (V2X) communication to improve road safety and to ensure reliable and low latency services [3; 4; 5; 6]. Two communication modes in cellular V2X, vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) have been modified to accommodate various vehicular applications.  
In this paper, we propose a deep reinforcement learning (DRL) based resource allocation algorithm with a clustering method in cellular V2X communication. In our algorithm, neighboring RRHs are grouped to reduce signaling overhead in DRL deployments. Vehicles are clustered to collect periodical messages and request resource allocation to minimize system energy consumption. The DRL based resource allocation learns an optimal allocation strategy to maximize the sum of achievable data rates. As there is a trade-off relationship between the energy consumption and achievable data rate, it is important to identify the strategy that optimizes both objectives. Hence, the main goal of our proposed algorithm is to maximize the system energy efficiency while ensuring quality of service (QoS) and reliability.  
The remainder of this paper is organized as follows. Section 2 provides an overview of related works. In Section 3, we describe the system model and formulate the problem. Section 4 presents the proposed algorithm, which consists of RRH grouping, vehicle clustering, and DRL based resource allocation. A performance evaluation and discussion on our proposed algorithm are presented in Section 5. Finally, we present the conclusion in Section 6.