# Deep Reinforcement Learning-Based Dynamic Pricing for Parking Solutions
## 3 Proposed Method
### Deep Reinforcement Learning-Based Dynamic Pricing (DRP-DP)
#### 3.2.2 Modules  
In this proposed solution, the dynamic pricing model is divided into different modules to perform different tasks. With the input of parking environments such as vehicle arrival rate and pricing scheme used by the target parking vendor, the model will produce accurate performance, reward results and sequence of decision making. Accuracy performance and reward results are used for the evaluation of DRL-DP, while a sequence of decision making assists the parking vendor to make a decision on price regulation. Figure 7 shows the modules and the corresponding tasks in the proposed method.  
**Parking Operator** An organisation that offers parking services is the parking operator. The capacity, pricing and location of the parking vendor will be specified. It will function as a player in this RL setting. The entity attribute such as capacity, price rate and location will refer to actual data collected from Location A and Location B.  
**Driver** The driver is an entity that requires a parking lot. The drivers can be classified as commuters (long-term), frequent visitors (medium-term) and intermittent visitors (short-term) [17]. The driver is defined by its current location, destination and preferences between price and distance (described as price-focused, balanced and distance-focused) [13]. The price and distance preference rates for the driver are normalised to a 0 to 1 interval. The higher the value, the higher the drivers' preference. For the driver to decide which parking  
Figure 7: Relationship of module interactions.  
area to park his/her vehicle, this entity will calculate the preference score for each player and opponent. A lower score of preference will be chosen as the decided parking lot for the driver. The driver will be randomly initiated with a destination and starting location inside the defined grip boundary. The distance between the driver and the parking vendor will be pre-calculated by the grid module before the driver makes the decision.  
**Grid Module** The grid module is a two-dimensional (2D) Euclidean plane to represent the location of the parking operator and driver in the city's urban area [13]. The 2D grip map refers to the geographic coordinate system using latitude and longitude. Each parking operator and driver will have their own location. The grid module consists of information about this parking area, such as road capacity, vehicle volume and direction. The grid module also calculates the distance between the driver and the parking vendor upon request from the driver.  
**Pricing Engine** The pricing engine fetches information about the parking service, such as the current pricing, the parking availability at the moment and the likelihood of the next arrival. According to the specified pricing scheme, such as arrival-time-dependent pricing (ATP), usage-aware pricing (UAP), and flat rate, the engine runs and designs new pricing for the next condition.  
**Arrival Process** SARIMAX is used to forecast the arrival of the vehicle at one-hour intervals using [21]. The forecasted arrival flow will be utilised in the RL simulation.  
**Reward Module** The reward module calculates the reward as environmental feedback of an action taken by the learning agent. The reward is calculated with the MDP and normalised to avoid bias.