# A Stock Market Decision-Making Framework Based on CMR-DQN
## 4 Experiments and Results
### Trading Experimental Setup  
The trading experiment setup includes the following key components:  
1. Processing Initialization: We conducted normalization on each input variable to ensure consistent scaling and prevent gradient explosions. This normalization resulted in a mean of 0 and a standard deviation of 1 for the normalized data, thereby enhancing stability during the training process. In the CMR-DQN framework, model initialization is a critical step to ensure network weights start training from a reasonable point. Proper initialization facilitates faster convergence and helps avoid undesirable local optima;
2. Evaluation metrics: The performance metrics for the trading experiment include several key indicators. The average annualized return rate (AAR) measures the average annual return on investment. The maximum drawdown (MDD) evaluates the largest peak-to-trough decline in investment value over a specific period. The Sharpe ratio (SP) quantifies the risk-adjusted return of the investment strategy, reflecting its efficiency. The cumulative return (CR) represents the total return on investment over the entire trading period;
3. Comparison Experiment: To objectively evaluate the strengths and weaknesses of the CMR-DQN algorithm, we set the initial capital to 200,000 and compared it with other trading methods (such as B&H and DQN). Additionally, by comparing it with Rainbow DQN and CMR-DQN without the Rainbow components (i.e., DQN-CLA), we conducted an ablation study to further validate its performance.  
During the training process of our CMR-DQN model, we utilized graphical representations to visualize different functions. Figure 5 illustrates both the variation of cumulative rewards and the fluctuation of the loss function during training. This reflects the model's progress in learning and optimizing its decision-making strategies, as well as its performance in parameter optimization and data fitting, across six distinct datasets.  
Observing these two plots enables us to understand the model's performance in reward acquisition and parameter optimization simultaneously, while also allowing for a comparative analysis of its performance across diverse datasets. These visualizations provide comprehensive monitoring of the training process, aiding in the assessment of the model's performance across various environments and guiding further adjustments in training strategies.  
By observing the results from these two charts, it is evident that the CMR-DQN model has converged in both cumulative reward and loss functions, indicating stable and promising performance during training. This convergence not only showcases the model's capability in exploring and exploiting potential returns in the stock market but also reflects its flexibility in dynamically adjusting and optimizing trading strategies. The model can adeptly adapt to evolving market conditions and adjust decisions promptly to maximize portfolio efficiency.  
Figure 5: The accumulation of rewards and the variation trend of the loss function during the training process of the CMR-DQN model on six datasets.