###### Abstract
In principle, the agent should be able to learn representations that can recover the task-related state information. Suppose the state information is covered by pixels. In that case, it should be possible to learn from pixels as efficient as from ground-truth states given the reasonable representation. There are many different SRL methods proposed for this objective, which have made remarkable progress, such as autoencoders, learning forward models, inverse dynamics and learning from robotic priors from the state characteristics [11]. Furthermore, with the tremendous development in label-efficient learning for image classification using contrastive unsupervised representations [13; 14] and data augmentation [15; 16], [PERSON] et al. [17; 18] combined contrastive learning and data augmentation techniques from computer vision with model-free RL to demonstrate significant sample-efficiency gains on standard RL benchmarks. Besides, transfer learning has improved vision-based RL  
Figure 1: The unsupervised state representation learning (SRL) process and the corresponding influence factors from high-dimensional pixels for DRL. performance between different tasks further. PAD [19] performs unsupervised policy adaption with a test-time auxiliary task (e.g., inverse dynamic prediction) to fine-tune the visual encoder of the policy network. By means of unsupervised keypoint detection [20], a clean foreground mask could be reconstructed from noise-augmented observations and keep the RL policy learning between different tasks stable and fast [21]. So far, however, there has been little discussion about the influence factors (as shown in Figure 1) and evaluation metrics of state representation learning. Most previous studies in this field only focus on improving the effectiveness of SRL by means of designing new mechanisms and auxiliary tasks. The evaluation metric is also limited to RL's performance, which means the RL agent's learning process is the only criteria to evaluate the SRL approaches in most cases. The diversity in methods and applications makes this field lack statistically relevant results to analyze the influence factors of the SRL methods, which creates the potential for misleading reporting of results and the demands for tighter standardization of experimental reporting. More importantly, reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining the progress of SRL. Therefore, this paper is set out to analyze the effects of SRL influence factors on vision-based DRL (Section 2). We summarized the criteria that could be used for evaluating SRL performance and compared the SRL performance on different kinds of RL algorithms (including value-based and policy-based methods):  
* Firstly, in order to give more statistically representative experimental results and quantitative analysis, we adopted 8 different tasks, including mobile robot navigation and robot arm manipulation environments (Section 3). The complexity of the tasks and the observations increases gradually;
* Secondly, three evaluation criteria, including the quantitative evaluation with reinforcement learning performance, are summarized to evaluate the SRL performance based on the extensive investigation of the related works in Section 4. * Thirdly, a large number of contrastive experiments were conducted using the control variates approach to investigate SRL influence factors such as hyperparameter, embedding network design, embedded dimension, regularization methods, and sample quality (Figure 1). In each part of the experimental analysis, we posed questions about major factors affecting final performance. We discovered that state embedded dimension and sample quality are crucial in the SRL model, and carefully adjusting the other illustrated factors can assist improve the SRL model's representative property to a some extent (Section 5). * Finally, we investigated the effects of these variables in reported results through a representative set of RL algorithms.