The paper investigates different representation learning methods to create a latent space for intrinsic goal generation in guided exploration algorithms. The research is in principle very important and interesting. The introduction discusses a great deal about intrinsic motivations and about goal generating algorithms. This is really great, just that the paper only focuses on a very small aspect of learning a state representation in an agent that has no intrinsic motivation other than trying to achieve random goals. I think the paper (not only the Intro) could be a bit condensed to more concentrate on the actual contribution. The contribution is that the quality of the representation and the sampling of goals is important for the exploration performance and that classical methods like ISOMap are better than Autoencoder-type methods. Also, it is written in the Conclusions (and in other places): "[..] we propose a new intrinsically Motivated goal exploration strategy....". This is not really true. There is nothing new with the intrinsically motivated selection of goals here, just that they are in another space. Also, there is no intrinsic motivation. I also think the title is misleading. The paper is in principle interesting. However, I doubt that the experimental evaluations are substantial enough for profound conclusion. Several points of critic:  - the input space was very simple in all experiments, not suitable for distinguishing between the algorithms, for instance, ISOMap typically suffers from noise and higher dimensional manifolds, etc. - only the ball/arrow was in the input image, not the robotic arm. I understand this because in phase 1 the robot would not move, but this connects to the next point:- The representation learning is only a preprocessing step requiring a magic first phase. -> Representation is not updated during exploration - The performance of any algorithm (except FI) in the Arm-Arrow task is really bad but without comment. - I am skeptical about the VAE  and RFVAE results. The difference between Gaussian sampling and the KDE is a bit alarming, as the KL in the VAE training is supposed to match the p(z) with N(0,1). Given the power of the encoder/decoder it should be possible to properly represent the simple embedded 2D/3D manifold and not just a very small part of it as suggested by Fig 10. I have a hard time believing these results. I urge you to check for any potential errors made. If there are not mistakes then this is indeed alarming. Questions: - Is it true that the robot always starts from same initial condition?! Context=Emptyset. - For ISOMap etc, you also used a 10dim embedding? Suggestion: - The main problem seems to be that some algorithms are not representing the whole input space. - an additional measure that quantifies the difference between true input distribution and reproduced input distribution could tier the algorithms apart and would measure more what seems to be relevant here. One could for instance measure the KL-divergence between the true input and the sampled (reconstructed) input (using samples and KDE or the like). - This could be evaluated on many different inputs (also those with a bit more complicated structure) without actually performing the goal finding. - BTW: I think Fig 10 is rather illustrative and should be somehow in the main part of the paper On the positive side, the paper provides lots of details in the Appendix. Also, it uses many different Representation Learning algorithms and uses measures from manifold learning to access their quality. In the related literature, in particular concerning the intrinsic motivation, I think the following papers are relevant:J. Schmidhuber, PowerPlay: training an increasingly general problem solver by continually searching for the simplest still unsolvable problem. Front. Psychol., 2013. and G. Martius, R. Der, and N. Ay. Information driven self-organization of complex robotic behaviors. PLoS ONE, 8(5):e63400, 2013. Typos and small details:p3 par2: for PCA you cited Bishop. Not critical, but either cite one the original papers or maybe remove the cite altogether p4 par-2: has multiple interests...: interests -> purposes? p4 par-1: Outcome Space to the agent is is ... Sec 2.2 par1: are rapidly mentioned... -> briefly Sec 2.3 ...Outcome Space O, we can rewrite the architecture as: and then comes the algorithm. This is a bit weird Sec 3: par1: experimental campaign -> experiments? p7: Context Space: the object was reset to a random position or always to the same position? Footnote 14: superior to -> larger than p8 par2: Exploration Ratio Ratio_expl... probably also want to add (ER) as it is later used Sec 4: slightly underneath -> slightly below p9 par1: unfinished sentence: It is worth noting that the.... one sentence later: RP architecture? RPE? Fig 3: the error of the methods (except FI) are really bad. An MSE of 1 means hardly any performance! p11 par2: for e.g. with the SAGG..... grammar? Plots in general: use bigger font sizes.