 ] In this paper, we address the problem of generating a compact representation of 2D-lidar scans  for reinforcement learning in navigation tasks. By now only little work focuses on the compactness of the provided state, which is a necessary condition to successfully and efficiently train a navigation agent. Our approach works in three stages. First, we propose a novel preprocessing of the distance measurements and compute a local , egocentric, binary grid map based on the current range measurements. We then autoencode the local map using a variational autoencoder, where the latent space serves as state representation . An important key for a compact and , at the same time, meaningful representation is the degree of disentanglement, which describes the correlation between each latent dimension. Therefore, we finally apply state-of-the-art disentangling methods to improve the representation power . Furthermore, we investige the possibilities of incorporating time-dependent information into the latent space. In particular, we incorporate the relation of consecutive scans, especially ego-motion, by applying a memory model. We implemented our approach in python using tensorflow. Our datasets are simulated with pybullet as well as recorded using a slamtec rplidar A3. The experiments show the capability of our approach to highly compress lidar data, maintain a meaningful distribution of the latent space, and even incorporate time-depended information .
The benefit of pretrained autoencoders for reinforcement learning in comparison to training on raw observations is already known 1]. In this paper, we address the generation of a compact and information-rich state representation. In particular, we train a variational autoencoder for 2D-lidar scans to use its latent state for reinforcement learning of navigation tasks. To achieve high reconstruction power of our autoencoding pipeline, we propose an - in the context of autoencoding 2D-lidar scans - novel preprocessing into a local binary occupancy image. This has no additional requirements, neither self-localization nor robust mapping, and therefore can be applied in any setting and easily transferred from simulation in real-world. In a second stage, we show the usage of the compact state representation generated by our autoencoding pipeline in a simplistic navigation task and expose the pitfall that increased reconstruction power will always lead to an improved performance. We implemented our approach in python using tensorflow. Our datasets are simulated with pybullet as well as recorded using a slamtec rplidar A3. The experiments show the significantly improved reconstruction capabilities of our approach for 2D-lidar scans w.r.t. the state of the art. However, as we demonstrate in the experiments the impact on reinforcement learning in lidar-based navigation tasks is non-predictable when improving the latent state representation generated by an autoencoding pipeline. This is surprising and needs to be taken into account during the process of optimizing a pretrained autoencoder for reinforcement learning tasks .
A
The benefit of pretrained autoencoders for reinforcement learning in comparison to training on raw observations is already known
A
1
A
.
R
problem of generating a compact representation of
generation of a compact and information-rich state representation. In particular, we train a variational autoencoder for
A
to use its latent state
R
in
of
R
By now only little work focuses on the compactness of the provided state, which is a necessary condition to successfully and efficiently train a navigation agent. Our approach works in three stages. First, we propose a novel preprocessing of the distance measurements and compute a local , egocentric, binary grid map based on the current range measurements. We then autoencode the local map using a variational autoencoder, where the latent space serves as state representation . An important key for a compact and , at the same time, meaningful representation is the degree of disentanglement, which describes the correlation between each latent dimension. Therefore, we finally apply state-of-the-art disentangling methods to improve the representation power . Furthermore, we investige the possibilities of incorporating time-dependent information into the latent space. In particular, we incorporate the relation of consecutive scans, especially ego-motion, by applying a memory model.
To achieve high reconstruction power of our autoencoding pipeline, we propose an - in the context of autoencoding 2D-lidar scans - novel preprocessing into a local binary occupancy image. This has no additional requirements, neither self-localization nor robust mapping, and therefore can be applied in any setting and easily transferred from simulation in real-world. In a second stage, we show the usage of the compact state representation generated by our autoencoding pipeline in a simplistic navigation task and expose the pitfall that increased reconstruction power will always lead to an improved performance.
R
capability
significantly improved reconstruction capabilities
R
to highly compress lidar data, maintain a meaningful distribution of the latent space, and even incorporate time-depended information
for 2D-lidar scans w.r.t. the state of the art. However, as we demonstrate in the experiments the impact on reinforcement learning in lidar-based navigation tasks is non-predictable when improving the latent state representation generated by an autoencoding pipeline. This is surprising and needs to be taken into account during the process of optimizing a pretrained autoencoder for reinforcement learning tasks