  A state space representation of an environment is a classic and yet powerful
tool used by many autonomous robotic systems for efficient and often optimal
solution planning. However, designing these representations with high
performance is laborious and costly, necessitating an effective and versatile
tool for autonomous generation of state spaces for autonomous robots. We
present a novel state space exploration algorithm by extending the original
surprise-based partially-observable Markov Decision Processes (sPOMDP), and
demonstrate its effective long-term exploration planning performance in various
environments. Through extensive simulation experiments, we show the proposed
model significantly increases efficiency and scalability of the original sPOMDP
learning techniques with a range of 31-63% gain in training speed while
improving robustness in environments with less deterministic transitions. Our
results pave the way for extending sPOMDP solutions to a broader set of
environments.
