Title: Improving the Exploration Process by Learning from the Past