[19] N. Entezari, M. E. Shiri, and P. Moradi. A local graph clustering algorithm for discovering subgoals in reinforcement learning. In International Conference on Future Generation Communication and Networking, pages 41-50. Springer, 2010.
[20] P.-L. Bacon and D. Precup. Using label propagation for learning temporally abstract actions in reinforcement learning. In Proceedings of the Workshop on Multiagent Interaction Networks, MAIN '13, 2013.
[21] N. Taghizadeh and H. Beigy. A novel graphical approach to automatic abstraction in reinforcement learning. Robotics and Autonomous Systems, 61(8):821-835, 2013.
[22] S. J. Kazemitabar, N. Taghizadeh, and H. Beigy. A graph-theoretic approach toward autonomous skill acquisition in reinforcement learning. *Evolving Systems*, 9(3):227–244, 2018.
[23] R. Ramesh, M. Tomar, and B. Ravindran. Successor options: an option discovery framework for reinforcement learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI '19, pages 3304-3310. AAAI Press, 2019.
[24] S. Mannor, I. Menache, A. Hoze, and U. Klein. Dynamic abstraction in reinforcement learning via clustering. In Proceedings of the 21st International Conference on Machine Learning, ICML '04, pages 71-78. ACM, 2004.
[25] M. Davoodabadi and H. Beigy. A new method for discovering subgoals and constructing options in reinforcement learning. In Proceedings of the 5th Indian International Conference on Artificial Intelligence, pages 441-450, 2011.
[26] F. Shoeleh and M. Asadpour. Graph based skill acquisition and transfer learning for continuous reinforcement learning domains. Pattern Recognition Letters, 87:104-116, 2017.
[27] X. Xu, M. Yang, and G. Li. Constructing temporally extended actions through incremental community detection. Computational Intelligence and Neuroscience, 2018.
[28] M. D. Farahani and N. Mozayani. Automatic construction and evaluation of macro-actions in reinforcement learning. Applied Soft Computing, 82:105574, 2019.
[29] M. C. Machado, M. G. Bellemare, and M. Bowling. A Laplacian framework for option discovery in reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning, ICML '17, pages 2295-2304. PMLR, 2017.
[30] Y. Jinnai, J. W. Park, D. Abel, and G. Konidaris. Discovering options for exploration by minimizing cover time. In Proceedings of the 36th International Conference on Machine Learning, ICML '19, pages 3130-3139. PMLR, 2019.
[31] P.-L. Bacon, J. Harb, and D. Precup. The option-critic architecture. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
[32] R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems 12, NeurIPS '99. MIT Press, 1999.
[33] M. Riemer, M. Liu, and G. Tesauro. Learning abstract options. In Advances in Neural Information Processing Systems 31, NeurIPS '18. Curran Associates, Inc., 2018.
[34] R. Fox, S. Krishnan, I. Stoica, and K. Goldberg. Multi-level discovery of deep options. arXiv preprint:1703.08294, 2017.
[35] A. Levy, G. Konidaris, R. Platt, and K. Saenko. Learning multi-level hierarchies with hindsight. In Proceedings of the 7th International Conference on Learning Representations, ICLR '19, 2019.
[36] T. G. Dietterich. Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Researcsh, 13(1):227-303, 2000.