Illustrated Technical Paper: Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning
Creators: name: Arruda, Carlos E. Arruda
name: Moraes, Pedro F,
name: Agoulmine, Nazim
name: Martins, Joberto S. B.
Publisher: name: Zenodo
Year: 2021
Subjects: Publish/ Subscribe
Reinforcement Learning
SARSA
IoT
Massive IoT Traffic
Resource Allocation
Network Communications

description: This illustrated technical paper presents the slides describing the contents of the paper "Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning". The talk was presented at the 3rd International Conference on Machine Learning for Networking (MLN'2020), 24 - 26 November 2020 at Paris, France - http://www.adda-association.org/mln-2020. The illustrated technical paper format is intended to complement, enrich and subsidize the technical paper content and contains slides, complementary text and additional and/or focused bibliographic references.
description: {"references": ["Carlos E. Arruda, Pedro F. Moraes, Nazim Agoulmine, and Joberto S. B. Martins. Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning. In Proceedings of the 3rd International Conference on Machine Learning for Networking (MLN'2020), pages 1\u201320, Paris, France, November 2020.", "Pedro Francesco Moraes, Rafael F. Reale, and Joberto S. B. Martins. A Publish/Subscribe QoS-aware Framework for Massive IoT Traffic Orchestration. In Proceedings of the 6th International Workshop on ADVANCEs in ICT Infrastructures and Services (ADVANCE), pages 1\u201314, Santiago, January 2018.", "A. M. Farid, M. Alshareef, P. S. Badhesha, C. Boccaletti, N. A. A. Cacho, C.-I. Carlier, A. Corriveau, I. Khayal, B. Liner, J. S. B. Martins, F. Rahimi, R. Rossett,W. C. H. Schoonenberg, A. Stillwell, and Y.Wang. Smart City Drivers and Challenges in Urban- Mobility, Health-Care, and Interdependent Infrastructure Systems. IEEE Potentials, 40(1):11\u201316, January 2021. Conference Name: IEEE Potentials.", "A. M. Farid, M. Alshareef, P. S. Badhesha, C. Boccaletti, N. A. A. Cacho, C.-I. Carlier, A. Corriveau, I. Khayal, B. Liner, J. S. B. Martins, F. Rahimi, R. Rossetti, W. C. H. Schoonenberg, A. Stillwell, and Y. Wang. Smart City Drivers and Challenges in Energy and Water Systems. IEEE Potentials, 40(1):6\u201310, January 2021. Conference Name: IEEE Potentials.", "R. Martins da Silva Bezerra, Flavia Maristela, and Joberto Martins. On Computational Infraestruture Requirements to Smart and Autonomic Cities Framework. In IEEE International Smart Cities Conference - ISC2-2015, pages 1\u20136, Guadalajara, Mexico, January 2015. IEEE.", "Eliseu Torres, Rafael F Reale, Leobino Sampaio, and Joberto S. B. Martins. A SDN/OpenFlow Framework for Dynamic Resource Allocation based on Bandwidth Allocation Model. IEEE Latin America Transactions, 18(5):853\u2013860, April 2020.", "Joberto S. B. Martins. Towards Smart City Innovation Under the Perspective of Software-Defined Networking, Artificial Intelligence and Big Data. Revista de Tecnologia da Informa\u00e7\u00e3o e Comunica\u00e7\u00e3o, 8(2):1\u20137, October 2018.", "Jos\u00e9 Santos, Tim Wauters, Bruno Volckaert, and Filip De Turck. Resource Provisioning in Fog Computing: From Theory to Practice. Sensors (Basel, Switzerland), 19(10), May 2019.", "Xingzhou Zhang, Yifan Wang, Sidi Lu, Liangkai Liu, Lanyu Xu, and Weisong Shi. OpenEI: An Open Framework for Edge Intelligence. In 39th IEEE International Conference on Distributed Computing Systems (ICDCS), pages 1\u201312, Dallas, US, July 2019.", "M. Mukherjee, L. Shu, and D. Wang. Survey of Fog Computing: Fundamental, Network Applications, and Research Challenges. IEEE Communications Surveys Tutorials, 20(3):1826\u20131857, thirdquarter 2018.", "Raouf Boutaba, Mohammad A. Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M. Caicedo. A Comprehensive Survey on Machine Learning for Networking: Evolution, Applications and Research Opportunities. Journal of Internet Services and Applications, 9(1):16, June 2018.", "Richard S. Sutton and Andrew G. Barto. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998.", "Jaehoon Koo, Veena B. Mendiratta, Muntasir Raihan Rahman, and Anwar Walid. Deep Reinforcement Learning for Network Slicing with Heterogeneous Resource Requirements and Time Varying Traffic Dynamics. In 2019 15th International Conference on Network and Service Management, pages 1\u20135, October 2019.", "Taha Alfakih, Mohammad Mehedi Hassan, Abdu Gumaei, Claudio Savaglio, and Giancarlo Fortino. Task Offloading and Resource Allocation for Mobile Edge Computing by Deep Reinforcement Learning Based on SARSA. IEEE Access, 8:54074\u2013 54084, 2020.", "D. Cote. Using Machine Learning in Communication Networks. IEEE/OSA Journal of Optical Communications and Networking, 10(10):D100\u2013D109, October 2018.", "Oscar Mauricio Caicedo Rendon, Felipe Estrada-Solano, Raouf Boutaba, Nashid Shahriar, Mohammad Salahuddin, Noura Liman, and Sara Ayoubi. Machine Learning for Cognitive Network Management. IEEE Communications Magazine, pages 1\u20139, 2018.", "Boubakr Nour, Kashif Sharif, Fan Li, Song Yang, Hassine Moungla, and Yu Wang. ICN Publisher-Subscriber Models: Challenges and Group-based Communication. IEEE Network, 33(6):156\u2013163, November 2019.", "D. Happ and A. Wolisz. Limitations of the Pub/Sub Pattern for Cloud Based Iot and Their Implications. In Cloudification of the Internet of Things (CIoT), pages 1\u20136, Paris, November 2016. IEEE - Cloudification of the Internet of Things (CIoT).", "Kyoungho An, Aniruddha Gokhale, Sumant Tambe, and Takayuki Kuroda. Wide Area Network-scale Discovery and Data Dissemination in Data-centric Publish/Subscribe Systems. pages 1\u20132. ACM Press, 2015.", "M. Mohammadi and A. Al-Fuqaha. Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges. IEEE Communications Magazine, 56(2):94\u2013101, February 2018.", "M. Mazhar Rathore, Awais Ahmad, Anand Paul, and Seungmin Rho. Urban Planning and Building Smart Cities Based on the Internet of Things Using Big Data Analytics. Computer Networks, 101:63\u201380, June 2016.", "Joberto S. B. Martins. Towards Smart City Innovation Under the Perspective of Software-Defined Networking, Artificial Intelligence and Big Data. Revista de Tecnologia da Informa\u00e7\u00e3o e Comunica\u00e7\u00e3o, 8(2):1\u20137, October 2018.", "Diego Kreutz, Fernando M. V. Ramos, Paulo Verissimo, Christian Esteve Rothenberg, Siamak Azodolmolky, and Steve Uhlig. Software-Defined Networking: A Comprehensive Survey. Proceedings of the IEEE, 103(1):14\u201376, December 2014.", "J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu, C. Wang, and Y. Liu. A Survey of Machine Learning Techniques Applied to Software Defined Networking (SDN): Research Issues and Challenges. IEEE Communications Surveys Tutorials, 21(1):393\u2013430, Firstquarter 2019.", "L. Zhao, J. Wang, J. Liu, and N. Kato. Routing for Crowd Management in Smart Cities: A Deep Reinforcement Learning Perspective. IEEE Communications Magazine, 57(4):88\u201393, April 2019.", "M. Latah and L. Toker. Artificial intelligence enabled software-defined networking: A comprehensive overview. IET Networks, 8(2):79\u201399, 2019.", "Thomas M. Moerland, Joost Broekens, and Catholijn M. Jonker. A Framework for Reinforcement Learning and Planning. PhD thesis, TU Delft, June 2020.", "Xiaofei Liao, DequanWu, and YangWang. Dynamic Spectrum Access Based on Improved SARSA Algorithm. IOP Conference Series: Materials Science and Engineering, 768(7):072015, March 2020. Publisher: IOP Publishing.", "Lauren S. R. Sampaio, Pedro H. A. Faustini, Anderson S. Silva, Lisandro Z. Granville, and Alberto Schaeffer-Filho. Using NFV and Reinforcement Learning for Anomalies Detection and Mitigation in SDN. In 2018 IEEE Symposium on Computers and Communications (ISCC), pages 00432\u201300437, June 2018.", "J. V. Bibal Benifa and D. Dejey. RLPAS: Reinforcement Learning-based Proactive Auto-Scaler for Resource Provisioning in Cloud Environment. Mobile Networks and Applications, 24(4):1348\u20131363, August 2019.", "Ali Asghari, Mohammad Karim Sohrabi, and Farzin Yaghmaee. Task Scheduling, Resource Provisioning, and Load Balancing on Scientific Workflows Using Parallel SARSA Reinforcement Learning Agents and Genetic Algorithm. The Journal of Supercomputing, July 2020.", "Almuthanna Nassar and Yasin Yilmaz. Reinforcement Learning for Adaptive Resource Allocation in Fog RAN for IoT With Heterogeneous Latency Requirements. IEEE Access, 7:128014\u2013128025, 2019.", "Aaron Defazio and Thore Graepel. A Comparison of Learning Algorithms on the Arcade Learning Environment. arXiv:1410.8620 [cs], October 2014. arXiv: 1410.8620.", "Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, and Michael Bowling. Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. Journal of Artificial Intelligence Research, 61:523\u2013562, March 2018.", "Jen-Hung Wang, Ping-En Lu, Cheng-Shang Chang, and Duan-Shin Lee. A Reinforcement Learning Approach for the Multichannel Rendezvous Problem. In 2019 IEEE Globecom Workshops (GC Wkshps), pages 1\u20135, December 2019.", "Sridhar Mahadevan. Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results. Machine Learning, 22(1):159\u2013195, March 1996.", "Yue Wang and Shaofeng Zou. Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise. Proceedings of Machine Learning Research, 124(Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)):1\u201326, 2020.", "Kamil Ciosek, Quan Vuong, Robert Loftin, and Katja Hofmann. Better Exploration with Optimistic Actor Critic. pages 1787\u20131798, 2019.", "Pedro F. Moraes and Joberto S. B. Martins. A Pub/Sub SDN-Integrated Framework for IoT Traffic Orchestration. In Proceedings of the 3rd International Conference on Future Networks and Distributed Systems, ICFNDS '19, pages 1\u20139, 2019. Paris, France."]}