Title: Virne: A Comprehensive Benchmark for RL-based Network Resource Allocation in NFV
Primary area: datasets and benchmarks
Keywords: Network Resource Allocation, Deep Reinforcement Learning, Library, Benchmark, Network Simulation

Abstract: Resource allocation (RA) is critical to efficient service deployment in Network Function Virtualization (NFV), a transformative networking paradigm.  Recently, deep Reinforcement Learning (RL)-based methods have been showing promising potential to address this complexity. However, the field lacks a systematic benchmark for comprehensive simulation and rigorous evaluation. This gap hinders in-depth performance analysis and slows algorithm development for emerging networks, resulting in fragmented assessments. In this paper, we introduce Virne, a comprehensive benchmarking framework designed to accelerate the research and application of deep RL for NFV-RA. Virne provides customizable simulations for diverse network scenarios, including cloud, edge, and 5G environments. It features a modular and extensible implementation pipeline that integrates over 30 methods of various types. Virne also establishes a rigorous evaluation protocol that extends beyond online effectiveness to include practical perspectives such as solvability, generalizability, and scalability. Furthermore, we conduct in-depth analysis through extensive experiments to provide valuable insights into performance trade-offs for efficient implementation and offer actionable guidance for future research directions. Overall, with its capabilities of diverse simulations, rich implementations, and thorough evaluation, Virne could serve as a comprehensive benchmark for advancing NFV-RA methods and deep RL applications. The code and resources are available at https://anonymous.4open.science/r/anonymous-virne.

TL;DR: We introduce Virne, a benchmarking framework for NFV-RA that offers comprehensive simulations, implementations, and evaluations, along with insightful findings to advance research in ML for networking.