OpenRaiser/Intern-Atlas huggingface dataset:
Mean-Field Game and Reinforcement Learning MEC Resource Provisioning for SFC
Venue: arXiv cs.NI 2022
Year: 2022

In this paper, we address the resource provisioning problem for service function chaining (SFC) in terms of the placement and chaining of virtual network functions (VNFs) within a multi-access edge computing (MEC) infrastructure to reduce service delay. We consider the VNFs as the main entities of the system and propose a mean-field game (MFG) framework to model their behavior for their placement and chaining. Then, to achieve the optimal resource provisioning policy without considering the system control parameters, we reduce the proposed MFG to a Markov decision process (MDP). In this way, we leverage reinforcement learning with an actor-critic approach for MEC nodes to learn complex placement and chaining policies. Simulation results show that our proposed approach outperforms benchmark state-of-the-art approaches.
A Deep Reinforcement Learning Approach for Service Migration in MEC-enabled Vehicular Networks
Venue: arXiv cs.NI 2022
Year: 2022

Multi-access edge computing (MEC) is a key enabler to reduce the latency of vehicular network. Due to the vehicles mobility, their requested services (e.g., infotainment services) should frequently be migrated across different MEC servers to guarantee their stringent quality of service requirements. In this paper, we study the problem of service migration in a MEC-enabled vehicular network in order to minimize the total service latency and migration cost. This problem is formulated as a nonlinear integer program and is linearized to help obtaining the optimal solution using off-the-shelf solvers. Then, to obtain an efficient solution, it is modeled as a multi-agent Markov decision process and solved by leveraging deep Q learning (DQL) algorithm. The proposed DQL scheme performs a proactive services migration while ensuring their continuity under high mobility constraints. Finally, simulations results show that the proposed DQL scheme achieves close-to-optimal performance.
Decentralized Fictitious Play Converges Near a Nash Equilibrium in Near-Potential Games
Venue: arXiv cs.GT 2022
Year: 2022

We investigate convergence of decentralized fictitious play (DFP) in near-potential games, wherein agents preferences can almost be captured by a potential function. In DFP agents keep local estimates of other agents' empirical frequencies, best-respond against these estimates, and receive information over a time-varying communication network. We prove that empirical frequencies of actions generated by DFP converge around a single Nash Equilibrium (NE) assuming that there are only finitely many Nash equilibria, and the difference in utility functions resulting from unilateral deviations is close enough to the difference in the potential function values. This result assures that DFP has the same convergence properties of standard Fictitious play (FP) in near-potential games.
The Challenges of Exploration for Offline Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Offline Reinforcement Learning (ORL) enablesus to separately study the two interlinked processes of reinforcement learning: collecting informative experience and inferring optimal behaviour. The second step has been widely studied in the offline setting, but just as critical to data-efficient RL is the collection of informative data. The task-agnostic setting for data collection, where the task is not known a priori, is of particular interest due to the possibility of collecting a single dataset and using it to solve several downstream tasks as they arise. We investigate this setting via curiosity-based intrinsic motivation, a family of exploration methods which encourage the agent to explore those states or transitions it has not yet learned to model. With Explore2Offline, we propose to evaluate the quality of collected data by transferring the collected data and inferring policies with reward relabelling and standard offline RL algorithms. We evaluate a wide variety of data collection strategies, including a new exploration agent, Intrinsic Model Predictive Control (IMPC), using this scheme and demonstrate their performance on various tasks. We use this decoupled framework to strengthen intuitions about exploration and the data prerequisites for effective offline RL.
Regionalized Optimization
Venue: arXiv cs.AI 2022
Year: 2022

We propose a theoretical framework for non redundant reconstruction of a global loss from a collection of local ones under constraints given by a functor; we call this loss the regionalized loss in honor to Yedidia, Freeman, Weiss' celebrated article `Constructing free-energy approximations and generalized belief propagation algorithms' where a first example of regionalized loss, for entropy and the marginal functor, is built. We show how one can associate to these regionalized losses message passing algorithms for finding their critical points. It is a natural mathematical framework for optimization problems where there are multiple points of views on a dataset and replaces message passing algorithms as canonical ways of finding the optima of these problems. We explain how Generalized Belief propagation algorithms fall into the framework we propose and propose novel message passing algorithms for noisy channel networks.
Random Caching Design for Multi-User Multi-Antenna HetNets with Interference Nulling
Venue: arXiv cs.IT 2022
Year: 2022

The strong interference suffered by users can be a severe problem in cache-enabled networks (CENs) due to the content-centric user association mechanism. To tackle this issue, multi-antenna technology may be employed for interference management. In this paper, we consider a user-centric interference nulling (IN) scheme in two-tier multi-user multi-antenna CEN, with a hybrid most-popular and random caching policy at macro base stations (MBSs) and small base stations (SBSs) to provide file diversity. All the interfering SBSs within the IN range of a user are requested to suppress the interference at this user using zero-forcing beamforming. Using stochastic geometry analysis techniques, we derive a tractable expression for the area spectral efficiency (ASE). A lower bound on the ASE is also obtained, with which we then consider ASE maximization, by optimizing the caching policy and IN coefficient. To solve the resultant mixed integer programming problem, we design an alternating optimization algorithm to minimize the lower bound of the ASE. Our numerical results demonstrate that the proposed caching policy yields performance that is close to the optimum, and it outperforms several existing baselines.
Indicative Image Retrieval: Turning Blackbox Learning into Grey
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning became the game changer for image retrieval soon after it was introduced. It promotes the feature extraction (by representation learning) as the core of image retrieval, with the relevance/matching evaluation being degenerated into simple similarity metrics. In many applications, we need the matching evidence to be indicated rather than just have the ranked list (e.g., the locations of the target proteins/cells/lesions in medical images). It is like the matched words need to be highlighted in search engines. However, this is not easy to implement without explicit relevance/matching modeling. The deep representation learning models are not feasible because of their blackbox nature. In this paper, we revisit the importance of relevance/matching modeling in deep learning era with an indicative retrieval setting. The study shows that it is possible to skip the representation learning and model the matching evidence directly. By removing the dependency on the pre-trained models, it has avoided a lot of related issues (e.g., the domain gap between classification and retrieval, the detail-diffusion caused by convolution, and so on). More importantly, the study demonstrates that the matching can be explicitly modeled and backtracked later for generating the matching evidence indications. It can improve the explainability of deep inference. Our method obtains a best performance in literature on both Oxford-5k and Paris-6k, and sets a new record of 97.77% on Oxford-5k (97.81% on Paris-6k) without extracting any deep features.
POLYLLA: Polygonal meshing algorithm based on terminal-edge regions
Venue: arXiv cs.CG 2022
Year: 2022

This paper presents an algorithm to generate a new kind of polygonal mesh obtained from triangulations. Each polygon is built from a terminal-edge region surrounded by edges that are not the longest-edge of any of the two triangles that share them. The algorithm is termed Polylla and is divided into three phases. The first phase consists of labeling each edge of the input triangulation according to its size; the second phase builds polygons (simple or not) from terminal-edges regions using the label system; and the third phase transforms each non simple polygon into simple ones. The final mesh contains polygons with convex and non convex shape. Since Voronoi based meshes are currently the most used polygonal meshes, we compare some geometric properties of our meshes against constrained Voronoi meshes. Several experiments were run to compare the shape and size of polygons, the number of final mesh points and polygons. For the same input, Polylla meshes contain less polygons than Voronoi meshes, and the algorithm is simpler and faster than the algorithm to generate constrained Voronoi meshes. Finally, we have validated Polylla meshes by solving the Laplace equation on an L-shaped domain using the Virtual Element Method (VEM). We show that the numerical performance of the VEM using Polylla meshes and Voronoi meshes is similar.
Dynamic Temporal Reconciliation by Reinforcement learning
Venue: arXiv cs.LG 2022
Year: 2022

Planning based on long and short term time series forecasts is a common practice across many industries. In this context, temporal aggregation and reconciliation techniques have been useful in improving forecasts, reducing model uncertainty, and providing a coherent forecast across different time horizons. However, an underlying assumption spanning all these techniques is the complete availability of data across all levels of the temporal hierarchy, while this offers mathematical convenience but most of the time low frequency data is partially completed and it is not available while forecasting. On the other hand, high frequency data can significantly change in a scenario like the COVID pandemic and this change can be used to improve forecasts that will otherwise significantly diverge from long term actuals. We propose a dynamic reconciliation method whereby we formulate the problem of informing low frequency forecasts based on high frequency actuals as a Markov Decision Process (MDP) allowing for the fact that we do not have complete information about the dynamics of the process. This allows us to have the best long term estimates based on the most recent data available even if the low frequency cycles have only been partially completed. The MDP has been solved using a Time Differenced Reinforcement learning (TDRL) approach with customizable actions and improves the long terms forecasts dramatically as compared to relying solely on historical low frequency data. The result also underscores the fact that while low frequency forecasts can improve the high frequency forecasts as mentioned in the temporal reconciliation literature (based on the assumption that low frequency forecasts have lower noise to signal ratio) the high frequency forecasts can also be used to inform the low frequency forecasts.
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Venue: arXiv cs.CL 2022
Year: 2022

Pretrained general-purpose language models can achieve state-of-the-art accuracies in various natural language processing domains by adapting to downstream tasks via zero-shot, few-shot and fine-tuning techniques. Because of their success, the size of these models has increased rapidly, requiring high-performance hardware, software, and algorithmic techniques to enable training such large models. As the result of a joint effort between Microsoft and NVIDIA, we present details on the training of the largest monolithic transformer based language model, Megatron-Turing NLG 530B (MT-NLG), with 530 billion parameters. In this paper, we first focus on the infrastructure as well as the 3D parallelism methodology used to train this model using DeepSpeed and Megatron. Next, we detail the training process, the design of our training corpus, and our data curation techniques, which we believe is a key ingredient to the success of the model. Finally, we discuss various evaluation results, as well as other interesting observations and new properties exhibited by MT-NLG. We demonstrate that MT-NLG achieves superior zero-, one-, and few-shot learning accuracies on several NLP benchmarks and establishes new state-of-the-art results. We believe that our contributions will help further the development of large-scale training infrastructures, large-scale language models, and natural language generations.
Hybrid Contrastive Learning with Cluster Ensemble for Unsupervised Person Re-identification
Venue: arXiv cs.CV 2022
Year: 2022

Unsupervised person re-identification (ReID) aims to match a query image of a pedestrian to the images in gallery set without supervision labels. The most popular approaches to tackle unsupervised person ReID are usually performing a clustering algorithm to yield pseudo labels at first and then exploit the pseudo labels to train a deep neural network. However, the pseudo labels are noisy and sensitive to the hyper-parameter(s) in clustering algorithm. In this paper, we propose a Hybrid Contrastive Learning (HCL) approach for unsupervised person ReID, which is based on a hybrid between instance-level and cluster-level contrastive loss functions. Moreover, we present a Multi-Granularity Clustering Ensemble based Hybrid Contrastive Learning (MGCE-HCL) approach, which adopts a multi-granularity clustering ensemble strategy to mine priority information among the pseudo positive sample pairs and defines a priority-weighted hybrid contrastive loss for better tolerating the noises in the pseudo positive samples. We conduct extensive experiments on two benchmark datasets Market-1501 and DukeMTMC-reID. Experimental results validate the effectiveness of our proposals.
Network Selection schemes in Heterogeneous Wireless Networks
Venue: arXiv cs.NI 2022
Year: 2022

Heterogeneous Wireless Networks HWNs are combined networks made of different Radio Access Technologies RAT. Next-Generation Networks NGN will provide high bandwidth connectivity and high data throughput with smooth support for the user's QoS requirements, in this context, users with multi-interface terminals will be able to connect to different wireless technologies such as 802.16, 802.11 families and cellular families UMTS, HSPA and LTE in the same time. The idea of NGN is that users will not be tied with a contract with one single operator but, users will be able to choose the Radio Access Network RAT considering the user's QoS requested. This paper focuses on the network selection strategies and the inter technologies Handoff, we will present a description of the existed methods of network selection, we will discuss the merits and the weakness of such method and we will give our point of view.
1-2-3 Reproducibility for Quantum Software Experiments
Venue: arXiv cs.SE 2022
Year: 2022

Various fields of science face a reproducibility crisis. For quantum software engineering as an emerging field, it is therefore imminent to focus on proper reproducibility engineering from the start. Yet the provision of reproduction packages is almost universally lacking. Actionable advice on how to build such packages is rare, particularly unfortunate in a field with many contributions from researchers with backgrounds outside computer science. In this article, we argue how to rectify this deficiency by proposing a 1-2-3~approach to reproducibility engineering for quantum software experiments: Using a meta-generation mechanism, we generate DOI-safe, long-term functioning and dependency-free reproduction packages. They are designed to satisfy the requirements of professional and learned societies solely on the basis of project-specific research artefacts (source code, measurement and configuration data), and require little temporal investment by researchers. Our scheme ascertains long-term traceability even when the quantum processor itself is no longer accessible. By drastically lowering the technical bar, we foster the proliferation of reproduction packages in quantum software experiments and ease the inclusion of non-CS researchers entering the field.
Autonomous, Mobile Manipulation in a Wall-building Scenario: Team LARICS at MBZIRC 2020
Venue: arXiv cs.RO 2022
Year: 2022

In this paper we present our hardware design and control approaches for a mobile manipulation platform used in Challenge 2 of the MBZIRC 2020 competition. In this challenge, a team of UAVs and a single UGV collaborate in an autonomous, wall-building scenario, motivated by construction automation and large-scale robotic 3D printing. The robots must be able, autonomously, to detect, manipulate, and transport bricks in an unstructured, outdoor environment. Our control approach is based on a state machine that dictates which controllers are active at each stage of the Challenge. In the first stage our UGV uses visual servoing and local controllers to approach the target object without considering its orientation. The second stage consists of detecting the object's global pose using OpenCV-based processing of RGB-D image and point-cloud data, and calculating an alignment goal within a global map. The map is built with Google Cartographer and is based on onboard LIDAR, IMU, and GPS data. Motion control in the second stage is realized using the ROS Move Base package with Time-Elastic Band trajectory optimization. Visual servo algorithms guide the vehicle in local object-approach movement and the arm in manipulating bricks. To ensure a stable grasp of the brick's magnetic patch, we developed a passively-compliant, electromagnetic gripper with tactile feedback. Our fully-autonomous UGV performed well in Challenge 2 and in post-competition evaluations of its brick pick-and-place algorithms.
HEAT: Hyperedge Attention Networks
Venue: arXiv cs.LG 2022
Year: 2022

Learning from structured data is a core machine learning task. Commonly, such data is represented as graphs, which normally only consider (typed) binary relationships between pairs of nodes. This is a substantial limitation for many domains with highly-structured data. One important such domain is source code, where hypergraph-based representations can better capture the semantically rich and structured nature of code. In this work, we present HEAT, a neural model capable of representing typed and qualified hypergraphs, where each hyperedge explicitly qualifies how participating nodes contribute. It can be viewed as a generalization of both message passing neural networks and Transformers. We evaluate HEAT on knowledge base completion and on bug detection and repair using a novel hypergraph representation of programs. In both settings, it outperforms strong baselines, indicating its power and generality.
Can Wikipedia Help Offline Reinforcement Learning?
Venue: arXiv cs.LG 2022
Year: 2022

Fine-tuning reinforcement learning (RL) models has been challenging because of a lack of large scale off-the-shelf datasets as well as high variance in transferability among different environments. Recent work has looked at tackling offline RL from the perspective of sequence modeling with improved results as result of the introduction of the Transformer architecture. However, when the model is trained from scratch, it suffers from slow convergence speeds. In this paper, we look to take advantage of this formulation of reinforcement learning as sequence modeling and investigate the transferability of pre-trained sequence models on other domains (vision, language) when finetuned on offline RL tasks (control, games). To this end, we also propose techniques to improve transfer between these domains. Results show consistent performance gains in terms of both convergence speed and reward on a variety of environments, accelerating training by 3-6x and achieving state-of-the-art performance in a variety of tasks using Wikipedia-pretrained and GPT2 language models. We hope that this work not only brings light to the potentials of leveraging generic sequence modeling techniques and pre-trained models for RL, but also inspires future work on sharing knowledge between generative modeling tasks of completely different domains.
Adaptive Optimizer for Automated Hyperparameter Optimization Problem
Venue: arXiv cs.LG 2022
Year: 2022

The choices of hyperparameters have critical effects on the performance of machine learning models. In this paper, we present a general framework that is able to construct an adaptive optimizer, which automatically adjust the appropriate algorithm and parameters in the process of optimization. Examining the method of adaptive optimizer, we product an example of using genetic algorithm to construct an adaptive optimizer based on Bayesian Optimizer and compared effectiveness with original optimizer. Especially, It has great advantages in parallel optimization.
Multi-objective learner performance-based behavior algorithm with five multi-objective real-world engineering problems
Venue: arXiv cs.NE 2022
Year: 2022

In this work, a new multiobjective optimization algorithm called multiobjective learner performance-based behavior algorithm is proposed. The proposed algorithm is based on the process of transferring students from high school to college. The proposed technique produces a set of non-dominated solutions. To judge the ability and efficacy of the proposed multiobjective algorithm, it is evaluated against a group of benchmarks and five real-world engineering optimization problems. Additionally, to evaluate the proposed technique quantitatively, several most widely used metrics are applied. Moreover, the results are confirmed statistically. The proposed work is then compared with three multiobjective algorithms, which are MOWCA, NSGA-II, and MODA. Similar to the proposed technique, the other algorithms in the literature were run against the benchmarks, and the real-world engineering problems utilized in the paper. The algorithms are compared with each other employing descriptive, tabular, and graphical demonstrations. The results proved the ability of the proposed work in providing a set of non-dominated solutions, and that the algorithm outperformed the other participated algorithms in most of the cases.