OpenRaiser/Intern-Atlas huggingface dataset:
Wideband Beam Tracking in THz Massive MIMO Systems
Venue: arXiv cs.IT 2021
Year: 2021

Terahertz (THz) massive multiple-input multipleoutput (MIMO) has been considered as one of the promising technologies for future 6G wireless communications. It is essential to obtain channel information by beam tracking scheme to track mobile users in THz massive MIMO systems. However, the existing beam tracking schemes designed for narrowband systems with the traditional hybrid precoding structure suffer from a severe performance loss caused by the beam split effect, and thus cannot be directly applied to wideband THz massive MIMO systems. To solve this problem, in this paper we propose a beam zooming based beam tracking scheme by considering the recently proposed delay-phase precoding structure for THz massive MIMO. Specifically, we firstly prove the beam zooming mechanism to flexibly control the angular coverage of frequencydependent beams over the whole bandwidth, which can be realized by the elaborate design of time delays in the delay-phase precoding structure. Then, based on this beam zooming mechanism, we propose to track multiple user physical directions simultaneously in each time slot by generating multiple beams. The angular coverage of these beams is flexibly zoomed to adapt to the potential variation range of the user physical direction. After several time slots, the base station is able to obtain the exact user physical direction by finding out the beam with the largest user received power. Unlike traditional schemes where only one frequency-independent beam can be usually generated by one radio-frequency chain, the proposed beam zooming based beam tracking scheme can simultaneously track multiple user physical directions by using multiple frequency-dependent beams generated by one radio-frequency chain. Theoretical analysis shows that the proposed scheme can achieve the near-optimal achievable sum-rate performance with low beam training overhead.
Towards Uncovering the Intrinsic Data Structures for Unsupervised Domain Adaptation using Structurally Regularized Deep Clustering
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised domain adaptation (UDA) is to learn classification models that make predictions for unlabeled data on a target domain, given labeled data on a source domain whose distribution diverges from the target one. Mainstream UDA methods strive to learn domain-aligned features such that classifiers trained on the source features can be readily applied to the target ones. Although impressive results have been achieved, these methods have a potential risk of damaging the intrinsic data structures of target discrimination, raising an issue of generalization particularly for UDA tasks in an inductive setting. To address this issue, we are motivated by a UDA assumption of structural similarity across domains, and propose to directly uncover the intrinsic target discrimination via constrained clustering, where we constrain the clustering solutions using structural source regularization that hinges on the very same assumption. Technically, we propose a hybrid model of Structurally Regularized Deep Clustering, which integrates the regularized discriminative clustering of target data with a generative one, and we thus term our method as H-SRDC. Our hybrid model is based on a deep clustering framework that minimizes the Kullback-Leibler divergence between the distribution of network prediction and an auxiliary one, where we impose structural regularization by learning domain-shared classifier and cluster centroids. By enriching the structural similarity assumption, we are able to extend H-SRDC for a pixel-level UDA task of semantic segmentation. We conduct extensive experiments on seven UDA benchmarks of image classification and semantic segmentation. With no explicit feature alignment, our proposed H-SRDC outperforms all the existing methods under both the inductive and transductive settings. We make our implementation codes publicly available at https://github.com/huitangtang/H-SRDC.
Discrete Signal Processing on Meet/Join Lattices
Venue: arXiv cs.IT 2021
Year: 2021

A lattice is a partially ordered set supporting a meet (or join) operation that returns the largest lower bound (smallest upper bound) of two elements. Just like graphs, lattices are a fundamental structure that occurs across domains including social data analysis, natural language processing, computational chemistry and biology, and database theory. In this paper we introduce discrete-lattice signal processing (DLSP), an SP framework for data, or signals, indexed by such lattices. We use the meet (or join) to define a shift operation and derive associated notions of filtering, Fourier basis and transform, and frequency response. We show that the spectrum of a lattice signal inherits the lattice structure of the signal domain and derive a sampling theorem. Finally, we show two prototypical applications: spectral analysis of formal concept lattices in social science and sampling and Wiener filtering of multiset lattices in combinatorial auctions. Formal concept lattices are a compressed representation of relations between objects and attributes. Since relations are equivalent to bipartite graphs and hypergraphs, DLSP offers a form of Fourier analysis for these structures.
Construction of optimal spectral methods in phase retrieval
Venue: arXiv cs.IT 2021
Year: 2021

We consider the phase retrieval problem, in which the observer wishes to recover a $n$-dimensional real or complex signal $\mathbf{X}^\star$ from the (possibly noisy) observation of $|\mathbfΦ \mathbf{X}^\star|$, in which $\mathbfΦ$ is a matrix of size $m \times n$. We consider a \emph{high-dimensional} setting where $n,m \to \infty$ with $m/n = \mathcal{O}(1)$, and a large class of (possibly correlated) random matrices $\mathbfΦ$ and observation channels. Spectral methods are a powerful tool to obtain approximate observations of the signal $\mathbf{X}^\star$ which can be then used as initialization for a subsequent algorithm, at a low computational cost. In this paper, we extend and unify previous results and approaches on spectral methods for the phase retrieval problem. More precisely, we combine the linearization of message-passing algorithms and the analysis of the \emph{Bethe Hessian}, a classical tool of statistical physics. Using this toolbox, we show how to derive optimal spectral methods for arbitrary channel noise and right-unitarily invariant matrix $\mathbfΦ$, in an automated manner (i.e. with no optimization over any hyperparameter or preprocessing function).
Bayesian Image Reconstruction using Deep Generative Models
Venue: arXiv cs.CV 2021
Year: 2021

Machine learning models are commonly trained end-to-end and in a supervised setting, using paired (input, output) data. Examples include recent super-resolution methods that train on pairs of (low-resolution, high-resolution) images. However, these end-to-end approaches require re-training every time there is a distribution shift in the inputs (e.g., night images vs daylight) or relevant latent variables (e.g., camera blur or hand motion). In this work, we leverage state-of-the-art (SOTA) generative models (here StyleGAN2) for building powerful image priors, which enable application of Bayes' theorem for many downstream reconstruction tasks. Our method, Bayesian Reconstruction through Generative Models (BRGM), uses a single pre-trained generator model to solve different image restoration tasks, i.e., super-resolution and in-painting, by combining it with different forward corruption models. We keep the weights of the generator model fixed, and reconstruct the image by estimating the Bayesian maximum a-posteriori (MAP) estimate over the input latent vector that generated the reconstructed image. We further use variational inference to approximate the posterior distribution over the latent vectors, from which we sample multiple solutions. We demonstrate BRGM on three large and diverse datasets: (i) 60,000 images from the Flick Faces High Quality dataset (ii) 240,000 chest X-rays from MIMIC III and (iii) a combined collection of 5 brain MRI datasets with 7,329 scans. Across all three datasets and without any dataset-specific hyperparameter tuning, our simple approach yields performance competitive with current task-specific state-of-the-art methods on super-resolution and in-painting, while being more generalisable and without requiring any training. Our source code and pre-trained models are available online: https://razvanmarinescu.github.io/brgm/.
A novel machine learning-based optimization algorithm (ActivO) for accelerating simulation-driven engine design
Venue: arXiv cs.LG 2021
Year: 2021

A novel design optimization approach (ActivO) that employs an ensemble of machine learning algorithms is presented. The proposed approach is a surrogate-based scheme, where the predictions of a weak leaner and a strong learner are utilized within an active learning loop. The weak learner is used to identify promising regions within the design space to explore, while the strong learner is used to determine the exact location of the optimum within promising regions. For each design iteration, exploration is done by randomly selecting evaluation points within regions where the weak learner-predicted fitness is high. The global optimum obtained by using the strong learner as a surrogate is also evaluated to enable rapid convergence once the most promising region has been identified. First, the performance of ActivO was compared against five other optimizers on a cosine mixture function with 25 local optima and one global optimum. In the second problem, the objective was to minimize indicated specific fuel consumption of a compression-ignition internal combustion (IC) engine while adhering to desired constraints associated with in-cylinder pressure and emissions. Here, the efficacy of the proposed approach is compared to that of a genetic algorithm, which is widely used within the internal combustion engine community for engine optimization, showing that ActivO reduces the number of function evaluations needed to reach the global optimum, and thereby time-to-design by 80%. Furthermore, the optimization of engine design parameters leads to savings of around 1.9% in energy consumption, while maintaining operability and acceptable pollutant emissions.
Fusing Context Into Knowledge Graph for Commonsense Question Answering
Venue: arXiv cs.CL 2021
Year: 2021

Commonsense question answering (QA) requires a model to grasp commonsense and factual knowledge to answer questions about world events. Many prior methods couple language modeling with knowledge graphs (KG). However, although a KG contains rich structural information, it lacks the context to provide a more precise understanding of the concepts. This creates a gap when fusing knowledge graphs into language modeling, especially when there is insufficient labeled data. Thus, we propose to employ external entity descriptions to provide contextual information for knowledge understanding. We retrieve descriptions of related concepts from Wiktionary and feed them as additional input to pre-trained language models. The resulting model achieves state-of-the-art result in the CommonsenseQA dataset and the best result among non-generative models in OpenBookQA.
Multi Agent Team Learning in Disaggregated Virtualized Open Radio Access Networks (O-RAN)
Venue: arXiv cs.MA 2021
Year: 2021

Starting from the Cloud Radio Access Network (C-RAN), continuing with the virtual Radio Access Network (vRAN) and most recently with Open RAN (O-RAN) initiative, Radio Access Network (RAN) architectures have significantly evolved in the past decade. In the last few years, the wireless industry has witnessed a strong trend towards disaggregated, virtualized and open RANs, with numerous tests and deployments world wide. One unique aspect that motivates this paper is the availability of new opportunities that arise from using machine learning to optimize the RAN in closed-loop, i.e. without human intervention, where the complexity of disaggregation and virtualization makes well-known Self-Organized Networking (SON) solutions inadequate. In our view, Multi-Agent Systems (MASs) with team learning, can play an essential role in the control and coordination of controllers of O-RAN, i.e. near-real-time and non-real-time RAN Intelligent Controller (RIC). In this article, we first present the state-of-the-art research in multi-agent systems and team learning, then we provide an overview of the landscape in RAN disaggregation and virtualization, as well as O-RAN which emphasizes the open interfaces introduced by the O-RAN Alliance. We present a case study for agent placement and the AI feedback required in O-RAN, and finally, we identify challenges and open issues to provide a roadmap for researchers.
Generating semantic maps through multidimensional scaling: linguistic applications and theory
Venue: arXiv cs.CL 2021
Year: 2021

This paper reports on the state-of-the-art in application of multidimensional scaling (MDS) techniques to create semantic maps in linguistic research. MDS refers to a statistical technique that represents objects (lexical items, linguistic contexts, languages, etc.) as points in a space so that close similarity between the objects corresponds to close distances between the corresponding points in the representation. We focus on the use of MDS in combination with parallel corpus data as used in research on cross-linguistic variation. We first introduce the mathematical foundations of MDS and then give an exhaustive overview of past research that employs MDS techniques in combination with parallel corpus data. We propose a set of terminology to succinctly describe the key parameters of a particular MDS application. We then show that this computational methodology is theory-neutral, i.e. it can be employed to answer research questions in a variety of linguistic theoretical frameworks. Finally, we show how this leads to two lines of future developments for MDS research in linguistics.
A Critique of the Google Apple Exposure Notification (GAEN) Framework
Venue: arXiv cs.CY 2021
Year: 2021

As a response to the COVID-19 pandemic digital contact tracing has been proposed as a tool to support the health authorities in their quest to determine who has been in close and sustained contact with a person infected by the coronavirus. In April 2020 Google and Apple released the Google Apple Exposure Notification (GAEN) framework, as a decentralised and more privacy friendly platform for contact tracing. The GAEN framework implements exposure notification mostly at the operating system layer, instead of fully at the app(lication) layer. In this paper we study the consequences of this approach. We argue that this creates a dormant functionality for mass surveillance at the operating system layer. We show how it does not technically prevent the health authorities from implementing a purely centralised form of contact tracing (even though that is the stated aim). We highlight that GAEN allows Google and Apple to dictate how contact tracing is (or rather isn't) implemented in practice by health authorities, and how it introduces the risk of function creep.
Video Deblurring by Fitting to Test Data
Venue: arXiv cs.CV 2021
Year: 2021

Motion blur in videos captured by autonomous vehicles and robots can degrade their perception capability. In this work, we present a novel approach to video deblurring by fitting a deep network to the test video. Our key observation is that some frames in a video with motion blur are much sharper than others, and thus we can transfer the texture information in those sharp frames to blurry frames. Our approach heuristically selects sharp frames from a video and then trains a convolutional neural network on these sharp frames. The trained network often absorbs enough details in the scene to perform deblurring on all the video frames. As an internal learning method, our approach has no domain gap between training and test data, which is a problematic issue for existing video deblurring approaches. The conducted experiments on real-world video data show that our model can reconstruct clearer and sharper videos than state-of-the-art video deblurring approaches. Code and data are available at https://github.com/xrenaa/Deblur-by-Fitting.
Optimising cost vs accuracy of decentralised analytics in fog computing environments
Venue: arXiv cs.DC 2021
Year: 2021

The exponential growth of devices and data at the edges of the Internet is rising scalability and privacy concerns on approaches based exclusively on remote cloud platforms. Data gravity, a fundamental concept in Fog Computing, points towards decentralisation of computation for data analysis, as a viable alternative to address those concerns. Decentralising AI tasks on several cooperative devices means identifying the optimal set of locations or Collection Points (CP for short) to use, in the continuum between full centralisation (i.e., all data on a single device) and full decentralisation (i.e., data on source locations). We propose an analytical framework able to find the optimal operating point in this continuum, linking the accuracy of the learning task with the corresponding network and computational cost for moving data and running the distributed training at the CPs. We show through simulations that the model accurately predicts the optimal trade-off, quite often an intermediate point between full centralisation and full decentralisation, showing also a significant cost saving w.r.t. both of them. Finally, the analytical model admits closed-form or numeric solutions, making it not only a performance evaluation instrument but also a design tool to configure a given distributed learning task optimally before its deployment.
LaSeSOM: A Latent and Semantic Representation Framework for Soft Object Manipulation
Venue: arXiv cs.RO 2021
Year: 2021

Soft object manipulation has recently gained popularity within the robotics community due to its potential applications in many economically important areas. Although great progress has been recently achieved in these types of tasks, most state-of-the-art methods are case-specific; They can only be used to perform a single deformation task (e.g. bending), as their shape representation algorithms typically rely on "hard-coded" features. In this paper, we present LaSeSOM, a new feedback latent representation framework for semantic soft object manipulation. Our new method introduces internal latent representation layers between low-level geometric feature extraction and high-level semantic shape analysis; This allows the identification of each compressed semantic function and the formation of a valid shape classifier from different feature extraction levels. The proposed latent framework makes soft object representation more generic (independent from the object's geometry and its mechanical properties) and scalable (it can work with 1D/2D/3D tasks). Its high-level semantic layer enables to perform (quasi) shape planning tasks with soft objects, a valuable and underexplored capability in many soft manipulation tasks. To validate this new methodology, we report a detailed experimental study with robotic manipulators.
An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Search
Venue: arXiv cs.LG 2021
Year: 2021

Deep reinforcement learning (DRL) algorithms and evolution strategies (ES) have been applied to various tasks, showing excellent performances. These have the opposite properties, with DRL having good sample efficiency and poor stability, while ES being vice versa. Recently, there have been attempts to combine these algorithms, but these methods fully rely on synchronous update scheme, making it not ideal to maximize the benefits of the parallelism in ES. To solve this challenge, asynchronous update scheme was introduced, which is capable of good time-efficiency and diverse policy exploration. In this paper, we introduce an Asynchronous Evolution Strategy-Reinforcement Learning (AES-RL) that maximizes the parallel efficiency of ES and integrates it with policy gradient methods. Specifically, we propose 1) a novel framework to merge ES and DRL asynchronously and 2) various asynchronous update methods that can take all advantages of asynchronism, ES, and DRL, which are exploration and time efficiency, stability, and sample efficiency, respectively. The proposed framework and update methods are evaluated in continuous control benchmark work, showing superior performance as well as time efficiency compared to the previous methods.
Exploiting Group-level Behavior Pattern forSession-based Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Session-based recommendation (SBR) is a challenging task, which aims to predict users' future interests based on anonymous behavior sequences. Existing methods leverage powerful representation learning approaches to encode sessions into a low-dimensional space. However, despite such achievements, all the existing studies focus on the instance-level session learning, while neglecting the group-level users' preference, which is significant to model the users' behavior. To this end, we propose a novel Repeat-aware Neural Mechanism for Session-based Recommendation (RNMSR). In RNMSR, we propose to learn the user preference from both instance-level and group-level, respectively: (i) instance-level, which employs GNNs on a similarity-based item-pairwise session graph to capture the users' preference in instance-level. (ii) group-level, which converts sessions into group-level behavior patterns to model the group-level users' preference. In RNMSR, we combine instance-level user preference and group-level user preference to model the repeat consumption of users, \ie whether users take repeated consumption and which items are preferred by users. Extensive experiments are conducted on three real-world datasets, \ie Diginetica, Yoochoose, and Nowplaying, demonstrating that the proposed method consistently achieves state-of-the-art performance in all the tests.
Neural-Swarm2: Planning and Control of Heterogeneous Multirotor Swarms using Learned Interactions
Venue: arXiv cs.RO 2021
Year: 2021

We present Neural-Swarm2, a learning-based method for motion planning and control that allows heterogeneous multirotors in a swarm to safely fly in close proximity. Such operation for drones is challenging due to complex aerodynamic interaction forces, such as downwash generated by nearby drones and ground effect. Conventional planning and control methods neglect capturing these interaction forces, resulting in sparse swarm configuration during flight. Our approach combines a physics-based nominal dynamics model with learned Deep Neural Networks (DNNs) with strong Lipschitz properties. We make use of two techniques to accurately predict the aerodynamic interactions between heterogeneous multirotors: i) spectral normalization for stability and generalization guarantees of unseen data and ii) heterogeneous deep sets for supporting any number of heterogeneous neighbors in a permutation-invariant manner without reducing expressiveness. The learned residual dynamics benefit both the proposed interaction-aware multi-robot motion planning and the nonlinear tracking control design because the learned interaction forces reduce the modelling errors. Experimental results demonstrate that Neural-Swarm2 is able to generalize to larger swarms beyond training cases and significantly outperforms a baseline nonlinear tracking controller with up to three times reduction in worst-case tracking errors.
The natural frequencies of masonry beams
Venue: arXiv cs.CE 2021
Year: 2021

The present paper aims at analytically evaluating the natural frequencies of cracked slender masonry elements. The problem is dealt with in the framework of linear perturbation, and the small oscillations of the structure are studied under loaded conditions, after the equilibrium for permanent loads has been achieved. A masonry beam element made of masonry-like material is considered, and some explicit expressions of the beam's fundamental frequency as a function of the external loads and the amplitude of imposed deformations are derived. The analytical results are validated via finite-element analysis.