OpenRaiser/Intern-Atlas huggingface dataset:
Performance Analysis of Electrical Machines Using a Hybrid Data- and Physics-Driven Model
Venue: arXiv cs.LG 2022
Year: 2022

In the design phase of an electrical machine, finite element (FE) simulation are commonly used to numerically optimize the performance. The output of the magneto-static FE simulation characterizes the electromagnetic behavior of the electrical machine. It usually includes intermediate measures such as nonlinear iron losses, electromagnetic torque, and flux values at each operating point to compute the key performance indicators (KPIs). We present a data-driven deep learning approach that replaces the computationally heavy FE calculations by a deep neural network (DNN). The DNN is trained by a large volume of stored FE data in a supervised manner. During the learning process, the network response (intermediate measures) is fed as input to a physics-based post-processing to estimate characteristic maps and KPIs. Results indicate that the predictions of intermediate measures and the subsequent computations of KPIs are close to the ground truth for new machine designs. We show that this hybrid approach yields flexibility in the simulation process. Finally, the proposed hybrid approach is quantitatively compared to existing deep neural network-based direct prediction approach of KPIs.
Artefact Retrieval: Overview of NLP Models with Knowledge Base Access
Venue: arXiv cs.CL 2022
Year: 2022

Many NLP models gain performance by having access to a knowledge base. A lot of research has been devoted to devising and improving the way the knowledge base is accessed and incorporated into the model, resulting in a number of mechanisms and pipelines. Despite the diversity of proposed mechanisms, there are patterns in the designs of such systems. In this paper, we systematically describe the typology of artefacts (items retrieved from a knowledge base), retrieval mechanisms and the way these artefacts are fused into the model. This further allows us to uncover combinations of design decisions that had not yet been tried. Most of the focus is given to language models, though we also show how question answering, fact-checking and knowledgable dialogue models fit into this system as well. Having an abstract model which can describe the architecture of specific models also helps with transferring these architectures between multiple NLP tasks.
DuVisor: a User-level Hypervisor Through Delegated Virtualization
Venue: arXiv cs.OS 2022
Year: 2022

Today's mainstream virtualization systems comprise of two cooperative components: a kernel-resident driver that accesses virtualization hardware and a user-level helper process that provides VM management and I/O virtualization. However, this virtualization architecture has intrinsic issues in both security (a large attack surface) and performance. While there is a long thread of work trying to minimize the kernel-resident driver by offloading functions to user mode, they face a fundamental tradeoff between security and performance: more offloading may reduce the kernel attack surface, yet increase the runtime ring crossings between the helper process and the driver, and thus more performance cost. This paper explores a new design called delegated virtualization, which completely separates the control plane (the kernel driver) from the data plane (the helper process) and thus eliminates the kernel driver from runtime intervention. The resulting user-level hypervisor, called DuVisor, can handle all VM operations without trapping into the kernel once the kernel driver has done the initialization. DuVisor retrofits existing hardware virtualization support with a new delegated virtualization extension to directly handle VM exits, configure virtualization registers, manage the stage-2 page table and virtual devices in user mode. We have implemented the hardware extension on an open-source RISC-V CPU and built a Rust-based hypervisor atop the hardware. Evaluation on FireSim shows that DuVisor outperforms KVM by up to 47.96\% in a variety of real-world applications and significantly reduces the attack surface.
EASY: Ensemble Augmented-Shot Y-shaped Learning: State-Of-The-Art Few-Shot Classification with Simple Ingredients
Venue: arXiv cs.LG 2022
Year: 2022

Few-shot learning aims at leveraging knowledge learned by one or more deep learning models, in order to obtain good classification performance on new problems, where only a few labeled samples per class are available. Recent years have seen a fair number of works in the field, introducing methods with numerous ingredients. A frequent problem, though, is the use of suboptimally trained models to extract knowledge, leading to interrogations on whether proposed approaches bring gains compared to using better initial models without the introduced ingredients. In this work, we propose a simple methodology, that reaches or even beats state of the art performance on multiple standardized benchmarks of the field, while adding almost no hyperparameters or parameters to those used for training the initial deep learning models on the generic dataset. This methodology offers a new baseline on which to propose (and fairly compare) new techniques or adapt existing ones.
Online AutoML: An adaptive AutoML framework for online learning
Venue: arXiv cs.LG 2022
Year: 2022

Automated Machine Learning (AutoML) has been used successfully in settings where the learning task is assumed to be static. In many real-world scenarios, however, the data distribution will evolve over time, and it is yet to be shown whether AutoML techniques can effectively design online pipelines in dynamic environments. This study aims to automate pipeline design for online learning while continuously adapting to data drift. For this purpose, we design an adaptive Online Automated Machine Learning (OAML) system, searching the complete pipeline configuration space of online learners, including preprocessing algorithms and ensembling techniques. This system combines the inherent adaptation capabilities of online learners with the fast automated pipeline (re)optimization capabilities of AutoML. Focusing on optimization techniques that can adapt to evolving objectives, we evaluate asynchronous genetic programming and asynchronous successive halving to optimize these pipelines continually. We experiment on real and artificial data streams with varying types of concept drift to test the performance and adaptation capabilities of the proposed system. The results confirm the utility of OAML over popular online learning algorithms and underscore the benefits of continuous pipeline redesign in the presence of data drift.
Evaluation of data imputation strategies in complex, deeply-phenotyped data sets: the case of the EU-AIMS Longitudinal European Autism Project
Venue: arXiv cs.LG 2022
Year: 2022

An increasing number of large-scale multi-modal research initiatives has been conducted in the typically developing population, as well as in psychiatric cohorts. Missing data is a common problem in such datasets due to the difficulty of assessing multiple measures on a large number of participants. The consequences of missing data accumulate when researchers aim to explore relationships between multiple measures. Here we aim to evaluate different imputation strategies to fill in missing values in clinical data from a large (total N=764) and deeply characterised (i.e. range of clinical and cognitive instruments administered) sample of N=453 autistic individuals and N=311 control individuals recruited as part of the EU-AIMS Longitudinal European Autism Project (LEAP) consortium. In particular we consider a total of 160 clinical measures divided in 15 overlapping subsets of participants. We use two simple but common univariate strategies, mean and median imputation, as well as a Round Robin regression approach involving four independent multivariate regression models including a linear model, Bayesian Ridge regression, as well as several non-linear models, Decision Trees, Extra Trees and K-Neighbours regression. We evaluate the models using the traditional mean square error towards removed available data, and consider in addition the KL divergence between the observed and the imputed distributions. We show that all of the multivariate approaches tested provide a substantial improvement compared to typical univariate approaches. Further, our analyses reveal that across all 15 data-subsets tested, an Extra Trees regression approach provided the best global results. This allows the selection of a unique model to impute missing data for the LEAP project and deliver a fixed set of imputed clinical data to be used by researchers working with the LEAP dataset in the future.
Pneumatic Computers for Embedded Control of Microfluidics
Venue: arXiv cs.ET 2022
Year: 2022

Alternative computing approaches that interface readily with physical systems are well suited for embedded control of those systems. We demonstrate finite state machines implemented as pneumatic circuits of microfluidic valves, and we employ these controllers to direct microfluidic liquid handling procedures such as serial dilution on the same chip. These monolithic integrated systems require only power to be supplied externally, in the form of a vacuum source. User input can be provided directly to the chip by covering pneumatic ports with a finger. State machines with up to four bits of state memory are demonstrated, and next-state combinational logic can be fully reprogrammed by changing the hole-punch pattern on a membrane in the chip. These pneumatic computers demonstrate a new framework for the embedded control of physical systems.
Learning Graph Augmentations to Learn Graph Representations
Venue: arXiv cs.LG 2022
Year: 2022

Devising augmentations for graph contrastive learning is challenging due to their irregular structure, drastic distribution shifts, and nonequivalent feature spaces across datasets. We introduce LG2AR, Learning Graph Augmentations to Learn Graph Representations, which is an end-to-end automatic graph augmentation framework that helps encoders learn generalizable representations on both node and graph levels. LG2AR consists of a probabilistic policy that learns a distribution over augmentations and a set of probabilistic augmentation heads that learn distributions over augmentation parameters. We show that LG2AR achieves state-of-the-art results on 18 out of 20 graph-level and node-level benchmarks compared to previous unsupervised models under both linear and semi-supervised evaluation protocols. The source code will be released here: https://github.com/kavehhassani/lg2ar
Deep Reinforcement Learning for Random Access in Machine-Type Communication
Venue: arXiv cs.IT 2022
Year: 2022

Random access (RA) schemes are a topic of high interest in machine-type communication (MTC). In RA protocols, backoff techniques such as exponential backoff (EB) are used to stabilize the system to avoid low throughput and excessive delays. However, these backoff techniques show varying performance for different underlying assumptions and analytical models. Therefore, finding a better transmission policy for slotted ALOHA RA is still a challenge. In this paper, we show the potential of deep reinforcement learning (DRL) for RA. We learn a transmission policy that balances between throughput and fairness. The proposed algorithm learns transmission probabilities using previous action and binary feedback signal, and it is adaptive to different traffic arrival rates. Moreover, we propose average age of packet (AoP) as a metric to measure fairness among users. Our results show that the proposed policy outperforms the baseline EB transmission schemes in terms of throughput and fairness.
A Novel Mix-normalization Method for Generalizable Multi-source Person Re-identification
Venue: arXiv cs.CV 2022
Year: 2022

Person re-identification (Re-ID) has achieved great success in the supervised scenario. However, it is difficult to directly transfer the supervised model to arbitrary unseen domains due to the model overfitting to the seen source domains. In this paper, we aim to tackle the generalizable multi-source person Re-ID task (i.e., there are multiple available source domains, and the testing domain is unseen during training) from the data augmentation perspective, thus we put forward a novel method, termed MixNorm, which consists of domain-aware mix-normalization (DMN) and domain-ware center regularization (DCR). Different from the conventional data augmentation, the proposed domain-aware mix-normalization to enhance the diversity of features during training from the normalization view of the neural network, which can effectively alleviate the model overfitting to the source domains, so as to boost the generalization capability of the model in the unseen domain. To better learn the domain-invariant model, we further develop the domain-aware center regularization to better map the produced diverse features into the same space. Extensive experiments on multiple benchmark datasets validate the effectiveness of the proposed method and show that the proposed method can outperform the state-of-the-art methods. Besides, further analysis also reveals the superiority of the proposed method.
Faithiful Embeddings for EL++ Knowledge Bases
Venue: arXiv cs.AI 2022
Year: 2022

Recently, increasing efforts are put into learning continual representations for symbolic knowledge bases (KBs). However, these approaches either only embed the data-level knowledge (ABox) or suffer from inherent limitations when dealing with concept-level knowledge (TBox), i.e., they cannot faithfully model the logical structure present in the KBs. We present BoxEL, a geometric KB embedding approach that allows for better capturing the logical structure (i.e., ABox and TBox axioms) in the description logic EL++. BoxEL models concepts in a KB as axis-parallel boxes that are suitable for modeling concept intersection, entities as points inside boxes, and relations between concepts/entities as affine transformations. We show theoretical guarantees (soundness) of BoxEL for preserving logical structure. Namely, the learned model of BoxEL embedding with loss 0 is a (logical) model of the KB. Experimental results on (plausible) subsumption reasonings and a real-world application for protein-protein prediction show that BoxEL outperforms traditional knowledge graph embedding methods as well as state-of-the-art EL++ embedding approaches.
ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields
Venue: arXiv cs.CV 2022
Year: 2022

High-resolution optical satellite sensors, combined with dense stereo algorithms, have made it possible to reconstruct 3D city models from space. However, these models are, in practice, rather noisy and tend to miss small geometric features that are clearly visible in the images. We argue that one reason for the limited quality may be a too early, heuristic reduction of the triangulated 3D point cloud to an explicit height field or surface mesh. To make full use of the point cloud and the underlying images, we introduce ImpliCity, a neural representation of the 3D scene as an implicit, continuous occupancy field, driven by learned embeddings of the point cloud and a stereo pair of ortho-photos. We show that this representation enables the extraction of high-quality DSMs: with image resolution 0.5$\,$m, ImpliCity reaches a median height error of $\approx\,$0.7$\,$m and outperforms competing methods, especially w.r.t. building reconstruction, featuring intricate roof details, smooth surfaces, and straight, regular outlines.
Bayesian Inference with Nonlinear Generative Models: Comments on Secure Learning
Venue: arXiv cs.IT 2022
Year: 2022

Unlike the classical linear model, nonlinear generative models have been addressed sparsely in the literature of statistical learning. This work aims to bringing attention to these models and their secrecy potential. To this end, we invoke the replica method to derive the asymptotic normalized cross entropy in an inverse probability problem whose generative model is described by a Gaussian random field with a generic covariance function. Our derivations further demonstrate the asymptotic statistical decoupling of the Bayesian estimator and specify the decoupled setting for a given nonlinear model. The replica solution depicts that strictly nonlinear models establish an all-or-nothing phase transition: There exists a critical load at which the optimal Bayesian inference changes from perfect to an uncorrelated learning. Based on this finding, we design a new secure coding scheme which achieves the secrecy capacity of the wiretap channel. This interesting result implies that strictly nonlinear generative models are perfectly secured without any secure coding. We justify this latter statement through the analysis of an illustrative model for perfectly secure and reliable inference.
First-Order Game Logic and Modal Mu-Calculus
Venue: arXiv cs.LO 2022
Year: 2022

This paper investigates first-order game logic and first-order modal mu-calculus, which extend their propositional modal logic counterparts with first-order modalities of interpreted effects such as variable assignments. Unlike in the propositional case, both logics are shown to have the same expressive power and their proof calculi to have the same deductive power. Both calculi are also mutually relatively complete. In the presence of differential equations, corollaries obtain usable and complete translations between differential game logic, a logic for the deductive verification of hybrid games, and the differential mu-calculus, the modal mu-calculus for hybrid systems. The differential mu-calculus is complete with respect to first-order fixpoint logic and differential game logic is complete with respect to its ODE-free fragment.
Performance Analysis of Multiple-Antenna Ambient Backscatter Systems at Finite Blocklengths
Venue: arXiv cs.IT 2022
Year: 2022

This paper analyzes the maximal achievable rate for a given blocklength and error probability over a multiple-antenna ambient backscatter channel with perfect channel state information at the receiver. The result consists of a finite blocklength channel coding achievability bound and a converse bound based on the Neyman-Pearson test and the normal approximation based on the Berry- Esseen Theorem. Numerical evaluation of these bounds shows fast convergence to the channel capacity as the blocklength increases and also proves that the channel dispersion is an accurate measure of the backoff from capacity due to finite blocklength.
Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources
Venue: arXiv cs.CL 2022
Year: 2022

In recent years, large-scale data collection efforts have prioritized the amount of data collected in order to improve the modeling capabilities of large language models. This prioritization, however, has resulted in concerns with respect to the rights of data subjects represented in data collections, particularly when considering the difficulty in interrogating these collections due to insufficient documentation and tools for analysis. Mindful of these pitfalls, we present our methodology for a documentation-first, human-centered data collection project as part of the BigScience initiative. We identified a geographically diverse set of target language groups (Arabic, Basque, Chinese, Catalan, English, French, Indic languages, Indonesian, Niger-Congo languages, Portuguese, Spanish, and Vietnamese, as well as programming languages) for which to collect metadata on potential data sources. To structure this effort, we developed our online catalogue as a supporting tool for gathering metadata through organized public hackathons. We present our development process; analyses of the resulting resource metadata, including distributions over languages, regions, and resource types; and our lessons learned in this endeavor.
Polar Coded Computing: The Role of the Scaling Exponent
Venue: arXiv cs.IT 2022
Year: 2022

We consider the problem of coded distributed computing using polar codes. The average execution time of a coded computing system is related to the error probability for transmission over the binary erasure channel in recent work by Soleymani, Jamali and Mahdavifar, where the performance of binary linear codes is investigated. In this paper, we focus on polar codes and unveil a connection between the average execution time and the scaling exponent $μ$ of the family of codes. The scaling exponent has emerged as a central object in the finite-length characterization of polar codes, and it captures the speed of convergence to capacity. In particular, we show that (i) the gap between the normalized average execution time of polar codes and that of optimal MDS codes is $O(n^{-1/μ})$, and (ii) this upper bound can be improved to roughly $O(n^{-1/2})$ by considering polar codes with large kernels. We conjecture that these bounds could be improved to $O(n^{-2/μ})$ and $O(n^{-1})$, respectively, and provide a heuristic argument as well as numerical evidence supporting this view.
LP-UIT: A Multimodal Framework for Link Prediction in Social Networks
Venue: arXiv cs.SI 2022
Year: 2022

With the rapid information explosion on online social network sites (SNSs), it becomes difficult for users to seek new friends or broaden their social networks in an efficient way. Link prediction, which can effectively conquer this problem, has thus attracted wide attention. Previous methods on link prediction fail to comprehensively capture the factors leading to new link formation: 1) few models have considered the varied impacts of users' short-term and long-term interests on link prediction. Besides, they fail to jointly model the influence from social influence and "weak links"; 2) considering that different factors should be derived from information sources of different modalities, there is a lack of effective multi-modal framework for link prediction. In this view, we propose a novel multi-modal framework for link prediction (referred as LP-UIT) which fuses a comprehensive set of features (i.e., user information and topological features) extracted from multi-modal information (i.e., textual information, graph information, and numerical information). Specifically, we adopt graph convolutional network to process the network information to capture topological features, employ natural language processing techniques (i.e., TF-IDF and word2Vec) to model users' short-term and long-term interests, and identify social influence and "weak links" from numerical features. We further use an attention mechanism to model the relationship between textual and topological features. Finally, a multi-layer perceptron (MLP) is designed to combine the representations from three modalities for link prediction. Extensive experiments on two real-world datasets demonstrate the superiority of LP-UIT over the state-of-the-art methods.