OpenRaiser/Intern-Atlas huggingface dataset:
Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark
Venue: arXiv cs.LG 2021
Year: 2021

Meta and transfer learning are two successful families of approaches to few-shot learning. Despite highly related goals, state-of-the-art advances in each family are measured largely in isolation of each other. As a result of diverging evaluation norms, a direct or thorough comparison of different approaches is challenging. To bridge this gap, we perform a cross-family study of the best transfer and meta learners on both a large-scale meta-learning benchmark (Meta-Dataset, MD), and a transfer learning benchmark (Visual Task Adaptation Benchmark, VTAB). We find that, on average, large-scale transfer methods (Big Transfer, BiT) outperform competing approaches on MD, even when trained only on ImageNet. In contrast, meta-learning approaches struggle to compete on VTAB when trained and validated on MD. However, BiT is not without limitations, and pushing for scale does not improve performance on highly out-of-distribution MD tasks. In performing this study, we reveal a number of discrepancies in evaluation norms and study some of these in light of the performance gap. We hope that this work facilitates sharing of insights from each community, and accelerates progress on few-shot learning.
Some comments about CRC selection for the 5G NR specification
Venue: arXiv cs.IT 2021
Year: 2021

In this work the undetected error probability performance of the proposed in the technical specification of 5G new radio cyclic redundancy-check codes taking into consideration that they have to work for fixed interval of lengths are investigated. Codes of 6, 11 and 16 check bits with better error detection performance than the proposed in the specification ones are suggested. For 24 check bits the 3 best CRC codes are presented, but no only one best for the whole interval of lengths code was found.
Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge
Venue: arXiv cs.CL 2021
Year: 2021

Cant is important for understanding advertising, comedies and dog-whistle politics. However, computational research on cant is hindered by a lack of available datasets. In this paper, we propose a large and diverse Chinese dataset for creating and understanding cant from a computational linguistics perspective. We formulate a task for cant understanding and provide both quantitative and qualitative analysis for tested word embedding similarity and pretrained language models. Experiments suggest that such a task requires deep language understanding, common sense, and world knowledge and thus can be a good testbed for pretrained language models and help models perform better on other tasks. The code is available at https://github.com/JetRunner/dogwhistle. The data and leaderboard are available at https://competitions.codalab.org/competitions/30451.
Hierarchical compressed sensing
Venue: arXiv cs.IT 2021
Year: 2021

Compressed sensing is a paradigm within signal processing that provides the means for recovering structured signals from linear measurements in a highly efficient manner. Originally devised for the recovery of sparse signals, it has become clear that a similar methodology would also carry over to a wealth of other classes of structured signals. In this work, we provide an overview over the theory of compressed sensing for a particularly rich family of such signals, namely those of hierarchically structured signals. Examples of such signals are constituted by blocked vectors, with only few non-vanishing sparse blocks. We present recovery algorithms based on efficient hierarchical hard-thresholding. The algorithms are guaranteed to converge, in a stable fashion both with respect to measurement noise as well as to model mismatches, to the correct solution provided the measurement map acts isometrically restricted to the signal class. We then provide a series of results establishing the required condition for large classes of measurement ensembles. Building upon this machinery, we sketch practical applications of this framework in machine-type communications and quantum tomography.
The Power of Subsampling in Submodular Maximization
Venue: arXiv cs.DS 2021
Year: 2021

We propose subsampling as a unified algorithmic technique for submodular maximization in centralized and online settings. The idea is simple: independently sample elements from the ground set, and use simple combinatorial techniques (such as greedy or local search) on these sampled elements. We show that this approach leads to optimal/state-of-the-art results despite being much simpler than existing methods. In the usual offline setting, we present SampleGreedy, which obtains a $(p + 2 + o(1))$-approximation for maximizing a submodular function subject to a $p$-extendible system using $O(n + nk/p)$ evaluation and feasibility queries, where $k$ is the size of the largest feasible set. The approximation ratio improves to $p+1$ and $p$ for monotone submodular and linear objectives, respectively. In the streaming setting, we present SampleStreaming, which obtains a $(4p +2 - o(1))$-approximation for maximizing a submodular function subject to a $p$-matchoid using $O(k)$ memory and $O(km/p)$ evaluation and feasibility queries per element, where $m$ is the number of matroids defining the $p$-matchoid. The approximation ratio improves to $4p$ for monotone submodular objectives. We empirically demonstrate the effectiveness of our algorithms on video summarization, location summarization, and movie recommendation tasks.
Bayesian adversarial multi-node bandit for optimal smart grid protection against cyber attacks
Venue: arXiv cs.CR 2021
Year: 2021

The cybersecurity of smart grids has become one of key problems in developing reliable modern power and energy systems. This paper introduces a non-stationary adversarial cost with a variation constraint for smart grids and enables us to investigate the problem of optimal smart grid protection against cyber attacks in a relatively practical scenario. In particular, a Bayesian multi-node bandit (MNB) model with adversarial costs is constructed and a new regret function is defined for this model. An algorithm called Thompson-Hedge algorithm is presented to solve the problem and the superior performance of the proposed algorithm is proven in terms of the convergence rate of the regret function. The applicability of the algorithm to real smart grid scenarios is verified and the performance of the algorithm is also demonstrated by numerical examples.
Misbehavior Detection in Wi-Fi/LTE Coexistence over Unlicensed Bands
Venue: arXiv cs.NI 2021
Year: 2021

We consider the problem of fair coexistence between LTE and Wi-Fi systems in the unlicensed 5 GHz U-NII bands. We focus on the misbehavior opportunities due to the heterogeneity in channel access mechanism and the lack of a common control plane. We define selfish misbehavior strategies for the LTE that yield an unfair share of the spectrum resources. Such strategies are based on manipulating the operational parameters of the LTE-LAA standard, namely the backoff mechanism, the traffic class parameters, the clear channel access (CCA) threshold, and others. Prior methods for detecting misbehavior in homogeneous settings are not applicable in a spectrum sharing scenario because the devices of one system cannot decode the transmissions of another. We develop implicit sensing techniques that can accurately estimate the operational parameters of LTE transmissions under various topological scenarios and {\em without decoding.} These techniques apply correlation-based signal detection to infer the required information. Our techniques are validated through experiments on a USRP testbed. We further apply a statistical inference framework for determining deviations of the LTE behavior from the coexistence etiquette. By characterizing the detection and false alarm probabilities, we show that our framework yields high detection accuracy at a very low false alarm rate. Although our methods focus on detecting misbehavior of the LTE system, they can be generalized to other coexistence scenarios.
Hybrid QSS and Dynamic Extended-Term Simulation Based on Holomorphic Embedding
Venue: arXiv cs.CE 2021
Year: 2021

Power system simulations that extend over a time period of minutes, hours, or even longer are called extended-term simulations. As power systems evolve into complex systems with increasing interdependencies and richer dynamic behaviors across a wide range of timescales, extended-term simulation is needed for many power system analysis tasks (e.g., resilience analysis, renewable energy integration, cascading failures), and there is an urgent need for efficient and robust extended-term simulation approaches. The conventional approaches are insufficient for dealing with the extended-term simulation of multi-timescale processes. This paper proposes an extended-term simulation approach based on the holomorphic embedding (HE) methodology. Its accuracy and computational efficiency are backed by HE's high accuracy in event-driven simulation, larger and adaptive time steps, and flexible switching between full-dynamic and quasi-steady-state (QSS) models. We used this proposed extended-term simulation approach to evaluate bulk power system restoration plans, and it demonstrates satisfactory accuracy and efficiency in this complex simulation task.
Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data
Venue: arXiv cs.LG 2021
Year: 2021

Electronic Health Record (EHR) data has been of tremendous utility in Artificial Intelligence (AI) for healthcare such as predicting future clinical events. These tasks, however, often come with many challenges when using classical machine learning models due to a myriad of factors including class imbalance and data heterogeneity (i.e., the complex intra-class variances). To address some of these research gaps, this paper leverages the exciting contrastive learning framework and proposes a novel contrastive regularized clinical classification model. The contrastive loss is found to substantially augment EHR-based prediction: it effectively characterizes the similar/dissimilar patterns (by its "push-and-pull" form), meanwhile mitigating the highly skewed class distribution by learning more balanced feature spaces (as also echoed by recent findings). In particular, when naively exporting the contrastive learning to the EHR data, one hurdle is in generating positive samples, since EHR data is not as amendable to data augmentation as image data. To this end, we have introduced two unique positive sampling strategies specifically tailored for EHR data: a feature-based positive sampling that exploits the feature space neighborhood structure to reinforce the feature learning; and an attribute-based positive sampling that incorporates pre-generated patient similarity metrics to define the sample proximity. Both sampling approaches are designed with an awareness of unique high intra-class variance in EHR data. Our overall framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data with a total of 5,712 patients admitted to a large, urban health system. Specifically, our method reaches a high AUROC prediction score of 0.959, which outperforms other baselines and alternatives: cross-entropy(0.873) and focal loss(0.931).
A Question-answering Based Framework for Relation Extraction Validation
Venue: arXiv cs.CL 2021
Year: 2021

Relation extraction is an important task in knowledge acquisition and text understanding. Existing works mainly focus on improving relation extraction by extracting effective features or designing reasonable model structures. However, few works have focused on how to validate and correct the results generated by the existing relation extraction models. We argue that validation is an important and promising direction to further improve the performance of relation extraction. In this paper, we explore the possibility of using question answering as validation. Specifically, we propose a novel question-answering based framework to validate the results from relation extraction models. Our proposed framework can be easily applied to existing relation classifiers without any additional information. We conduct extensive experiments on the popular NYT dataset to evaluate the proposed framework, and observe consistent improvements over five strong baselines.
Optimal Control for Structurally Sparse Systems using Graphical Inference
Venue: arXiv cs.RO 2021
Year: 2021

Dynamical systems with a distributed yet interconnected structure, like multi-rigid-body robots or large-scale multi-agent systems, introduce valuable sparsity into the system dynamics that can be exploited in an optimal control setting for speeding up computation and improving numerical conditioning. Conventional approaches for solving the Optimal Control Problem (OCP) rarely capitalize on such structural sparsity, and hence suffer from a cubic computational complexity growth as the dimensionality of the system scales. In this paper, we present an OCP formulation that relies on graphical models to capture the sparsely-interconnected nature of the system dynamics. Such a representational choice allows the use of contemporary graphical inference algorithms that enable our solver to achieve a linear time complexity in the state and control dimensions as well as the time horizon. We demonstrate the numerical and computational advantages of our approach on a canonical dynamical system in simulation.
Few-Shot Meta-Learning on Point Cloud for Semantic Segmentation
Venue: arXiv cs.RO 2021
Year: 2021

The promotion of construction robots can solve the problem of human resource shortage and improve the quality of decoration. To help the construction robots obtain environmental information, we need to use 3D point cloud, which is widely used in robotics, autonomous driving, and so on. With a good understanding of environmental information, construction robots can work better. However, the dynamic changes of 3D point cloud data may bring difficulties for construction robots to understand environmental information, such as when construction robots renovate houses. The paper proposes a semantic segmentation method for point cloud based on meta-learning. The method includes a basic learning module and a meta-learning module. The basic learning module is responsible for learning data features and evaluating the model, while the meta-learning module is responsible for updating the parameters of the model and improving the model generalization ability. In our work, we pioneered the method of producing datasets for meta-learning in 3D scenes, as well as demonstrated that the Model-Agnostic Meta-Learning (MAML) algorithm can be applied to process 3D point cloud data. At the same time, experiments show that our method can allow the model to be quickly applied to new environments with a few samples. Our method has important applications.
[RE] CNN-generated images are surprisingly easy to spot...for now
Venue: arXiv cs.CV 2021
Year: 2021

This work evaluates the reproducibility of the paper "CNN-generated images are surprisingly easy to spot... for now" by Wang et al. published at CVPR 2020. The paper addresses the challenge of detecting CNN-generated imagery, which has reached the potential to even fool humans. The authors propose two methods which help an image classifier to generalize from being trained on one specific CNN to detecting imagery produced by unseen architectures, training methods, or data sets. The paper proposes two methods to help a classifier generalize: (i) utilizing different kinds of data augmentations and (ii) using a diverse data set. This report focuses on assessing if these techniques indeed help the generalization process. Furthermore, we perform additional experiments to study the limitations of the proposed techniques.
Multimodal Continuous Visual Attention Mechanisms
Venue: arXiv cs.CV 2021
Year: 2021

Visual attention mechanisms are a key component of neural network models for computer vision. By focusing on a discrete set of objects or image regions, these mechanisms identify the most relevant features and use them to build more powerful representations. Recently, continuous-domain alternatives to discrete attention models have been proposed, which exploit the continuity of images. These approaches model attention as simple unimodal densities (e.g. a Gaussian), making them less suitable to deal with images whose region of interest has a complex shape or is composed of multiple non-contiguous patches. In this paper, we introduce a new continuous attention mechanism that produces multimodal densities, in the form of mixtures of Gaussians. We use the EM algorithm to obtain a clustering of relevant regions in the image, and a description length penalty to select the number of components in the mixture. Our densities decompose as a linear combination of unimodal attention mechanisms, enabling closed-form Jacobians for the backpropagation step. Experiments on visual question answering in the VQA-v2 dataset show competitive accuracies and a selection of regions that mimics human attention more closely in VQA-HAT. We present several examples that suggest how multimodal attention maps are naturally more interpretable than their unimodal counterparts, showing the ability of our model to automatically segregate objects from ground in complex scenes.
LIFE: Lighting Invariant Flow Estimation
Venue: arXiv cs.CV 2021
Year: 2021

We tackle the problem of estimating flow between two images with large lighting variations. Recent learning-based flow estimation frameworks have shown remarkable performance on image pairs with small displacement and constant illuminations, but cannot work well on cases with large viewpoint change and lighting variations because of the lack of pixel-wise flow annotations for such cases. We observe that via the Structure-from-Motion (SfM) techniques, one can easily estimate relative camera poses between image pairs with large viewpoint change and lighting variations. We propose a novel weakly supervised framework LIFE to train a neural network for estimating accurate lighting-invariant flows between image pairs. Sparse correspondences are conventionally established via feature matching with descriptors encoding local image contents. However, local image contents are inevitably ambiguous and error-prone during the cross-image feature matching process, which hinders downstream tasks. We propose to guide feature matching with the flows predicted by LIFE, which addresses the ambiguous matching by utilizing abundant context information in the image pairs. We show that LIFE outperforms previous flow learning frameworks by large margins in challenging scenarios, consistently improves feature matching, and benefits downstream tasks.
A matrix math facility for Power ISA(TM) processors
Venue: arXiv cs.AR 2021
Year: 2021

Power ISA(TM) Version 3.1 has introduced a new family of matrix math instructions, collectively known as the Matrix-Multiply Assist (MMA) facility. The instructions in this facility implement numerical linear algebra operations on small matrices and are meant to accelerate computation-intensive kernels, such as matrix multiplication, convolution and discrete Fourier transform. These instructions have led to a power- and area-efficient implementation of a high throughput math engine in the future POWER10 processor. Performance per core is 4 times better, at constant frequency, than the previous generation POWER9 processor. We also advocate the use of compiler built-ins as the preferred way of leveraging these instructions, which we illustrate through case studies covering matrix multiplication and convolution.
GrammarTagger: A Multilingual, Minimally-Supervised Grammar Profiler for Language Education
Venue: arXiv cs.CL 2021
Year: 2021

We present GrammarTagger, an open-source grammar profiler which, given an input text, identifies grammatical features useful for language education. The model architecture enables it to learn from a small amount of texts annotated with spans and their labels, which 1) enables easier and more intuitive annotation, 2) supports overlapping spans, and 3) is less prone to error propagation, compared to complex hand-crafted rules defined on constituency/dependency parses. We show that we can bootstrap a grammar profiler model with $F_1 \approx 0.6$ from only a couple hundred sentences both in English and Chinese, which can be further boosted via learning a multilingual model. With GrammarTagger, we also build Octanove Learn, a search engine of language learning materials indexed by their reading difficulty and grammatical features. The code and pretrained models are publicly available at \url{https://github.com/octanove/grammartagger}.
The Use of Video Captioning for Fostering Physical Activity
Venue: arXiv cs.CV 2021
Year: 2021

Video Captioning is considered to be one of the most challenging problems in the field of computer vision. Video Captioning involves the combination of different deep learning models to perform object detection, action detection, and localization by processing a sequence of image frames. It is crucial to consider the sequence of actions in a video in order to generate a meaningful description of the overall action event. A reliable, accurate, and real-time video captioning method can be used in many applications. However, this paper focuses on one application: video captioning for fostering and facilitating physical activities. In broad terms, the work can be considered to be assistive technology. Lack of physical activity appears to be increasingly widespread in many nations due to many factors, the most important being the convenience that technology has provided in workplaces. The adopted sedentary lifestyle is becoming a significant public health issue. Therefore, it is essential to incorporate more physical movements into our daily lives. Tracking one's daily physical activities would offer a base for comparison with activities performed in subsequent days. With the above in mind, this paper proposes a video captioning framework that aims to describe the activities in a video and estimate a person's daily physical activity level. This framework could potentially help people trace their daily movements to reduce an inactive lifestyle's health risks. The work presented in this paper is still in its infancy. The initial steps of the application are outlined in this paper. Based on our preliminary research, this project has great merit.
The Value of Excess Supply in Spatial Matching Markets
Venue: arXiv cs.DS 2021
Year: 2021

We study dynamic matching in a spatial setting. Drivers are distributed at random on some interval. Riders arrive in some (possibly adversarial) order at randomly drawn points. The platform observes the location of the drivers, and can match newly arrived riders immediately, or can wait for more riders to arrive. Unmatched riders incur a waiting cost $c$ per period. The platform can match riders and drivers, irrevocably. The cost of matching a driver to a rider is equal to the distance between them. We quantify the value of slightly increasing supply. We prove that when there are $(1+ε)$ drivers per rider (for any $ε> 0$), the cost of matching returned by a simple greedy algorithm which pairs each arriving rider to the closest available driver is $O(\log^3(n))$, where $n$ is the number of riders. On the other hand, with equal number of drivers and riders, even the \emph{ex post} optimal matching does not have a cost less than $Θ(\sqrt{n})$. Our results shed light on the important role of (small) excess supply in spatial matching markets.