OpenRaiser/Intern-Atlas huggingface dataset:
Learning Deep Parameterized Skills from Demonstration for Re-targetable Visuomotor Control
Venue: arXiv cs.RO 2021
Year: 2021

Robots need to learn skills that can not only generalize across similar problems but also be directed to a specific goal. Previous methods either train a new skill for every different goal or do not infer the specific target in the presence of multiple goals from visual data. We introduce an end-to-end method that represents targetable visuomotor skills as a goal-parameterized neural network policy. By training on an informative subset of available goals with the associated target parameters, we are able to learn a policy that can zero-shot generalize to previously unseen goals. We evaluate our method in a representative 2D simulation of a button-grid and on both button-pressing and peg-insertion tasks on two different physical arms. We demonstrate that our model trained on 33% of the possible goals is able to generalize to more than 90% of the targets in the scene for both simulation and robot experiments. We also successfully learn a mapping from target pixel coordinates to a robot policy to complete a specified goal.
Optimizing Percentile Criterion Using Robust MDPs
Venue: arXiv cs.LG 2021
Year: 2021

We address the problem of computing reliable policies in reinforcement learning problems with limited data. In particular, we compute policies that achieve good returns with high confidence when deployed. This objective, known as the \emph{percentile criterion}, can be optimized using Robust MDPs~(RMDPs). RMDPs generalize MDPs to allow for uncertain transition probabilities chosen adversarially from given ambiguity sets. We show that the RMDP solution's sub-optimality depends on the spans of the ambiguity sets along the value function. We then propose new algorithms that minimize the span of ambiguity sets defined by weighted $L_1$ and $L_\infty$ norms. Our primary focus is on Bayesian guarantees, but we also describe how our methods apply to frequentist guarantees and derive new concentration inequalities for weighted $L_1$ and $L_\infty$ norms. Experimental results indicate that our optimized ambiguity sets improve significantly on prior construction methods.
Selective Attention Based Graph Convolutional Networks for Aspect-Level Sentiment Classification
Venue: arXiv cs.CL 2021
Year: 2021

Aspect-level sentiment classification aims to identify the sentiment polarity towards a specific aspect term in a sentence. Most current approaches mainly consider the semantic information by utilizing attention mechanisms to capture the interactions between the context and the aspect term. In this paper, we propose to employ graph convolutional networks (GCNs) on the dependency tree to learn syntax-aware representations of aspect terms. GCNs often show the best performance with two layers, and deeper GCNs do not bring additional gain due to over-smoothing problem. However, in some cases, important context words cannot be reached within two hops on the dependency tree. Therefore we design a selective attention based GCN block (SA-GCN) to find the most important context words, and directly aggregate these information into the aspect-term representation. We conduct experiments on the SemEval 2014 Task 4 datasets. Our experimental results show that our model outperforms the current state-of-the-art.
Revisiting LTE LAA: Channel Access, QoS, and Coexistence with WiFi
Venue: arXiv cs.NI 2021
Year: 2021

Network operators are looking towards LTE License Assisted Access (LAA) as a means of extending capacity by offloading traffic to unlicensed bands. However, operation in these bands requires abiding to certain coexistence rules in terms of channel access. The description of these rules in existing literature is not always in line with the latest standards. Therefore, in this paper, we clarify the operation of LAA, focusing on channel access and methods of providing Quality of Service (QoS) support. In terms of coexistence, we evaluate the impact of LAA under its various QoS settings on Wi-Fi performance in an experimental testbed. Finally, we describe the upcoming research challenges for LTE-based technologies in unlicensed bands considering the latest developments.
Diverse Video Captioning Through Latent Variable Expansion
Venue: arXiv cs.CV 2021
Year: 2021

Automatically describing video content with text description is challenging but important task, which has been attracting a lot of attention in computer vision community. Previous works mainly strive for the accuracy of the generated sentences, while ignoring the sentences diversity, which is inconsistent with human behavior. In this paper, we aim to caption each video with multiple descriptions and propose a novel framework. Concretely, for a given video, the intermediate latent variables of conventional encode-decode process are utilized as input to the conditional generative adversarial network (CGAN) with the purpose of generating diverse sentences. We adopt different Convolutional Neural Networks (CNNs) as our generator that produces descriptions conditioned on latent variables and discriminator that assesses the quality of generated sentences. Simultaneously, a novel DCE metric is designed to assess the diverse captions. We evaluate our method on the benchmark datasets, where it demonstrates its ability to generate diverse descriptions and achieves superior results against other state-of-the-art methods.
Pre-train and Learn: Preserve Global Information for Graph Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Graph neural networks (GNNs) have shown great power in learning on attributed graphs. However, it is still a challenge for GNNs to utilize information faraway from the source node. Moreover, general GNNs require graph attributes as input, so they cannot be appled to plain graphs. In the paper, we propose new models named G-GNNs (Global information for GNNs) to address the above limitations. First, the global structure and attribute features for each node are obtained via unsupervised pre-training, which preserve the global information associated to the node. Then, using the global features and the raw network attributes, we propose a parallel framework of GNNs to learn different aspects from these features. The proposed learning methods can be applied to both plain graphs and attributed graphs. Extensive experiments have shown that G-GNNs can outperform other state-of-the-art models on three standard evaluation graphs. Specially, our methods establish new benchmark records on Cora (84.31\%) and Pubmed (80.95\%) when learning on attributed graphs.
A Fast and Tight Heuristic for A* in Road Networks
Venue: arXiv cs.DS 2021
Year: 2021

We study exact, efficient and practical algorithms for route planning in large road networks. Routing applications often require integrating the current traffic situation, planning ahead with traffic predictions for the future, respecting forbidden turns, and many other features depending on the exact application. While Dijkstra's algorithm can be used to solve these problems, it is too slow for many applications. A* is a classical approach to accelerate Dijkstra's algorithm. A* can support many extended scenarios without much additional implementation complexity. However, A*'s performance depends on the availability of a good heuristic that estimates distances. Computing tight distance estimates is a challenge on its own. On road networks, shortest paths can also be quickly computed using hierarchical speedup techniques. They achieve speed and exactness but sacrifice A*'s flexibility. Extending them to certain practical applications can be hard. In this paper, we present an algorithm to efficiently extract distance estimates for A* from Contraction Hierarchies (CH), a hierarchical technique. We call our heuristic CH-Potentials. Our approach allows decoupling the supported extensions from the hierarchical speed-up technique. Additionally, we describe A* optimizations to accelerate the processing of low degree nodes, which often occur in road networks.
Spatially Coupled Generalized LDPC Codes: Asymptotic Analysis and Finite Length Scaling
Venue: arXiv cs.IT 2021
Year: 2021

Generalized low-density parity-check (GLDPC) codes are a class of LDPC codes in which the standard single parity check (SPC) constraints are replaced by constraints defined by a linear block code. These stronger constraints typically result in improved error floor performance, due to better minimum distance and trapping set properties, at a cost of some increased decoding complexity. In this paper, we study spatially coupled generalized low-density parity-check (SC-GLDPC) codes and present a comprehensive analysis of these codes, including: (1) an iterative decoding threshold analysis of SC-GLDPC code ensembles demonstrating capacity approaching thresholds via the threshold saturation effect; (2) an asymptotic analysis of the minimum distance and free distance properties of SC-GLDPC code ensembles, demonstrating that the ensembles are asymptotically good; and (3) an analysis of the finite-length scaling behavior of both GLDPC block codes and SC-GLDPC codes based on a peeling decoder (PD) operating on a binary erasure channel (BEC). Results are compared to GLDPC block codes, and the advantages and disadvantages of SC-GLDPC codes are discussed.
Multivariate Uncertainty in Deep Learning
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning has the potential to dramatically impact navigation and tracking state estimation problems critical to autonomous vehicles and robotics. Measurement uncertainties in state estimation systems based on Kalman and other Bayes filters are typically assumed to be a fixed covariance matrix. This assumption is risky, particularly for "black box" deep learning models, in which uncertainty can vary dramatically and unexpectedly. Accurate quantification of multivariate uncertainty will allow for the full potential of deep learning to be used more safely and reliably in these applications. We show how to model multivariate uncertainty for regression problems with neural networks, incorporating both aleatoric and epistemic sources of heteroscedastic uncertainty. We train a deep uncertainty covariance matrix model in two ways: directly using a multivariate Gaussian density loss function, and indirectly using end-to-end training through a Kalman filter. We experimentally show in a visual tracking problem the large impact that accurate multivariate uncertainty quantification can have on Kalman filter performance for both in-domain and out-of-domain evaluation data. We additionally show in a challenging visual odometry problem how end-to-end filter training can allow uncertainty predictions to compensate for filter weaknesses.
Rotation Invariant Point Cloud Classification: Where Local Geometry Meets Global Topology
Venue: arXiv cs.CV 2021
Year: 2021

Point cloud analysis is a fundamental task in 3D computer vision. Most previous works have conducted experiments on synthetic datasets with well-aligned data; while real-world point clouds are often not pre-aligned. How to achieve rotation invariance remains an open problem in point cloud analysis. To meet this challenge, we propose a novel approach toward achieving rotation-invariant (RI) representations by combining local geometry with global topology. In our local-global-representation (LGR)-Net, we have designed a two-branch network where one stream encodes local geometric RI features and the other encodes global topology-preserving RI features. Motivated by the observation that local geometry and global topology have different yet complementary RI responses in varying regions, two-branch RI features are fused by an innovative multi-layer perceptron (MLP) based attention module. To the best of our knowledge, this work is the first principled approach toward adaptively combining global and local information under the context of RI point cloud analysis. Extensive experiments have demonstrated that our LGR-Net achieves the state-of-the-art performance on various rotation-augmented versions of ModelNet40, ShapeNet, ScanObjectNN, and S3DIS.
Niederreiter cryptosystems using quasi-cyclic codes that resist quantum Fourier sampling
Venue: arXiv cs.CR 2021
Year: 2021

McEliece and Niederreiter cryptosystems are robust and versatile cryptosystems. These cryptosystems work with many linear error-correcting codes. They are popular these days because they can be quantum-secure. In this paper, we study the Niederreiter cryptosystem using non-binary quasi-cyclic codes. We prove, if these quasi-cyclic codes satisfy certain conditions, the corresponding Niederreiter cryptosystem is resistant to the hidden subgroup problem using weak quantum Fourier sampling. Though our work uses the weak Fourier sampling, we argue that its conclusions should remain valid for the strong Fourier sampling as well.
Fairness Violations and Mitigation under Covariate Shift
Venue: arXiv cs.LG 2021
Year: 2021

We study the problem of learning fair prediction models for unseen test sets distributed differently from the train set. Stability against changes in data distribution is an important mandate for responsible deployment of models. The domain adaptation literature addresses this concern, albeit with the notion of stability limited to that of prediction accuracy. We identify sufficient conditions under which stable models, both in terms of prediction accuracy and fairness, can be learned. Using the causal graph describing the data and the anticipated shifts, we specify an approach based on feature selection that exploits conditional independencies in the data to estimate accuracy and fairness metrics for the test set. We show that for specific fairness definitions, the resulting model satisfies a form of worst-case optimality. In context of a healthcare task, we illustrate the advantages of the approach in making more equitable decisions.
Statistical EL is ExpTime-complete
Venue: arXiv cs.LO 2021
Year: 2021

We show that the consistency problem for Statistical EL ontologies, defined by Pe{ñ}aloza and Potyka, is ExpTime-hard. Together with existing ExpTime upper bounds, we conclude ExpTime-completeness of the logic. Our proof goes via a reduction from the consistency problem for EL extended with negation of atomic concepts.
A GenEO Domain Decomposition method for Saddle Point problems
Venue: arXiv cs.DC 2021
Year: 2021

We introduce an adaptive element-based domain decomposition (DD) method for solving saddle point problems defined as a block two by two matrix. The algorithm does not require any knowledge of the constrained space. We assume that all sub matrices are sparse and that the diagonal blocks are spectrally equivalent to a sum of positive semi definite matrices. The latter assumption enables the design of adaptive coarse space for DD methods that extends the GenEO theory to saddle point problems. Numerical results on three dimensional elasticity problems for steel-rubber structures discretized by a finite element with continuous pressure are shown for up to one billion degrees of freedom.
Dependency and Span, Cross-Style Semantic Role Labeling on PropBank and NomBank
Venue: arXiv cs.CL 2021
Year: 2021

The latest developments in neural semantic role labeling (SRL) have shown great performance improvements with both the dependency and span formalisms/styles. Although the two styles share many similarities in linguistic meaning and computation, most previous studies focus on a single style. In this paper, we define a new cross-style semantic role label convention and propose a new cross-style joint optimization model designed around the most basic linguistic meaning of a semantic role, providing a solution to make the results of the two styles more comparable and allowing both formalisms of SRL to benefit from their natural connections in both linguistics and computation. Our model learns a general semantic argument structure and is capable of outputting in either style. Additionally, we propose a syntax-aided method to uniformly enhance the learning of both dependency and span representations. Experiments show that the proposed methods are effective on both span and dependency SRL benchmarks.
A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness
Venue: arXiv cs.AI 2021
Year: 2021

Existing mathematical notions of fairness fail to account for the context of decision-making. We argue that moral consideration of contextual factors is an inherently human task. So we present a framework to learn context-aware mathematical formulations of fairness by eliciting people's situated fairness assessments. Our family of fairness notions corresponds to a new interpretation of economic models of Equality of Opportunity (EOP), and it includes most existing notions of fairness as special cases. Our human-in-the-loop approach is designed to learn the appropriate parameters of the EOP family by utilizing human responses to pair-wise questions about decision subjects' circumstance and deservingness, and the harm/benefit imposed on them. We illustrate our framework in a hypothetical criminal risk assessment scenario by conducting a series of human-subject experiments on Amazon Mechanical Turk. Our work takes an important initial step toward empowering stakeholders to have a voice in the formulation of fairness for Machine Learning.
Interactive Refinement of Cross-Lingual Word Embeddings
Venue: arXiv cs.CL 2021
Year: 2021

Cross-lingual word embeddings transfer knowledge between languages: models trained on high-resource languages can predict in low-resource languages. We introduce CLIME, an interactive system to quickly refine cross-lingual word embeddings for a given classification problem. First, CLIME ranks words by their salience to the downstream task. Then, users mark similarity between keywords and their nearest neighbors in the embedding space. Finally, CLIME updates the embeddings using the annotations. We evaluate CLIME on identifying health-related text in four low-resource languages: Ilocano, Sinhalese, Tigrinya, and Uyghur. Embeddings refined by CLIME capture more nuanced word semantics and have higher test accuracy than the original embeddings. CLIME often improves accuracy faster than an active learning baseline and can be easily combined with active learning to improve results.
Style is NOT a single variable: Case Studies for Cross-Style Language Understanding
Venue: arXiv cs.CL 2021
Year: 2021

Every natural text is written in some style. Style is formed by a complex combination of different stylistic factors, including formality markers, emotions, metaphors, etc. One cannot form a complete understanding of a text without considering these factors. The factors combine and co-vary in complex ways to form styles. Studying the nature of the co-varying combinations sheds light on stylistic language in general, sometimes called cross-style language understanding. This paper provides the benchmark corpus (xSLUE) that combines existing datasets and collects a new one for sentence-level cross-style language understanding and evaluation. The benchmark contains text in 15 different styles under the proposed four theoretical groupings: figurative, personal, affective, and interpersonal groups. For valid evaluation, we collect an additional diagnostic set by annotating all 15 styles on the same text. Using xSLUE, we propose three interesting cross-style applications in classification, correlation, and generation. First, our proposed cross-style classifier trained with multiple styles together helps improve overall classification performance against individually-trained style classifiers. Second, our study shows that some styles are highly dependent on each other in human-written text. Finally, we find that combinations of some contradictive styles likely generate stylistically less appropriate text. We believe our benchmark and case studies help explore interesting future directions for cross-style research. The preprocessed datasets and code are publicly available.
Equipping SBMs with RBMs: An Explainable Approach for Analysis of Networks with Covariates
Venue: arXiv cs.SI 2021
Year: 2021

Networks with node covariates offer two advantages to community detection methods, namely, (i) exploit covariates to improve the quality of communities, and more importantly, (ii) explain the discovered communities by identifying the relative importance of different covariates in them. Recent methods have almost exclusively focused on the first point above. However, the quantitative improvements offered by them are often due to complex black-box models like deep neural networks at the expense of explainability. Approaches that focus on the second point are either domain-specific or have poor performance in practice. This paper proposes explainable, domain-independent statistical models for networks with node covariates that additionally offer good quantitative performance. Our models combine the strengths of Stochastic Block Models and Restricted Boltzmann Machines to provide interpretable insights about the communities. They support both pure and mixed community memberships. Besides providing explainability, our approach's main strength is that it does not explicitly assume a causal direction between community memberships and node covariates, making it applicable in diverse domains. We derive efficient inference procedures for our models, which can, in some cases, run in linear time in the number of nodes and edges. Experiments on several synthetic and real-world networks demonstrate that our models achieve close to state-of-the-art performance on community detection and link prediction tasks while also providing explanations for the discovered communities.
Learning Weighted Automata over Principal Ideal Domains
Venue: arXiv cs.FL 2021
Year: 2021

In this paper, we study active learning algorithms for weighted automata over a semiring. We show that a variant of Angluin's seminal \LStar\ algorithm works when the semiring is a principal ideal domain, but not for general semirings such as the natural numbers.
Rate-Regularization and Generalization in VAEs
Venue: arXiv cs.LG 2021
Year: 2021

Variational autoencoders optimize an objective that combines a reconstruction loss (the distortion) and a KL term (the rate). The rate is an upper bound on the mutual information, which is often interpreted as a regularizer that controls the degree of compression. We here examine whether inclusion of the rate also acts as an inductive bias that improves generalization. We perform rate-distortion analyses that control the strength of the rate term, the network capacity, and the difficulty of the generalization problem. Decreasing the strength of the rate paradoxically improves generalization in most settings, and reducing the mutual information typically leads to underfitting. Moreover, we show that generalization continues to improve even after the mutual information saturates, indicating that the gap on the bound (i.e. the KL divergence relative to the inference marginal) affects generalization. This suggests that the standard Gaussian prior is not an inductive bias that typically aids generalization, prompting work to understand what choices of priors improve generalization in VAEs.