OpenRaiser/Intern-Atlas huggingface dataset:
Neural Causal Models for Counterfactual Identification and Estimation
Venue: arXiv cs.LG 2022
Year: 2022

Evaluating hypothetical statements about how the world would be had a different course of action been taken is arguably one key capability expected from modern AI systems. Counterfactual reasoning underpins discussions in fairness, the determination of blame and responsibility, credit assignment, and regret. In this paper, we study the evaluation of counterfactual statements through neural models. Specifically, we tackle two causal problems required to make such evaluations, i.e., counterfactual identification and estimation from an arbitrary combination of observational and experimental data. First, we show that neural causal models (NCMs) are expressive enough and encode the structural constraints necessary for performing counterfactual reasoning. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions. We show that this algorithm is sound and complete for deciding counterfactual identification in general settings. Third, considering the practical implications of these results, we introduce a new strategy for modeling NCMs using generative adversarial networks. Simulations corroborate with the proposed methodology.
Leveraging Industry 4.0 -- Deep Learning, Surrogate Model and Transfer Learning with Uncertainty Quantification Incorporated into Digital Twin for Nuclear System
Venue: arXiv cs.LG 2022
Year: 2022

Industry 4.0 targets the conversion of the traditional industries into intelligent ones through technological revolution. This revolution is only possible through innovation, optimization, interconnection, and rapid decision-making capability. Numerical models are believed to be the key components of Industry 4.0, facilitating quick decision-making through simulations instead of costly experiments. However, numerical investigation of precise, high-fidelity models for optimization or decision-making is usually time-consuming and computationally expensive. In such instances, data-driven surrogate models are excellent substitutes for fast computational analysis and the probabilistic prediction of the output parameter for new input parameters. The emergence of Internet of Things (IoT) and Machine Learning (ML) has made the concept of surrogate modeling even more viable. However, these surrogate models contain intrinsic uncertainties, originate from modeling defects, or both. These uncertainties, if not quantified and minimized, can produce a skewed result. Therefore, proper implementation of uncertainty quantification techniques is crucial during optimization, cost reduction, or safety enhancement processes analysis. This chapter begins with a brief overview of the concept of surrogate modeling, transfer learning, IoT and digital twins. After that, a detailed overview of uncertainties, uncertainty quantification frameworks, and specifics of uncertainty quantification methodologies for a surrogate model linked to a digital twin is presented. Finally, the use of uncertainty quantification approaches in the nuclear industry has been addressed.
Safety-Critical Adaptation in Self-Adaptive Systems
Venue: arXiv cs.SE 2022
Year: 2022

Modern systems are designed to operate in increasingly variable and uncertain environments. Not only are these environments complex, in the sense that they contain a tremendous number of variables, but they also change over time. Systems must be able to adjust their behaviour at run-time to manage these uncertainties. These self-adaptive systems have been studied extensively. This paper proposes a definition of a safety-critical self-adaptive system and then describes a taxonomy for classifying adaptations into different types based on their impact on the system's safety and the system's safety case. The taxonomy expresses criteria for classification and then describes specific criteria that the safety case for a self-adaptive system must satisfy, depending on the type of adaptations performed. Each type in the taxonomy is illustrated using the example of a safety-critical self-adaptive water heating system.
Institutional Foundations of Adaptive Planning: Exploration of Flood Planning in the Lower Rio Grande Valley, Texas, USA
Venue: arXiv cs.CL 2022
Year: 2022

Adaptive planning is ideally suited for the deep uncertainties presented by climate change. While there is a robust scholarship on the theory and methods of adaptive planning, this has largely neglected how adaptive planning is affected by existing planning institutions and how to move forward within the constraints of traditional planning organizations. This study asks: How do existing traditional planning institutions support adaptive planning? We explore this for flood planning in the Lower Rio Grande Valley of Texas, United States. We draw on county hazard plan and regional flood plan documents as well as transcripts of regional flood planning meetings to explore the emergent topics of these institutional outputs. Using Natural Language Processing to analyze this large amount of text, we find that hazard plans and discussions developing these plans are largely lacking an adaptive approach.
Adversarial Robustness of Representation Learning for Knowledge Graphs
Venue: arXiv cs.LG 2022
Year: 2022

Knowledge graphs represent factual knowledge about the world as relationships between concepts and are critical for intelligent decision making in enterprise applications. New knowledge is inferred from the existing facts in the knowledge graphs by encoding the concepts and relations into low-dimensional feature vector representations. The most effective representations for this task, called Knowledge Graph Embeddings (KGE), are learned through neural network architectures. Due to their impressive predictive performance, they are increasingly used in high-impact domains like healthcare, finance and education. However, are the black-box KGE models adversarially robust for use in domains with high stakes? This thesis argues that state-of-the-art KGE models are vulnerable to data poisoning attacks, that is, their predictive performance can be degraded by systematically crafted perturbations to the training knowledge graph. To support this argument, two novel data poisoning attacks are proposed that craft input deletions or additions at training time to subvert the learned model's performance at inference time. These adversarial attacks target the task of predicting the missing facts in knowledge graphs using KGE models, and the evaluation shows that the simpler attacks are competitive with or outperform the computationally expensive ones. The thesis contributions not only highlight and provide an opportunity to fix the security vulnerabilities of KGE models, but also help to understand the black-box predictive behaviour of KGE models.
zkBridge: Trustless Cross-chain Bridges Made Practical
Venue: arXiv cs.CR 2022
Year: 2022

Blockchains have seen growing traction with cryptocurrencies reaching a market cap of over 1 trillion dollars, major institution investors taking interests, and global impacts on governments, businesses, and individuals. Also growing significantly is the heterogeneity of the ecosystem where a variety of blockchains co-exist. Cross-chain bridge is a necessary building block in this multi-chain ecosystem. Existing solutions, however, either suffer from performance issues or rely on trust assumptions of committees that significantly lower the security. Recurring attacks against bridges have cost users more than 1.5 billion USD. In this paper, we introduce zkBridge, an efficient cross-chain bridge that guarantees strong security without external trust assumptions. With succinct proofs, zkBridge not only guarantees correctness, but also significantly reduces on-chain verification cost. We propose novel succinct proof protocols that are orders-of-magnitude faster than existing solutions for workload in zkBridge. With a modular design, zkBridge enables a broad spectrum of use cases and capabilities, including message passing, token transferring, and other computational logic operating on state changes from different chains. To demonstrate the practicality of zkBridge, we implemented a prototype bridge from Cosmos to Ethereum, a particularly challenging direction that involves large proof circuits that existing systems cannot efficiently handle. Our evaluation shows that zkBridge achieves practical performance: proof generation takes less than 20 seconds, while verifying proofs on-chain costs less than 230K gas. For completeness, we also implemented and evaluated the direction from Ethereum to other EVM-compatible chains (such as BSC) which involves smaller circuits and incurs much less overhead.
Offline Handwritten Amharic Character Recognition Using Few-shot Learning
Venue: arXiv cs.CV 2022
Year: 2022

Few-shot learning is an important, but challenging problem of machine learning aimed at learning from only fewer labeled training examples. It has become an active area of research due to deep learning requiring huge amounts of labeled dataset, which is not feasible in the real world. Learning from a few examples is also an important attempt towards learning like humans. Few-shot learning has proven a very good promise in different areas of machine learning applications, particularly in image classification. As it is a recent technique, most researchers focus on understanding and solving the issues related to its concept by focusing only on common image datasets like Mini-ImageNet and Omniglot. Few-shot learning also opens an opportunity to address low resource languages like Amharic. In this study, offline handwritten Amharic character recognition using few-shot learning is addressed. Particularly, prototypical networks, the popular and simpler type of few-shot learning, is implemented as a baseline. Using the opportunities explored in the nature of Amharic alphabet having row-wise and column-wise similarities, a novel way of augmenting the training episodes is proposed. The experimental results show that the proposed method outperformed the baseline method. This study has implemented few-shot learning for Amharic characters for the first time. More importantly, the findings of the study open new ways of examining the influence of training episodes in few-shot learning, which is one of the important issues that needs exploration. The datasets used for this study are collected from native Amharic language writers using an Android App developed as a part of this study.
Robust Power Allocation and Linear Precoding for Cell-Free and Multi-Cell Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Multi-Cell (MC) systems are present in mobile network operations from the first generation to the fifth generation of wireless networks, and considers the signals of all users to a base station (BS) centered in a cell. Cell-Free (CF) systems works with a large number of distributed antennas serving users at the same time. In this context, Multiple-input multiple-output (MIMO) techniques are used in both topologies and result in performance gains and interference reduction. In order to achieve the benefits mentioned, proper precoder design and power allocation techniques are required in the downlink (DL). In general, DL schemes assume perfect channel state information at the transmitter (CSIT), which is not realistic. This paper studies MC and CF with MIMO systems equipped with linear precoders in the DL and proposes an adaptive algorithm to allocate power in the presence of imperfect CSIT. The proposed robust adaptive power allocation outperforms standard adaptive and uniform power allocation. Simulations also compare the performance of both systems frameworks using minimum mean-square error (MMSE) precoders with robust adaptive power allocation and adaptive power allocation.
Federated Representation Learning via Maximal Coding Rate Reduction
Venue: arXiv cs.LG 2022
Year: 2022

We propose a federated methodology to learn low-dimensional representations from a dataset that is distributed among several clients. In particular, we move away from the commonly-used cross-entropy loss in federated learning, and seek to learn shared low-dimensional representations of the data in a decentralized manner via the principle of maximal coding rate reduction (MCR2). Our proposed method, which we refer to as FLOW, utilizes MCR2 as the objective of choice, hence resulting in representations that are both between-class discriminative and within-class compressible. We theoretically show that our distributed algorithm achieves a first-order stationary point. Moreover, we demonstrate, via numerical experiments, the utility of the learned low-dimensional representations.
MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation
Venue: arXiv cs.CL 2022
Year: 2022

Large pre-trained language models have brought remarkable progress in NLP. Pre-training and Fine-tuning have given state-of-art performance across tasks in text processing. Data Augmentation techniques have also helped build state-of-art models on low or zero resource tasks. Many works in the past have attempted at learning a single massively-multilingual machine translation model for zero-shot translation. Although those translation models are producing correct translations, the main challenge is those models are producing the wrong languages for zero-shot translation. This work and its results indicate that prompt conditioned large models do not suffer from off-target language errors i.e. errors arising due to translation to wrong languages. We empirically demonstrate the effectiveness of self-supervised pre-training and data augmentation for zero-shot multi-lingual machine translation.
Deep Recurrent Q-learning for Energy-constrained Coverage with a Mobile Robot
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we study the problem of coverage of an environment with an energy-constrained robot in the presence of multiple charging stations. As the robot's on-board power supply is limited, it might not have enough energy to cover all the points in the environment with a single charge. Instead, it will need to stop at one or more charging stations to recharge its battery intermittently. The robot cannot violate the energy constraint, i.e., visit a location with negative available energy. To solve this problem, we propose a deep Q-learning framework that produces a policy to maximize the coverage and minimize the budget violations. Our proposed framework also leverages the memory of a recurrent neural network (RNN) to better suit this multi-objective optimization problem. We have tested the presented framework within a 16 x 16 grid environment having charging stations and various obstacle configurations. Results show that our proposed method finds feasible solutions and outperforms a comparable existing technique.
Online Revenue Maximization with Unknown Concave Utilities
Venue: arXiv cs.GT 2022
Year: 2022

We study an online revenue maximization problem where the consumers arrive i.i.d from some unknown distribution and purchase a bundle of products from the sellers. The classical approach generally assumes complete knowledge of the consumer utility functions, while recent works have been devoted to unknown linear utility functions. This paper focuses on the online posted-price model with unknown consumer distribution and unknown consumer utilities, given they are concave. Hence, the two questions to ask are i) when is the seller's online maximization problem concave, and ii) how to find the optimal pricing strategy for non-linear utilities. We answer the first question by imposing a third-order smoothness condition on the utilities. The second question is addressed by two algorithms, which we prove to exhibit the sub-linear regrets of $O(T^{\frac{2}{3}} (\log T)^{\frac{1}{3}})$ and $O(T^{\frac{1}{2}} (\log T)^{\frac{1}{2}})$ respectively.
Leveraging unsupervised data and domain adaptation for deep regression in low-cost sensor calibration
Venue: arXiv cs.LG 2022
Year: 2022

Air quality monitoring is becoming an essential task with rising awareness about air quality. Low cost air quality sensors are easy to deploy but are not as reliable as the costly and bulky reference monitors. The low quality sensors can be calibrated against the reference monitors with the help of deep learning. In this paper, we translate the task of sensor calibration into a semi-supervised domain adaptation problem and propose a novel solution for the same. The problem is challenging because it is a regression problem with covariate shift and label gap. We use histogram loss instead of mean squared or mean absolute error, which is commonly used for regression, and find it useful against covariate shift. To handle the label gap, we propose weighting of samples for adversarial entropy optimization. In experimental evaluations, the proposed scheme outperforms many competitive baselines, which are based on semi-supervised and supervised domain adaptation, in terms of R2 score and mean absolute error. Ablation studies show the relevance of each proposed component in the entire scheme.
Comparison of Data Representations and Machine Learning Architectures for User Identification on Arbitrary Motion Sequences
Venue: arXiv cs.LG 2022
Year: 2022

Reliable and robust user identification and authentication are important and often necessary requirements for many digital services. It becomes paramount in social virtual reality (VR) to ensure trust, specifically in digital encounters with lifelike realistic-looking avatars as faithful replications of real persons. Recent research has shown that the movements of users in extended reality (XR) systems carry user-specific information and can thus be used to verify their identities. This article compares three different potential encodings of the motion data from head and hands (scene-relative, body-relative, and body-relative velocities), and the performances of five different machine learning architectures (random forest, multi-layer perceptron, fully recurrent neural network, long-short term memory, gated recurrent unit). We use the publicly available dataset "Talking with Hands" and publish all code to allow reproducibility and to provide baselines for future work. After hyperparameter optimization, the combination of a long-short term memory architecture and body-relative data outperformed competing combinations: the model correctly identifies any of the 34 subjects with an accuracy of 100% within 150 seconds. Altogether, our approach provides an effective foundation for behaviometric-based identification and authentication to guide researchers and practitioners. Data and code are published under https://go.uniwue.de/58w1r.
Fine-grained Contrastive Learning for Definition Generation
Venue: arXiv cs.CL 2022
Year: 2022

Recently, pre-trained transformer-based models have achieved great success in the task of definition generation (DG). However, previous encoder-decoder models lack effective representation learning to contain full semantic components of the given word, which leads to generating under-specific definitions. To address this problem, we propose a novel contrastive learning method, encouraging the model to capture more detailed semantic representations from the definition sequence encoding. According to both automatic and manual evaluation, the experimental results on three mainstream benchmarks demonstrate that the proposed method could generate more specific and high-quality definitions compared with several state-of-the-art models.
Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis
Venue: arXiv cs.LG 2022
Year: 2022

Adversarial Training (AT) has been demonstrated as one of the most effective methods against adversarial examples. While most existing works focus on AT with a single type of perturbation e.g., the $\ell_\infty$ attacks), DNNs are facing threats from different types of adversarial examples. Therefore, adversarial training for multiple perturbations (ATMP) is proposed to generalize the adversarial robustness over different perturbation types (in $\ell_1$, $\ell_2$, and $\ell_\infty$ norm-bounded perturbations). However, the resulting model exhibits trade-off between different attacks. Meanwhile, there is no theoretical analysis of ATMP, limiting its further development. In this paper, we first provide the smoothness analysis of ATMP and show that $\ell_1$, $\ell_2$, and $\ell_\infty$ adversaries give different contributions to the smoothness of the loss function of ATMP. Based on this, we develop the stability-based excess risk bounds and propose adaptive smoothness-weighted adversarial training for multiple perturbations. Theoretically, our algorithm yields better bounds. Empirically, our experiments on CIFAR10 and CIFAR100 achieve the state-of-the-art performance against the mixture of multiple perturbations attacks.
An Efficient Cyclic Entailment Procedure in a Fragment of Separation Logic
Venue: arXiv cs.LO 2022
Year: 2022

An efficient entailment proof system is essential to compositional verification using separation logic. Unfortunately, existing decision procedures are either inexpressive or inefficient. For example, Smallfoot is an efficient procedure but only works with hardwired lists and trees. Other procedures that can support general inductive predicates run exponentially in time as their proof search requires back-tracking to deal with a disjunction in the consequent. This paper presents a decision procedure to derive cyclic entailment proofs for general inductive predicates in polynomial time. Our procedure is efficient and does not require back-tracking; it uses normalisation rules that help avoid the introduction of disjunction in the consequent. Moreover, our decidable fragment is sufficiently expressive: It is based on compositional predicates and can capture a wide range of data structures, including sorted and nested list segments, skip lists with fast forward pointers, and binary search trees. We have implemented the proposal in a prototype tool and evaluated it over challenging problems taken from a recent separation logic competition. The experimental results confirm the efficiency of the proposed system.
Pixel-global Self-supervised Learning with Uncertainty-aware Context Stabilizer
Venue: arXiv cs.CV 2022
Year: 2022

We developed a novel SSL approach to capture global consistency and pixel-level local consistencies between differently augmented views of the same images to accommodate downstream discriminative and dense predictive tasks. We adopted the teacher-student architecture used in previous contrastive SSL methods. In our method, the global consistency is enforced by aggregating the compressed representations of augmented views of the same image. The pixel-level consistency is enforced by pursuing similar representations for the same pixel in differently augmented views. Importantly, we introduced an uncertainty-aware context stabilizer to adaptively preserve the context gap created by the two views from different augmentations. Moreover, we used Monte Carlo dropout in the stabilizer to measure uncertainty and adaptively balance the discrepancy between the representations of the same pixels in different views.