OpenRaiser/Intern-Atlas huggingface dataset:
Application of multilayer perceptron with data augmentation in nuclear physics
Venue: arXiv cs.LG 2022
Year: 2022

Neural networks have become popular in many fields of science since they serve as promising, reliable and powerful tools. In this work, we study the effect of data augmentation on the predictive power of neural network models for nuclear physics data. We present two different data augmentation techniques, and we conduct a detailed analysis in terms of different depths, optimizers, activation functions and random seed values to show the success and robustness of the model. Using the experimental uncertainties for data augmentation for the first time, the size of the training data set is artificially boosted and the changes in the root-mean-square error between the model predictions on the test set and the experimental data are investigated. Our results show that the data augmentation decreases the prediction errors, stabilizes the model and prevents overfitting. The extrapolation capabilities of the MLP models are also tested for newly measured nuclei in AME2020 mass table, and it is shown that the predictions are significantly improved by using data augmentation.
Deep Apprenticeship Learning for Playing Games
Venue: arXiv cs.LG 2022
Year: 2022

In the last decade, deep learning has achieved great success in machine learning tasks where the input data is represented with different levels of abstractions. Driven by the recent research in reinforcement learning using deep neural networks, we explore the feasibility of designing a learning model based on expert behaviour for complex, multidimensional tasks where reward function is not available. We propose a novel method for apprenticeship learning based on the previous research on supervised learning techniques in reinforcement learning. Our method is applied to video frames from Atari games in order to teach an artificial agent to play those games. Even though the reported results are not comparable with the state-of-the-art results in reinforcement learning, we demonstrate that such an approach has the potential to achieve strong performance in the future and is worthwhile for further research.
Harnessing Multilingual Resources to Question Answering in Arabic
Venue: arXiv cs.CL 2022
Year: 2022

The goal of the paper is to predict answers to questions given a passage of Qur'an. The answers are always found in the passage, so the task of the model is to predict where an answer starts and where it ends. As the initial data set is rather small for training, we make use of multilingual BERT so that we can augment the training data by using data available for languages other than Arabic. Furthermore, we crawl a large Arabic corpus that is domain specific to religious discourse. Our approach consists of two steps, first we train a BERT model to predict a set of possible answers in a passage. Finally, we use another BERT based model to rank the candidate answers produced by the first BERT model.
DeepSim: A Reinforcement Learning Environment Build Toolkit for ROS and Gazebo
Venue: arXiv cs.LG 2022
Year: 2022

We propose DeepSim, a reinforcement learning environment build toolkit for ROS and Gazebo. It allows machine learning or reinforcement learning researchers to access the robotics domain and create complex and challenging custom tasks in ROS and Gazebo simulation environments. This toolkit provides building blocks of advanced features such as collision detection, behaviour control, domain randomization, spawner, and many more. DeepSim is designed to reduce the boundary between robotics and machine learning communities by providing Python interface. In this paper, we discuss the components and design decisions of DeepSim Toolkit.
"What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation
Venue: arXiv cs.CL 2022
Year: 2022

We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and finetune models for type-controlled question generation. Empirical results demonstrate that we can generate a variety of questions that adhere to specific types while drawing from the source texts. We also investigate strategies for selecting a single question from a generated set, considering both an informative vs.~inquisitive question classifier and a pairwise ranker trained from a small set of expert annotations. Question selection using the pairwise ranker yields strong results in automatic and manual evaluation. Our human evaluation assesses multiple aspects of the generated questions, finding that the ranker chooses questions with the best syntax (4.59), semantics (4.37), and inquisitiveness (3.92) on a scale of 1-5, even rivaling the performance of human-written questions.
Strong Revenue (Non-)Monotonicity of Single-parameter Auctions
Venue: arXiv cs.GT 2022
Year: 2022

Consider Myerson's optimal auction with respect to an inaccurate prior, e.g., estimated from data, which is an underestimation of the true value distribution. Can the auctioneer expect getting at least the optimal revenue w.r.t. the inaccurate prior since the true value distribution is larger? This so-called strong revenue monotonicity is known to be true for single-parameter auctions when the feasible allocations form a matroid. We find that strong revenue monotonicity fails to generalize beyond the matroid setting, and further show that auctions in the matroid setting are the only downward-closed auctions that satisfy strong revenue monotonicity. On the flip side, we recover an approximate version of strong revenue monotonicity that holds for all single-parameter auctions, even without downward-closedness. As applications, we get sample complexity upper bounds for single-parameter auctions under matroid constraints, downward-closed constraints, and general constraints. They improve the state-of-the-art upper bounds and are tight up to logarithmic factors.
MATrIX -- Modality-Aware Transformer for Information eXtraction
Venue: arXiv cs.CV 2022
Year: 2022

We present MATrIX - a Modality-Aware Transformer for Information eXtraction in the Visual Document Understanding (VDU) domain. VDU covers information extraction from visually rich documents such as forms, invoices, receipts, tables, graphs, presentations, or advertisements. In these, text semantics and visual information supplement each other to provide a global understanding of the document. MATrIX is pre-trained in an unsupervised way with specifically designed tasks that require the use of multi-modal information (spatial, visual, or textual). We consider the spatial and text modalities all at once in a single token set. To make the attention more flexible, we use a learned modality-aware relative bias in the attention mechanism to modulate the attention between the tokens of different modalities. We evaluate MATrIX on 3 different datasets each with strong baselines.
Fast and Provably Convergent Algorithms for Gromov-Wasserstein in Graph Data
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we study the design and analysis of a class of efficient algorithms for computing the Gromov-Wasserstein (GW) distance tailored to large-scale graph learning tasks. Armed with the Luo-Tseng error bound condition~\citep{luo1992error}, two proposed algorithms, called Bregman Alternating Projected Gradient (BAPG) and hybrid Bregman Proximal Gradient (hBPG) enjoy the convergence guarantees. Upon task-specific properties, our analysis further provides novel theoretical insights to guide how to select the best-fit method. As a result, we are able to provide comprehensive experiments to validate the effectiveness of our methods on a host of tasks, including graph alignment, graph partition, and shape matching. In terms of both wall-clock time and modeling performance, the proposed methods achieve state-of-the-art results.
Pairwise Comparison Network for Remote Sensing Scene Classification
Venue: arXiv cs.CV 2022
Year: 2022

Remote sensing scene classification aims to assign a specific semantic label to a remote sensing image. Recently, convolutional neural networks have greatly improved the performance of remote sensing scene classification. However, some confused images may be easily recognized as the incorrect category, which generally degrade the performance. The differences between image pairs can be used to distinguish image categories. This paper proposed a pairwise comparison network, which contains two main steps: pairwise selection and pairwise representation. The proposed network first selects similar image pairs, and then represents the image pairs with pairwise representations. The self-representation is introduced to highlight the informative parts of each image itself, while the mutual-representation is proposed to capture the subtle differences between image pairs. Comprehensive experimental results on two challenging datasets (AID, NWPU-RESISC45) demonstrate the effectiveness of the proposed network. The codes are provided in https://github.com/spectralpublic/PCNet.git.
SEMI-FND: Stacked Ensemble Based Multimodal Inference For Faster Fake News Detection
Venue: arXiv cs.CL 2022
Year: 2022

Fake News Detection (FND) is an essential field in natural language processing that aims to identify and check the truthfulness of major claims in a news article to decide the news veracity. FND finds its uses in preventing social, political and national damage caused due to misrepresentation of facts which may harm a certain section of society. Further, with the explosive rise in fake news dissemination over social media, including images and text, it has become imperative to identify fake news faster and more accurately. To solve this problem, this work investigates a novel multimodal stacked ensemble-based approach (SEMIFND) to fake news detection. Focus is also kept on ensuring faster performance with fewer parameters. Moreover, to improve multimodal performance, a deep unimodal analysis is done on the image modality to identify NasNet Mobile as the most appropriate model for the task. For text, an ensemble of BERT and ELECTRA is used. The approach was evaluated on two datasets: Twitter MediaEval and Weibo Corpus. The suggested framework offered accuracies of 85.80% and 86.83% on the Twitter and Weibo datasets respectively. These reported metrics are found to be superior when compared to similar recent works. Further, we also report a reduction in the number of parameters used in training when compared to recent relevant works. SEMI-FND offers an overall parameter reduction of at least 20% with unimodal parametric reduction on text being 60%. Therefore, based on the investigations presented, it is concluded that the application of a stacked ensembling significantly improves FND over other approaches while also improving speed.
Probabilistic Automata of Bounded Ambiguity
Venue: arXiv cs.FL 2022
Year: 2022

Probabilistic automata are an extension of nondeterministic finite automata in which transitions are annotated with probabilities. Despite its simplicity, this model is very expressive and many of the associated algorithmic questions are undecidable. In this work we focus on the emptiness problem (and its variant the value problem), which asks whether a given probabilistic automaton accepts some word with probability greater than a given threshold. We consider a natural and well-studied structural restriction on automata, namely the degree of ambiguity, which is defined as the maximum number of accepting runs over all words. The known undecidability proofs exploits infinite ambiguity and so we focus on the case of finitely ambiguous probabilistic automata. Our main contributions are to construct efficient algorithms for analysing finitely ambiguous probabilistic automata through a reduction to a multi-objective optimisation problem called the stochastic path problem. We obtain a polynomial time algorithm for approximating the value of probabilistic automata of fixed ambiguity and a quasi-polynomial time algorithm for the emptiness problem for 2-ambiguous probabilistic automata. We complement these positive results by an inapproximability result stating that the value of finitely ambiguous probabilistic automata cannot be approximated unless PTIME = NP.
SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation
Venue: arXiv cs.CL 2022
Year: 2022

We propose the SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation learning framework. Unlike previous works on speech representation learning, which learns multilingual contextual speech embedding at the resolution of an acoustic frame (10-20ms), this work focuses on learning multimodal (speech-text) multilingual speech embedding at the resolution of a sentence (5-10s) such that the embedding vector space is semantically aligned across different languages. We combine state-of-the-art multilingual acoustic frame-level speech representation learning model XLS-R with the Language Agnostic BERT Sentence Embedding (LaBSE) model to create an utterance-level multimodal multilingual speech encoder SAMU-XLSR. Although we train SAMU-XLSR with only multilingual transcribed speech data, cross-lingual speech-text and speech-speech associations emerge in its learned representation space. To substantiate our claims, we use SAMU-XLSR speech encoder in combination with a pre-trained LaBSE text sentence encoder for cross-lingual speech-to-text translation retrieval, and SAMU-XLSR alone for cross-lingual speech-to-speech translation retrieval. We highlight these applications by performing several cross-lingual text and speech translation retrieval tasks across several datasets.
A Multi-level Methodology for Behavioral Comparison of Software-Intensive Systems
Venue: arXiv cs.SE 2022
Year: 2022

Software-intensive systems constantly evolve. To prevent software changes from unintentionally introducing costly system defects, it is important to understand their impact to reduce risk. However, it is in practice nearly impossible to foresee the full impact of software changes when dealing with huge industrial systems with many configurations and usage scenarios. To assist developers with change impact analysis we introduce a novel multi-level methodology for behavioral comparison of software-intensive systems. Our fully automated methodology is based on comparing state machine models of software behavior. We combine existing complementary comparison methods into a novel approach, guiding users step by step though relevant differences by gradually zooming into more and more detail. We empirically evaluate our work through a qualitative exploratory field study, showing its practical value using multiple case studies at ASML, a leading company in developing lithography systems. Our method shows great potential for preventing regressions in system behavior for software changes.
On the Hierarchy of Distributed Majority Protocols
Venue: arXiv cs.DC 2022
Year: 2022

We study the Consensus problem among $n$ agents, defined as follows. Initially, each agent holds one of two possible opinions. The goal is to reach a consensus configuration in which every agent shares the same opinion. To this end, agents randomly sample other agents and update their opinion according to a simple update function depending on the sampled opinions. We consider two communication models: the gossip model and a variant of the population model. In the gossip model, agents are activated in parallel, synchronous rounds. In the population model, one agent is activated after the other in a sequence of discrete time steps. For both models we analyze the following natural family of majority processes called $j$-Majority: when activated, every agent samples $j$ other agents uniformly at random (with replacement) and adopts the majority opinion among the sample (breaking ties uniformly at random). As our main result we show a hierarchy among majority protocols: $(j+1)$-Majority (for $j > 1$) converges stochastically faster than $j$-Majority for any initial opinion configuration. In our analysis we use Strassen's Theorem to prove the existence of a coupling. This gives an affirmative answer for the case of two opinions to an open question asked by Berenbrink et al. [2017].
Delaytron: Efficient Learning of Multiclass Classifiers with Delayed Bandit Feedbacks
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we present online algorithm called {\it Delaytron} for learning multi class classifiers using delayed bandit feedbacks. The sequence of feedback delays $\{d_t\}_{t=1}^T$ is unknown to the algorithm. At the $t$-th round, the algorithm observes an example $\mathbf{x}_t$ and predicts a label $\tilde{y}_t$ and receives the bandit feedback $\mathbb{I}[\tilde{y}_t=y_t]$ only $d_t$ rounds later. When $t+d_t>T$, we consider that the feedback for the $t$-th round is missing. We show that the proposed algorithm achieves regret of $\mathcal{O}\left(\sqrt{\frac{2 K}γ\left[\frac{T}{2}+\left(2+\frac{L^2}{R^2\Vert \W\Vert_F^2}\right)\sum_{t=1}^Td_t\right]}\right)$ when the loss for each missing sample is upper bounded by $L$. In the case when the loss for missing samples is not upper bounded, the regret achieved by Delaytron is $\mathcal{O}\left(\sqrt{\frac{2 K}γ\left[\frac{T}{2}+2\sum_{t=1}^Td_t+\vert \mathcal{M}\vert T\right]}\right)$ where $\mathcal{M}$ is the set of missing samples in $T$ rounds. These bounds were achieved with a constant step size which requires the knowledge of $T$ and $\sum_{t=1}^Td_t$. For the case when $T$ and $\sum_{t=1}^Td_t$ are unknown, we use a doubling trick for online learning and proposed Adaptive Delaytron. We show that Adaptive Delaytron achieves a regret bound of $\mathcal{O}\left(\sqrt{T+\sum_{t=1}^Td_t}\right)$. We show the effectiveness of our approach by experimenting on various datasets and comparing with state-of-the-art approaches.
Outage Analysis of Energy Efficiency in a Finite-Element-IRS Aided Communication System
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we study the performance of an energy efficient wireless communication system, assisted by a finite-element-intelligent reflecting surface (IRS). With no instantaneous channel state information (CSI) at the transmitter, we characterize the system performance in terms of the outage probability (OP) of energy efficiency (EE). Depending upon the availability of line-of-sight (LOS) paths, we analyze the system for two different channel models, viz. Rician and Rayleigh. For an arbitrary number of IRS elements $(N)$, we derive the approximate closed-form solutions for the OP of EE, using Laguerre series and moment matching methods. The analytical results are validated using the Monte-Carlo simulations. Moreover, we also quantify the rate of convergence of the derived expressions to the central limit theorem (CLT) approximations using the \textit{Berry-Esseen} inequality. Further, we prove that the OP of EE is a strict pseudo-convex function of the transmit power and hence, has a unique global minimum. To obtain the optimal transmit power, we solve the OP of EE as a constrained optimization problem. To the best of our knowledge, the OP of EE as a performance metric, has never been previously studied in IRS-assisted wireless communication systems.
Monotonicity Regularization: Improved Penalties and Novel Applications to Disentangled Representation Learning and Robust Classification
Venue: arXiv cs.LG 2022
Year: 2022

We study settings where gradient penalties are used alongside risk minimization with the goal of obtaining predictors satisfying different notions of monotonicity. Specifically, we present two sets of contributions. In the first part of the paper, we show that different choices of penalties define the regions of the input space where the property is observed. As such, previous methods result in models that are monotonic only in a small volume of the input space. We thus propose an approach that uses mixtures of training instances and random points to populate the space and enforce the penalty in a much larger region. As a second set of contributions, we introduce regularization strategies that enforce other notions of monotonicity in different settings. In this case, we consider applications, such as image classification and generative modeling, where monotonicity is not a hard constraint but can help improve some aspects of the model. Namely, we show that inducing monotonicity can be beneficial in applications such as: (1) allowing for controllable data generation, (2) defining strategies to detect anomalous data, and (3) generating explanations for predictions. Our proposed approaches do not introduce relevant computational overhead while leading to efficient procedures that provide extra benefits over baseline models.
Lossy Computing with Side Information via Multi-Hypergraphs
Venue: arXiv cs.IT 2022
Year: 2022

We consider a problem of coding for computing, where the decoder wishes to estimate a function of its local message and the source message at the encoder within a given distortion. We show that the rate-distortion function can be characterized through a characteristic multi-hypergraph, which simplifies the evaluation of the rate-distortion function.
NOMA-aided Joint Communication, Sensing, and Multi-tier Computing Systems
Venue: arXiv cs.IT 2022
Year: 2022

A non-orthogonal multiple access (NOMA)-aided joint communication, sensing, and multi-tier computing (JCSMC) framework is proposed. In this framework, a multi-functional base station (BS) carries out target sensing, while providing edge computing services to the nearby users. To enhance the computation efficiency, the multi-tier computing structure is exploited, where the BS can further offload the computation tasks to a powerful Cloud server (CS). The potential benefits of employing NOMA in the proposed JCSMC framework are investigated, which can maximize the computation offloading capacity and suppress the inter-function interference. Based on the proposed framework, the transmit beamformer of the BS and computation resource allocation at the BS and the CS are jointly optimized to maximize the computation rate subject to the communication-computation causality and the sensing quality constraints. Both partial and binary computation offloading modes are considered: 1) For the partial offloading mode, a weighted minimum mean square error based alternating optimization algorithm is proposed to solve the corresponding non-convex optimization problem. It is proved that a KKT optimal solution can be obtained; 2) For the binary offloading mode, the resultant highly-coupled mixed-integer optimization problem is first transformed to an equivalent but more tractable form. Then, the reformulated problem is solved by utilizing the alternating direction method of multipliers approach to obtain a nearly optimal solution. Finally, numerical results verify the effectiveness of the proposed algorithms and the proposed NOMA-aided JCSMC framework
Tackling Math Word Problems with Fine-to-Coarse Abstracting and Reasoning
Venue: arXiv cs.CL 2022
Year: 2022

Math Word Problems (MWP) is an important task that requires the ability of understanding and reasoning over mathematical text. Existing approaches mostly formalize it as a generation task by adopting Seq2Seq or Seq2Tree models to encode an input math problem in natural language as a global representation and generate the output mathematical expression. Such approaches only learn shallow heuristics and fail to capture fine-grained variations in inputs. In this paper, we propose to model a math word problem in a fine-to-coarse manner to capture both the local fine-grained information and the global logical structure of it. Instead of generating a complete equation sequence or expression tree from the global features, we iteratively combine low-level operands to predict a higher-level operator, abstracting the problem and reasoning about the solving operators from bottom to up. Our model is naturally more sensitive to local variations and can better generalize to unseen problem types. Extensive evaluations on Math23k and SVAMP datasets demonstrate the accuracy and robustness of our method.