Miracle-dz/UniFAR-FTUs huggingface dataset:
Alignment over Heterogeneous Embeddings for Question Answering

We propose a simple, fast, and mostly-unsupervised approach for non-factoid question answering (QA) called Alignment over Heterogeneous Embeddings (AHE). AHE simply aligns each word in the question and candidate answer with the most similar word in the retrieved supporting paragraph, and weighs each alignment score with the inverse document frequency of the corresponding question/answer term. AHE’s similarity function operates over embeddings that model the underlying text at different levels of abstraction: character (FLAIR), word (BERT and GloVe), and sentence (InferSent), where the latter is the only supervised component in the proposed approach. Despite its simplicity and lack of supervision, AHE obtains a new state-of-the-art performance on the “Easy” partition of the AI2 Reasoning Challenge (ARC) dataset (64.6% accuracy), top-two performance on the “Challenge” partition of ARC (34.1%), and top-three performance on the WikiQA dataset (74.08% MRR), outperforming many other complex, supervised approaches. Our error analysis indicates that alignments over character, word, and sentence embeddings capture substantially different semantic information. We exploit this with a simple meta-classifier that learns how much to trust the predictions over each representation, which further improves the performance of unsupervised AHE.
Using Natural Language Relations between Answer Choices for Machine Comprehension

While evaluating an answer choice for Reading Comprehension task, other answer choices available for the question and the answers of related questions about the same paragraph often provide valuable information. In this paper, we propose a method to leverage the natural language relations between the answer choices, such as entailment and contradiction, to improve the performance of machine comprehension. We use a stand-alone question answering (QA) system to perform QA task and a Natural Language Inference (NLI) system to identify the relations between the choice pairs. Then we perform inference using an Integer Linear Programming (ILP)-based relational framework to re-evaluate the decisions made by the standalone QA system in light of the relations identified by the NLI system. We also propose a multitask learning model that learns both the tasks jointly.
Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference

Mean field inference in probabilistic models is generally a highly nonconvex problem. Existing optimization methods, e.g., coordinate ascent algorithms, typically only find local optima. In this work we propose provable mean field methods for probabilistic log-submodular models and its posterior agreement (PA) with strong approximation guarantees. The main algorithmic technique is a new Double Greedy scheme, termed DR-DoubleGreedy, for continuous DR-submodular maximization with boxconstraints. This one-pass algorithm achieves the optimal 1/2 approximation ratio, which may be of independent interest. We validate the superior performance of our algorithms with baseline results on both synthetic and real-world datasets.
Learning Unsupervised Multilingual Word Embeddings with Incremental Multilingual Hubs

Recent research has discovered that a shared bilingual word embedding space can be induced by projecting monolingual word embedding spaces from two languages using a self-learning paradigm without any bilingual supervision. However, it has also been shown that for distant language pairs such fully unsupervised self-learning methods are unstable and often get stuck in poor local optima due to reduced isomorphism between starting monolingual spaces. In this work, we propose a new robust framework for learning unsupervised multilingual word embeddings that mitigates the instability issues. We learn a shared multilingual embedding space for a variable number of languages by incrementally adding new languages one by one to the current multilingual space. Through the gradual language addition the method can leverage the interdependencies between the new language and all other languages in the current multilingual space. We find that it is beneficial to project more distant languages later in the iterative process. Our fully unsupervised multilingual embedding spaces yield results that are on par with the state-of-the-art methods in the bilingual lexicon induction (BLI) task, and simultaneously obtain state-of-the-art scores on two downstream tasks: multilingual document classification and multilingual dependency parsing, outperforming even supervised baselines. This finding also accentuates the need to establish evaluation protocols for cross-lingual word embeddings beyond the omnipresent intrinsic BLI task in future work.
Omniring: Scaling Up Private Payments Without Trusted Setup - Formal Foundations and Constructions of Ring Confidential Transactions with Log-size Proofs

Monero is the largest cryptocurrency with built-in cryptographic privacy features. The transactions are authenticated using zero-knowledge spend proofs, which provide a certain level of anonymity by hiding the source accounts fromwhich the funds are sent among a set of other accounts. Due to its similarities to ring signatures, this core cryptographic component is called Ring Confidential Transactions (RingCT). Because of its practical relevance, several works attempt to analyze the security of RingCT. Since RingCT is rather complex, most of them are either informal, miss fundamental functionalities, or introduce undesirable trusted setup assumptions. Regarding efficiency, Monero currently deploys a scheme in which the size of the spend proof is linear in the ring size. This limits the ring size to only a few accounts, which in turn limits the acquired anonymity significantly and facilitates de-anonymization attacks. As a solution to these problems, we present the first rigorous formalization of RingCT as a cryptographic primitive. We then propose a generic construction of RingCT and prove it secure in our formal security model. By instantiating our generic construction with new efficient zero-knowledge proofs, we obtain Omniring, a fully-fledged RingCT scheme in the discrete logarithm setting that provides the highest concrete and asymptotic efficiency as of today. Omniring is the first RingCT scheme which 1) does not require a trusted setup or pairing-friendly elliptic curves, 2) has a proof size logarithmic in the size of the ring, and 3) allows to share the same ring between all source accounts in a transaction, thereby enabling significantly improved privacy level without sacrificing performance. Our zero-knowledge proofs rely on novel enhancements to the Bulletproofs framework (S&P 2018), which we believe are of independent interest. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. CCS ’19, November 11–15, 2019, London, United Kingdom ©2019Copyrightheldby theowner/author(s). Publication rights licensed toACM. ACM ISBN 978-1-4503-6747-9/19/11. $15.00 https://doi.org/10.1145/3319535.3345655
Evaluating the Potential of Multi-Seasonal CBERS-04 Imagery for Mapping the Quasi-Circular Vegetation Patches in the Yellow River Delta Using Random Forest

High-resolution satellite imagery enables decametric-scale quasi-circular vegetation patch (QVP) mapping, which greatly aids the monitoring of vegetation restoration projects and the development of theories in pattern evolution and maintenance research. This study analyzed the potential of employing five seasonal fused 5 m spatial resolution CBERS-04 satellite images to map QVPs in the Yellow River Delta, China, using the Random Forest (RF) classifier. The classification accuracies corresponding to individual and multi-season combined images were compared to understand the seasonal effect and the importance of optimal image timing and acquisition frequency for QVP mapping. For classification based on single season imagery, the early spring March imagery, with an overall accuracy (OA) of 98.1%, was proven to be more adequate than the other four individual seasonal images. The early spring (March) and winter (December) combined dataset produced the most accurate QVP detection results, with a precision rate of 66.3%, a recall rate of 43.9%, and an F measure of 0.528. For larger study areas, the gain in accuracy should be balanced against the increase in processing time and space when including the derived spectral indices in the RF classification model. Future research should focus on applying higher resolution imagery to QVP mapping.
Better Modeling of Incomplete Annotations for Named Entity Recognition

Supervised approaches to named entity recognition (NER) are largely developed based on the assumption that the training data is fully annotated with named entity information. However, in practice, annotated data can often be imperfect with one typical issue being the training data may contain incomplete annotations. We highlight several pitfalls associated with learning under such a setup in the context of NER and identify limitations associated with existing approaches, proposing a novel yet easy-to-implement approach for recognizing named entities with incomplete data annotations. We demonstrate the effectiveness of our approach through extensive experiments.
An optimised dataflow engine for GPGPU stream processing

Stream processing applications have high-demanding performance requirements that are hard to tackle using traditional parallel models on modern many-core architectures, such as GPUs. On the other hand, recent dataflow computing models can naturally expose and facilitate the parallelism exploitation for a wide class of applications. Thus, instead of following the program order, different operations can be run in parallel as soon as their input operands become available. This work presents an extension to an existing dataflow library for Java. The library extension implements high-level constructs with multiple command queues to enable the superposition of memory operations and kernel executions on GPUs. Experimental results show that significant speedup can be achieved for a subset of well-known stream processing applications: Volume Ray-Casting, Path-Tracing and Sobel Filter. Moreover, new contributions in respect to concurrency analysis and the Stream processing parallel model in dataflow are presented.
Word Embedding-Based Automatic MT Evaluation Metric using Word Position Information

We propose a new automatic evaluation metric for machine translation. Our proposed metric is obtained by adjusting the Earth Mover’s Distance (EMD) to the evaluation task. The EMD measure is used to obtain the distance between two probability distributions consisting of some signatures having a feature and a weight. We use word embeddings, sentence-level tf-idf, and cosine similarity between two word embeddings, respectively, as the features, weight, and the distance between two features. Results show that our proposed metric can evaluate machine translation based on word meaning. Moreover, for distance, cosine similarity and word position information are used to address word-order differences. We designate this metric as Word Embedding-Based automatic MT evaluation using Word Position Information (WE_WPI). A meta-evaluation using WMT16 metrics shared task set indicates that our WE_WPI achieves the highest correlation with human judgment among several representative metrics.
Schöpf–Schulz–Passarge syndrome resulting from a homozygous nonsense mutation, p.Cys107X, in WNT10A

Schöpf‐Schulz‐Passarge syndrome (SSPS; MIM224750) is a rare autosomal recessive form of ectodermal dysplasia that was recently shown to result from mutations in the WNT10A gene. We now report a 59‐year‐old woman with SSPS in whom a homozygous nonsense mutation (p.Cys107X) in WNT10A was detected. Mutations in this gene may also underlie odonto‐onycho‐dermal dysplasia and other ectodermal dysplasia syndromes. To date, 16 different WNT10A mutations have been reported, although considerable clinical and molecular overlap exists. This report demonstrates the molecular basis of a further case of SSPS and highlights the clinical features of this unusual ectodermal dysplasia syndrome.
Exploiting Worker Correlation for Label Aggregation in Crowdsourcing

Crowdsourcing has emerged as a core component of data science pipelines. From collected noisy worker labels, aggregation models that incorporate worker reliability parameters aim to infer a latent true annotation. In this paper, we argue that existing crowdsourcing approaches do not sufficiently model worker correlations observed in practical settings; we propose in response an enhanced Bayesian classifier combination (EBCC) model, with inference based on a mean-field variational approach. An introduced mixture of intra-class reliabilities---connected to tensor decomposition and item clustering---induces inter-worker correlation. EBCC does not suffer the limitations of existing correlation models: intractable marginalisation of missing labels and poor scaling to large worker cohorts. Extensive empirical comparison on 17 real-world datasets sees EBCC achieving the highest mean accuracy across 10 benchmark crowdsourcing methods.
Generalizing Unmasking for Short Texts

Authorship verification is the problem of inferring whether two texts were written by the same author. For this task, unmasking is one of the most robust approaches as of today with the major shortcoming of only being applicable to book-length texts. In this paper, we present a generalized unmasking approach which allows for authorship verification of texts as short as four printed pages with very high precision at an adjustable recall tradeoff. Our generalized approach therefore reduces the required material by orders of magnitude, making unmasking applicable to authorship cases of more practical proportions. The new approach is on par with other state-of-the-art techniques that are optimized for texts of this length: it achieves accuracies of 75–80%, while also allowing for easy adjustment to forensic scenarios that require higher levels of confidence in the classification.
Doubly Robust Joint Learning for Recommendation on Data Missing Not at Random

In recommender systems, usually the ratings of a user to most items are missing. A critical problem in recommendation learning is that, in reality, the missing ratings are often missing not at random (MNAR). MNAR ratings make it difﬁcult to accurately predict the ratings and unbiasedly estimate the performance of rating prediction. Recent approaches use imputed errors to recover the prediction errors for missing ratings, or weight observed ratings with the propensities of being observed. These approaches can still be severely biased in performance estimation or suffer from the variance of the propensities. We propose a principled approach to overcome these limitations. First, we propose an estimator that integrates the imputed errors and propensities in a doubly robust way to obtain unbiased performance estimation and alleviate the effect of the propensity variance. Based on this estimator, we propose joint learning of rating prediction and error imputation to achieve good performance guarantees. Extensive experiments show that our approach outperforms the state-of-the-art on four real-world datasets.
A Comparative Study of Texture and Convolutional Neural Network Features for Detecting Collapsed Buildings After Earthquakes Using Pre- and Post-Event Satellite Imagery

The accurate and quick derivation of the distribution of damaged building must be considered essential for the emergency response. With the success of deep learning, there is an increasing interest to apply it for earthquake-induced building damage mapping, and its performance has not been compared with conventional methods in detecting building damage after the earthquake. In the present study, the performance of grey-level co-occurrence matrix texture and convolutional neural network (CNN) features were comparatively evaluated with the random forest classifier. Pre- and post-event very high-resolution (VHR) remote sensing imagery were considered to identify collapsed buildings after the 2010 Haiti earthquake. Overall accuracy (OA), allocation disagreement (AD), quantity disagreement (QD), Kappa, user accuracy (UA), and producer accuracy (PA) were used as the evaluation metrics. The results showed that the CNN feature with random forest method had the best performance, achieving an OA of 87.6% and a total disagreement of 12.4%. CNNs have the potential to extract deep features for identifying collapsed buildings compared to the texture feature with random forest method by increasing Kappa from 61.7% to 69.5% and reducing the total disagreement from 16.6% to 14.1%. The accuracy for identifying buildings was improved by combining CNN features with random forest compared with the CNN approach. OA increased from 85.9% to 87.6%, and the total disagreement reduced from 14.1% to 12.4%. The results indicate that the learnt CNN features can outperform texture features for identifying collapsed buildings using VHR remotely sensed space imagery.
Towards a More Reliable Interpretation of Defect Models

Software Quality Assurance (SQA) activities are exercised to ensure high-quality software systems. Defect models help developers identify the most risky modules to prioritise their limited SQA resources. The interpretation of defect models also helps managers understand what factors impact software quality to chart quality improvement plans. Unfortunately, the commonly-used interpretation techniques (e.g., ANOVA for logistic regression and variable importance for random forests) only explain defect models at the high level (e.g., what factors impact software quality). Researchers and practitioners also raise concerns about a lack of explainability of defect models that hinders the adoption in practice. This thesis hypothesises that: A lack of explainability poses a critical challenge when adopting defect models in practice. To validate the hypothesis, we formulate 3 research questions, i.e., (1) what is the best defect modelling workflow that produces the most accurate and reliable interpretation of defect models? , (2) what is the best technique for explaining the predictions of defect models? , and (3) how do practitioners perceive when adopting explainable defect models? Through case studies of publicly-available open-source and industrial software systems, the results show that correlated variables impact the interpretation of defect models and must be mitigated; our proposed feature selection technique, AutoSpearman, is the only studied feature selection technique that can automatically mitigate correlated variables with a little impact on model performance; and the instance-level interpretation of defect models is needed to derive actionable insights to guide operational and technical decisions in SQA efforts.
Mass medication in reducing shipping fever-bovine respiratory disease complex in highly stressed calves.

One thousand and eighty-five newly received, stressed calves were used in studies to determine the effectiveness of certain mass medication procedures for reducing morbidity from shipping fever-bovine respiratory disease complex. In two experiments, im injections of oxytetracycline at 11 mg/kg body wt for 3 successive days reduced treatment days/calf purchased 21 (P less then .05) and 31% (P less than .05). Oral administration of 150 mg of sulfadimethoxine/kg body wt reduced treatment days/calf purchased 20 (P less than .05) and 54% (P less than .05) in the same two experiments. When sulfadimethoxine followed oxytetracycline on the third injection day an 81% reduction in treatment days/calf purchased was obtained, indicating an additive effect of the two drugs. The use of long acting oxytetracycline and sustained release sulfadimethoxine at the time of processing resulted in a 90% reduction in treatment days/calf purchased (P less than .01) and required only one handling of the calves for mass medication purposes.
Learning Context-dependent Label Permutations for Multi-label Classification

A key problem in multi-label classiﬁcation is to utilize dependencies among the labels. Chaining classiﬁers are a simple technique for addressing this problem but current algorithms all assume a ﬁxed, static label ordering. In this work, we pro-pose a multi-label classiﬁcation approach which allows to choose a dynamic, context-dependent label ordering. Our proposed approach consists of two sub-components: a simple EM-like algorithm which bootstraps the learned model, and a more principled approach based on reinforcement learning. Our experiments on three public multi-label classiﬁcation benchmarks show that our proposed dynamic label ordering approach based on reinforcement learning outperforms recurrent neural networks with ﬁxed label ordering across both bipartition and ranking measures on all the three datasets. As a result, we obtain a powerful sequence prediction-based algorithm for multi-label classiﬁcation, which is able to efﬁciently and explicitly exploit label dependencies.
Estimation and Measurement Algorithms

I. OVERVIEW Estimation can be considered as an algorithm that combines a priori system knowledge to infer information from noisy observations. To some extent, the principle of estimation is to act as a means for reducing data to measurements, a task that is central to science and engineering. However, estimation implies more than just data processing, and includes computation to reduce error. Hence optimal estimationis the process of finding an estimate that minimizes the error in accordance with a specific optimality criterion. The estimation problem need not only be considered as a single sensor measuring a single system. Rather, one of the most prominent applications of optimal estimation methods is their role in multisensor systems [13]. In this case, the job of the estimator is to distill a single optimal measure from data obtained by complementary, and potentially redundant, sensors, a familiar example of which is shown in Fig. 1.
Typological Features for Multilingual Delexicalised Dependency Parsing

The existence of universal models to describe the syntax of languages has been debated for decades. The availability of resources such as the Universal Dependencies treebanks and the World Atlas of Language Structures make it possible to study the plausibility of universal grammar from the perspective of dependency parsing. Our work investigates the use of high-level language descriptions in the form of typological features for multilingual dependency parsing. Our experiments on multilingual parsing for 40 languages show that typological information can indeed guide parsers to share information between similar languages beyond simple language identification.