pwc-archive/papers-with-abstracts huggingface dataset:
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP
Authors: Rob van der Goot, Ahmet Üstün, Alan Ramponi, Ibrahim Sharaf, Barbara Plank
Date: 2020-05-29 00:00:00
Proceeding: EACL 2021 2
Tasks: Dependency Parsing, Language Modeling, Language Modelling, Masked Language Modeling, Multi-Task Learning, text-classification, Text Classification, Text Generation, Transfer Learning

Transfer learning, particularly approaches that combine multi-task learning with pre-trained contextualized embeddings and fine-tuning, have advanced the field of Natural Language Processing tremendously in recent years. In this paper we present MaChAmp, a toolkit for easy fine-tuning of contextualized embeddings in multi-task settings. The benefits of MaChAmp are its flexible configuration options, and the support of a variety of natural language processing tasks in a uniform toolkit, from text classification and sequence labeling to dependency parsing, masked language modeling, and text generation.
The energy distance for ensemble and scenario reduction
Authors: Florian Ziel
Date: 2020-05-29 00:00:00
Tasks: Clustering

Scenario reduction techniques are widely applied for solving sophisticated dynamic and stochastic programs, especially in energy and power systems, but also used in probabilistic forecasting, clustering and estimating generative adversarial networks (GANs). We propose a new method for ensemble and scenario reduction based on the energy distance which is a special case of the maximum mean discrepancy (MMD). We discuss the choice of energy distance in detail, especially in comparison to the popular Wasserstein distance which is dominating the scenario reduction literature. The energy distance is a metric between probability measures that allows for powerful tests for equality of arbitrary multivariate distributions or independence. Thanks to the latter, it is a suitable candidate for ensemble and scenario reduction problems. The theoretical properties and considered examples indicate clearly that the reduced scenario sets tend to exhibit better statistical properties for the energy distance than a corresponding reduction with respect to the Wasserstein distance. We show applications to a Bernoulli random walk and two real data based examples for electricity demand profiles and day-ahead electricity prices.
First Neural Conjecturing Datasets and Experiments
Authors: Josef Urban, Jan Jakubův
Date: 2020-05-29 00:00:00

We describe several datasets and first experiments with creating conjectures by neural methods. The datasets are based on the Mizar Mathematical Library processed in several forms and the problems extracted from it by the MPTP system and proved by the E prover using the ENIGMA guidance. The conjecturing experiments use the Transformer architecture and in particular its GPT-2 implementation.
SLAM-Inspired Simultaneous Contextualization and Interpreting for Incremental Conversation Sentences
Authors: Yusuke Takimoto, Yosuke Fukuchi, Shoya Matsumori, Michita Imai
Date: 2020-05-29 00:00:00
Tasks: Simultaneous Localization and Mapping

Distributed representation of words has improved the performance for many natural language tasks. In many methods, however, only one meaning is considered for one label of a word, and multiple meanings of polysemous words depending on the context are rarely handled. Although research works have dealt with polysemous words, they determine the meanings of such words according to a batch of large documents. Hence, there are two problems with applying these methods to sequential sentences, as in a conversation that contains ambiguous expressions. The first problem is that the methods cannot sequentially deal with the interdependence between context and word interpretation, in which context is decided by word interpretations and the word interpretations are decided by the context. Context estimation must thus be performed in parallel to pursue multiple interpretations. The second problem is that the previous methods use large-scale sets of sentences for offline learning of new interpretations, and the steps of learning and inference are clearly separated. Such methods using offline learning cannot obtain new interpretations during a conversation. Hence, to dynamically estimate the conversation context and interpretations of polysemous words in sequential sentences, we propose a method of Simultaneous Contextualization And INterpreting (SCAIN) based on the traditional Simultaneous Localization And Mapping (SLAM) algorithm. By using the SCAIN algorithm, we can sequentially optimize the interdependence between context and word interpretation while obtaining new interpretations online. For experimental evaluation, we created two datasets: one from Wikipedia's disambiguation pages and the other from real conversations. For both datasets, the results confirmed that SCAIN could effectively achieve sequential optimization of the interdependence and acquisition of new interpretations.
Federated Face Presentation Attack Detection
Authors: Rui Shao, Pramuditha Perera, Pong C. Yuen, Vishal M. Patel
Date: 2020-05-29 00:00:00
Tasks: Face Anti-Spoofing, Face Presentation Attack Detection, Face Recognition, Federated Learning

Face presentation attack detection (fPAD) plays a critical role in the modern face recognition pipeline. A face presentation attack detection model with good generalization can be obtained when it is trained with face images from different input distributions and different types of spoof attacks. In reality, training data (both real face images and spoof images) are not directly shared between data owners due to legal and privacy issues. In this paper, with the motivation of circumventing this challenge, we propose Federated Face Presentation Attack Detection (FedPAD) framework. FedPAD simultaneously takes advantage of rich fPAD information available at different data owners while preserving data privacy. In the proposed framework, each data owner (referred to as \textit{data centers}) locally trains its own fPAD model. A server learns a global fPAD model by iteratively aggregating model updates from all data centers without accessing private data in each of them. Once the learned global model converges, it is used for fPAD inference. We introduce the experimental setting to evaluate the proposed FedPAD framework and carry out extensive experiments to provide various insights about federated learning for fPAD.
Machine learning methods to detect money laundering in the Bitcoin blockchain in the presence of label scarcity
Authors: Joana Lorenz, Maria Inês Silva, David Aparício, João Tiago Ascensão, Pedro Bizarro
Date: 2020-05-29 00:00:00
Tasks: Active Learning, Anomaly Detection, Unsupervised Anomaly Detection

Every year, criminals launder billions of dollars acquired from serious felonies (e.g., terrorism, drug smuggling, or human trafficking) harming countless people and economies. Cryptocurrencies, in particular, have developed as a haven for money laundering activity. Machine Learning can be used to detect these illicit patterns. However, labels are so scarce that traditional supervised algorithms are inapplicable. Here, we address money laundering detection assuming minimal access to labels. First, we show that existing state-of-the-art solutions using unsupervised anomaly detection methods are inadequate to detect the illicit patterns in a real Bitcoin transaction dataset. Then, we show that our proposed active learning solution is capable of matching the performance of a fully supervised baseline by using just 5\% of the labels. This solution mimics a typical real-life situation in which a limited number of labels can be acquired through manual annotation by experts.
Los perfiles de investigación y su implantación en la Universidad Publica de Navarra
Authors: Manuel Ruiz de Luzuriaga Peña, Isabel Muñoz Mouriño, Mercedes Bogino Larrambebere
Date: 2020-05-29 00:00:00

This work aims to monitor and control the presence of UPNA research staff in the main research profiles platforms, not only in the most obvious ones such as Google Scholar Citation, Researcher ID, Scopus ID and ORCID, but also in other services that, in practice, they function as research profiles, such as Mendeley, Linkedin, ResearchGate, Academia.edu and Academica-e. We also find it interesting to analyze that presence and see how it responds to a variables, such as the department, gender, job category, research group. In this study we have excluded some platforms for different reasons. Dialnet profiles are entered from the UPNA library (BUPNA), which means that all those who meet the requirements for inclusion would be there, so their analysis does not make much sense, since it depends on factors outside the will of the researcher himself. The same is the case with the UPNA Scientific Production Portal (PPC): the data is entered from the Vicerrectorado de Investigaci\'on and should include all members of the UPNA PDI. Using as a base the census of university research staff provided by the Vicerrectorado de Investigaci\'on, it has been verified, for each author, the existence or not of a profile in the different services studied. The results have been tabulated in an Excel file to be able to analyze them later. The data has been collected in March 2018 for Orcid, ResearcherID, ScopusID, Google Scholar Citations and Mendeley. In November 2018, data from Academica-e, Academia.edu, ResearchGate and Linkedin were taken. For each of the profiles, a search by institutional affiliation was used, when possible, to obtain a first list of UPNA research personnel with that profile. Subsequently, a search was carried out, person by person, of the rest of the research staff that did not appear in that first list.
Detection of Bangla Fake News using MNB and SVM Classifier
Authors: Md Gulzar Hussain, Md Rashidul Hasan, Mahmuda Rahman, Joy Protim, Sakib Al Hasan
Date: 2020-05-29 00:00:00

Fake news has been coming into sight in significant numbers for numerous business and political reasons and has become frequent in the online world. People can get contaminated easily by these fake news for its fabricated words which have enormous effects on the offline community. Thus, interest in research in this area has risen. Significant research has been conducted on the detection of fake news from English texts and other languages but a few in Bangla Language. Our work reflects the experimental analysis on the detection of Bangla fake news from social media as this field still requires much focus. In this research work, we have used two supervised machine learning algorithms, Multinomial Naive Bayes (MNB) and Support Vector Machine (SVM) classifiers to detect Bangla fake news with CountVectorizer and Term Frequency - Inverse Document Frequency Vectorizer as feature extraction. Our proposed framework detects fake news depending on the polarity of the corresponding article. Finally, our analysis shows SVM with the linear kernel with an accuracy of 96.64% outperform MNB with an accuracy of 93.32%.
Modeling System Dynamics with Physics-Informed Neural Networks Based on Lagrangian Mechanics
Authors: Manuel A. Roehrl, Thomas A. Runkler, Veronika Brandtstetter, Michel Tokic, Stefan Obermayer
Date: 2020-05-29 00:00:00

Identifying accurate dynamic models is required for the simulation and control of various technical systems. In many important real-world applications, however, the two main modeling approaches often fail to meet requirements: first principles methods suffer from high bias, whereas data-driven modeling tends to have high variance. Additionally, purely data-based models often require large amounts of data and are often difficult to interpret. In this paper, we present physics-informed neural ordinary differential equations (PINODE), a hybrid model that combines the two modeling techniques to overcome the aforementioned problems. This new approach directly incorporates the equations of motion originating from the Lagrange Mechanics into a deep neural network structure. Thus, we can integrate prior physics knowledge where it is available and use function approximation--e. g., neural networks--where it is not. The method is tested with a forward model of a real-world physical system with large uncertainties. The resulting model is accurate and data-efficient while ensuring physical plausibility. With this, we demonstrate a method that beneficially merges physical insight with real data. Our findings are of interest for model-based control and system identification of mechanical systems.
Using Large Pretrained Language Models for Answering User Queries from Product Specifications
Authors: Kalyani Roy, Smit Shah, Nithish Pai, Jaidam Ramtej, Prajit Prashant Nadkarn, Jyotirmoy Banerjee, Pawan Goyal, Surender Kumar
Date: 2020-05-29 00:00:00
Proceeding: WS 2020 7
Tasks: Question Answering

While buying a product from the e-commerce websites, customers generally have a plethora of questions. From the perspective of both the e-commerce service provider as well as the customers, there must be an effective question answering system to provide immediate answers to the user queries. While certain questions can only be answered after using the product, there are many questions which can be answered from the product specification itself. Our work takes a first step in this direction by finding out the relevant product specifications, that can help answering the user questions. We propose an approach to automatically create a training dataset for this problem. We utilize recently proposed XLNet and BERT architectures for this problem and find that they provide much better performance than the Siamese model, previously applied for this problem. Our model gives a good performance even when trained on one vertical and tested across different verticals.
Non-Local Graph Neural Networks
Authors: Meng Liu, Zhengyang Wang, Shuiwang Ji
Date: 2020-05-29 00:00:00
Tasks: Node Classification, Node Classification on Non-Homophilic (Heterophilic) Graphs

Modern graph neural networks (GNNs) learn node embeddings through multilayer local aggregation and achieve great success in applications on assortative graphs. However, tasks on disassortative graphs usually require non-local aggregation. In addition, we find that local aggregation is even harmful for some disassortative graphs. In this work, we propose a simple yet effective non-local aggregation framework with an efficient attention-guided sorting for GNNs. Based on it, we develop various non-local GNNs. We perform thorough experiments to analyze disassortative graph datasets and evaluate our non-local GNNs. Experimental results demonstrate that our non-local GNNs significantly outperform previous state-of-the-art methods on seven benchmark datasets of disassortative graphs, in terms of both model performance and efficiency.
CoolMomentum: A Method for Stochastic Optimization by Langevin Dynamics with Simulated Annealing
Authors: Oleksandr Borysenko, Maksym Byshkin
Date: 2020-05-29 00:00:00
Tasks: global-optimization, Physical Simulations, Stochastic Optimization, Unity

Deep learning applications require global optimization of non-convex objective functions, which have multiple local minima. The same problem is often found in physical simulations and may be resolved by the methods of Langevin dynamics with Simulated Annealing, which is a well-established approach for minimization of many-particle potentials. This analogy provides useful insights for non-convex stochastic optimization in machine learning. Here we find that integration of the discretized Langevin equation gives a coordinate updating rule equivalent to the famous Momentum optimization algorithm. As a main result, we show that a gradual decrease of the momentum coefficient from the initial value close to unity until zero is equivalent to application of Simulated Annealing or slow cooling, in physical terms. Making use of this novel approach, we propose CoolMomentum -- a new stochastic optimization method. Applying Coolmomentum to optimization of Resnet-20 on Cifar-10 dataset and Efficientnet-B0 on Imagenet, we demonstrate that it is able to achieve high accuracies.
Semi-supervised Embedding Learning for High-dimensional Bayesian Optimization
Authors: Jingfan Chen, Guanghui Zhu, Chunfeng Yuan, Yihua Huang
Date: 2020-05-29 00:00:00
Tasks: Bayesian Optimization, Dimensionality Reduction, Hyperparameter Optimization, Vocal Bursts Intensity Prediction

Bayesian optimization is a broadly applied methodology to optimize the expensive black-box function. Despite its success, it still faces the challenge from the high-dimensional search space. To alleviate this problem, we propose a novel Bayesian optimization framework (termed SILBO), which finds a low-dimensional space to perform Bayesian optimization iteratively through semi-supervised dimension reduction. SILBO incorporates both labeled points and unlabeled points acquired from the acquisition function to guide the embedding space learning. To accelerate the learning procedure, we present a randomized method for generating the projection matrix. Furthermore, to map from the low-dimensional space to the high-dimensional original space, we propose two mapping strategies: $\text{SILBO}_{FZ}$ and $\text{SILBO}_{FX}$ according to the evaluation overhead of the objective function. Experimental results on both synthetic function and hyperparameter optimization tasks demonstrate that SILBO outperforms the existing state-of-the-art high-dimensional Bayesian optimization methods.
Fixed-size Objects Encoding for Visual Relationship Detection
Authors: Hengyue Pan, Xin Niu, Rongchun Li, Siqi Shen, Yong Dou
Date: 2020-05-29 00:00:00
Tasks: General Classification, Object, Predicate Classification, Relationship Detection, Triplet, Visual Relationship Detection

In this paper, we propose a fixed-size object encoding method (FOE-VRD) to improve performance of visual relationship detection tasks. Comparing with previous methods, FOE-VRD has an important feature, i.e., it uses one fixed-size vector to encoding all objects in each input image to assist the process of relationship detection. Firstly, we use a regular convolution neural network as a feature extractor to generate high-level features of input images. Then, for each relationship triplet in input images, i.e., $<$subject-predicate-object$>$, we apply ROI-pooling to get feature vectors of two regions on the feature maps that corresponding to bounding boxes of the subject and object. Besides the subject and object, our analysis implies that the results of predicate classification may also related to the rest objects in input images (we call them background objects). Due to the variable number of background objects in different images and computational costs, we cannot generate feature vectors for them one-by-one by using ROI pooling technique. Instead, we propose a novel method to encode all background objects in each image by using one fixed-size vector (i.e., FBE vector). By concatenating the 3 vectors we generate above, we successfully encode the objects using one fixed-size vector. The generated feature vector is then feed into a fully connected neural network to get predicate classification results. Experimental results on VRD database (entire set and zero-shot tests) show that the proposed method works well on both predicate classification and relationship detection.
Improving Unsupervised Sparsespeech Acoustic Models with Categorical Reparameterization
Authors: Benjamin Milde, Chris Biemann
Date: 2020-05-29 00:00:00
Tasks: Speech Recognition

The Sparsespeech model is an unsupervised acoustic model that can generate discrete pseudo-labels for untranscribed speech. We extend the Sparsespeech model to allow for sampling over a random discrete variable, yielding pseudo-posteriorgrams. The degree of sparsity in this posteriorgram can be fully controlled after the model has been trained. We use the Gumbel-Softmax trick to approximately sample from a discrete distribution in the neural network and this allows us to train the network efficiently with standard backpropagation. The new and improved model is trained and evaluated on the Libri-Light corpus, a benchmark for ASR with limited or no supervision. The model is trained on 600h and 6000h of English read speech. We evaluate the improved model using the ABX error measure and a semi-supervised setting with 10h of transcribed speech. We observe a relative improvement of up to 31.4% on ABX error rates across speakers on the test set with the improved Sparsespeech model on 600h of speech data and further improvements when we scale the model to 6000h.
Harbsafe-162. A Domain-Specific Data Set for the Intrinsic Evaluation of Semantic Representations for Terminological Data
Authors: Susanne Arndt, Dieter Schnäpp
Date: 2020-05-29 00:00:00
Tasks: Sentence, Sentence Embedding, Sentence-Embedding

The article presents Harbsafe-162, a domain-specific data set for evaluating distributional semantic models. It originates from a cooperation by Technische Universit\"at Braunschweig and the German Commission for Electrical, Electronic & Information Technologies of DIN and VDE, the Harbsafe project. One objective of the project is to apply distributional semantic models to terminological entries, that is, complex lexical data comprising of at least one or several terms, term phrases and a definition. This application is needed to solve a more complex problem: the harmonization of terminologies of standards and standards bodies (i.e. resolution of doublettes and inconsistencies). Due to a lack of evaluation data sets for terminological entries, the creation of Harbsafe-162 was a necessary step towards harmonization assistance. Harbsafe-162 covers data from nine electrotechnical standards in the domain of functional safety, IT security, and dependability. An intrinsic evaluation method in the form of a similarity rating task has been applied in which two linguists and three domain experts from standardization participated. The data set is used to evaluate a specific implementation of an established sentence embedding model. This implementation proves to be satisfactory for the domain-specific data so that further implementations for harmonization assistance may be brought forward by the project. Considering recent criticism on intrinsic evaluation methods, the article concludes with an evaluation of Harbsafe-162 and joins a more general discussion about the nature of similarity rating tasks. Harbsafe-162 has been made available for the community.
Probabilistic Object Classification using CNN ML-MAP layers
Authors: G. Melotti, C. Premebida, J. J. Bird, D. R. Faria, N. Gonçalves
Date: 2020-05-29 00:00:00
Tasks: Autonomous Driving, Bayesian Inference, Classification, General Classification, Object

Deep networks are currently the state-of-the-art for sensory perception in autonomous driving and robotics. However, deep models often generate overconfident predictions precluding proper probabilistic interpretation which we argue is due to the nature of the SoftMax layer. To reduce the overconfidence without compromising the classification performance, we introduce a CNN probabilistic approach based on distributions calculated in the network's Logit layer. The approach enables Bayesian inference by means of ML and MAP layers. Experiments with calibrated and the proposed prediction layers are carried out on object classification using data from the KITTI database. Results are reported for camera ($RGB$) and LiDAR (range-view) modalities, where the new approach shows promising performance compared to SoftMax.