pwc-archive/papers-with-abstracts huggingface dataset:
KART: Parameterization of Privacy Leakage Scenarios from Pre-trained Language Models
Authors: Yuta Nakamura, Shouhei Hanaoka, Yukihiro Nomura, Naoto Hayashi, Osamu Abe, Shuntaro Yada, Shoko Wakamiya, Eiji Aramaki
Date: 2020-12-31 00:00:00

For the safe sharing pre-trained language models, no guidelines exist at present owing to the difficulty in estimating the upper bound of the risk of privacy leakage. One problem is that previous studies have assessed the risk for different real-world privacy leakage scenarios and attack methods, which reduces the portability of the findings. To tackle this problem, we represent complex real-world privacy leakage scenarios under a universal parameterization, \textit{Knowledge, Anonymization, Resource, and Target} (KART). KART parameterization has two merits: (i) it clarifies the definition of privacy leakage in each experiment and (ii) it improves the comparability of the findings of risk assessments. We show that previous studies can be simply reviewed by parameterizing the scenarios with KART. We also demonstrate privacy risk assessments in different scenarios under the same attack method, which suggests that KART helps approximate the upper bound of risk under a specific attack or scenario. We believe that KART helps integrate past and future findings on privacy risk and will contribute to a standard for sharing language models.
Modified Gaussian Process Regression Models for Cyclic Capacity Prediction of Lithium-ion Batteries
Authors: Kailong Liu, Xiaosong Hu, Zhongbao Wei, Yi Li, Yan Jiang
Date: 2020-12-31 00:00:00
Tasks: regression

This paper presents the development of machine learning-enabled data-driven models for effective capacity predictions for lithium-ion batteries under different cyclic conditions. To achieve this, a model structure is first proposed with the considerations of battery ageing tendency and the corresponding operational temperature and depth-of-discharge. Then based on a systematic understanding of covariance functions within the Gaussian process regression, two related data-driven models are developed. Specifically, by modifying the isotropic squared exponential kernel with an automatic relevance determination structure, 'Model A' could extract the highly relevant input features for capacity predictions. Through coupling the Arrhenius law and a polynomial equation into a compositional kernel, 'Model B' is capable of considering the electrochemical and empirical knowledge of battery degradation. The developed models are validated and compared on the Nickel Manganese Cobalt Oxide (NMC) lithium-ion batteries with various cycling patterns. Experimental results demonstrate that the modified Gaussian process regression model considering the battery electrochemical and empirical ageing signature outperforms other counterparts and is able to achieve satisfactory results for both one-step and multi-step predictions. The proposed technique is promising for battery capacity predictions under various cycling cases.
Random Embeddings with Optimal Accuracy
Authors: Maciej Skorski
Date: 2020-12-31 00:00:00

This work constructs Jonson-Lindenstrauss embeddings with best accuracy, as measured by variance, mean-squared error and exponential concentration of the length distortion. Lower bounds for any data and embedding dimensions are determined, and accompanied by matching and efficiently samplable constructions (built on orthogonal matrices). Novel techniques: a unit sphere parametrization, the use of singular-value latent variables and Schur-convexity are of independent interest.
A Predictive Model for Geographic Distributions of Mangroves
Authors: Lynn Wahab, Ezzat Chebaro, Jad Ismail, Amir Nasrelddine, Ali El-Zein
Date: 2020-12-30 00:00:00
Tasks: model

Climate change is an impending disaster which is of pressing concern more and more every year. Countless efforts have been made to study the long-term effects of climate change on agriculture, land resources, and biodiversity. Studies involving marine life, however, are less prevalent in the literature. Our research studies the available data on the population of mangroves (groups of shrubs or small trees living in saline coastal intertidal zones) and their correlations to climate change variables, specifically, temperature, heat content, various sea levels, and sea salinity. Mangroves are especially relevant to oceanic ecosystems because of their protective nature towards other marine life, as well as their high absorption rate of carbon dioxide, and their ability to withstand varying levels of salinity of our coasts. The change in global distribution was studied based on global distributions of the previous year, as well as ocean heat content, salinity, temperature, halosteric sea level, thermosteric sea level, and total steric sea level. The best performing predictive model was a support vector regressor, which yielded a correlation coefficient of 0.9998.
UnNatural Language Inference
Authors: Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, Adina Williams
Date: 2020-12-30 00:00:00
Proceeding: ACL 2021 5
Tasks: Natural Language Inference, Natural Language Understanding

Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to know humanlike syntax, at least to some extent. We provide novel evidence that complicates this claim: we find that state-of-the-art Natural Language Inference (NLI) models assign the same labels to permuted examples as they do to the original, i.e. they are largely invariant to random word-order permutations. This behavior notably differs from that of humans; we struggle with ungrammatical sentences. To measure the severity of this issue, we propose a suite of metrics and investigate which properties of particular permutations lead models to be word-order invariant. In the MNLI dataset, for example, we find almost all (98.7%) examples contain at least one permutation which elicits the gold label. Models are sometimes even able to assign gold labels to permutations that they originally failed to predict correctly. We provide a comprehensive empirical evaluation of this phenomenon, and further show that this issue exists for both Transformers and pre-Transformer RNN / ConvNet based encoders, as well as across multiple languages (English and Mandarin Chinese). Our code and data are available at https://github.com/facebookresearch/unlu.
Adversarial Estimation of Riesz Representers
Authors: Victor Chernozhukov, Whitney Newey, Rahul Singh, Vasilis Syrgkanis
Date: 2020-12-30 00:00:00

Many causal parameters are linear functionals of an underlying regression. The Riesz representer is a key component in the asymptotic variance of a semiparametrically estimated linear functional. We propose an adversarial framework to estimate the Riesz representer using general function spaces. We prove a nonasymptotic mean square rate in terms of an abstract quantity called the critical radius, then specialize it for neural networks, random forests, and reproducing kernel Hilbert spaces as leading cases. Our estimators are highly compatible with targeted and debiased machine learning with sample splitting; our guarantees directly verify general conditions for inference that allow mis-specification. We also use our guarantees to prove inference without sample splitting, based on stability or complexity. Our estimators achieve nominal coverage in highly nonlinear simulations where some previous methods break down. They shed new light on the heterogeneous effects of matching grants.
Transformer for Image Quality Assessment
Authors: Junyong You, Jari Korhonen
Date: 2020-12-30 00:00:00
Tasks: Image Quality Assessment

Transformer has become the new standard method in natural language processing (NLP), and it also attracts research interests in computer vision area. In this paper we investigate the application of Transformer in Image Quality (TRIQ) assessment. Following the original Transformer encoder employed in Vision Transformer (ViT), we propose an architecture of using a shallow Transformer encoder on the top of a feature map extracted by convolution neural networks (CNN). Adaptive positional embedding is employed in the Transformer encoder to handle images with arbitrary resolutions. Different settings of Transformer architectures have been investigated on publicly available image quality databases. We have found that the proposed TRIQ architecture achieves outstanding performance. The implementation of TRIQ is published on Github (https://github.com/junyongyou/triq).
Test Score Algorithms for Budgeted Stochastic Utility Maximization
Authors: Dabeen Lee, Milan Vojnovic, Se-Young Yun
Date: 2020-12-30 00:00:00

Motivated by recent developments in designing algorithms based on individual item scores for solving utility maximization problems, we study the framework of using test scores, defined as a statistic of observed individual item performance data, for solving the budgeted stochastic utility maximization problem. We extend an existing scoring mechanism, namely the replication test scores, to incorporate heterogeneous item costs as well as item values. We show that a natural greedy algorithm that selects items solely based on their replication test scores outputs solutions within a constant factor of the optimum for a broad class of utility functions. Our algorithms and approximation guarantees assume that test scores are noisy estimates of certain expected values with respect to marginal distributions of individual item values, thus making our algorithms practical and extending previous work that assumes noiseless estimates. Moreover, we show how our algorithm can be adapted to the setting where items arrive in a streaming fashion while maintaining the same approximation guarantee. We present numerical results, using synthetic data and data sets from the Academia.StackExchange Q&A forum, which show that our test score algorithm can achieve competitiveness, and in some cases better performance than a benchmark algorithm that requires access to a value oracle to evaluate function values.
Towards Robust Data Hiding Against (JPEG) Compression: A Pseudo-Differentiable Deep Learning Approach
Authors: Chaoning Zhang, Adil Karjauv, Philipp Benz, In So Kweon
Date: 2020-12-30 00:00:00

Data hiding is one widely used approach for protecting authentication and ownership. Most multimedia content like images and videos are transmitted or saved in the compressed form. This kind of lossy compression, such as JPEG, can destroy the hidden data, which raises the need of robust data hiding. It is still an open challenge to achieve the goal of data hiding that can be against these compressions. Recently, deep learning has shown large success in data hiding, while non-differentiability of JPEG makes it challenging to train a deep pipeline for improving robustness against lossy compression. The existing SOTA approaches replace the non-differentiable parts with differentiable modules that perform similar operations. Multiple limitations exist: (a) large engineering effort; (b) requiring a white-box knowledge of compression attacks; (c) only works for simple compression like JPEG. In this work, we propose a simple yet effective approach to address all the above limitations at once. Beyond JPEG, our approach has been shown to improve robustness against various image and video lossy compression algorithms.
Explainability Matters: Backdoor Attacks on Medical Imaging
Authors: Munachiso Nwadike, Takumi Miyawaki, Esha Sarkar, Michail Maniatakos, Farah Shamout
Date: 2020-12-30 00:00:00
Tasks: Diagnostic

Deep neural networks have been shown to be vulnerable to backdoor attacks, which could be easily introduced to the training set prior to model training. Recent work has focused on investigating backdoor attacks on natural images or toy datasets. Consequently, the exact impact of backdoors is not yet fully understood in complex real-world applications, such as in medical imaging where misdiagnosis can be very costly. In this paper, we explore the impact of backdoor attacks on a multi-label disease classification task using chest radiography, with the assumption that the attacker can manipulate the training dataset to execute the attack. Extensive evaluation of a state-of-the-art architecture demonstrates that by introducing images with few-pixel perturbations into the training set, an attacker can execute the backdoor successfully without having to be involved with the training procedure. A simple 3$\times$3 pixel trigger can achieve up to 1.00 Area Under the Receiver Operating Characteristic (AUROC) curve on the set of infected images. In the set of clean images, the backdoored neural network could still achieve up to 0.85 AUROC, highlighting the stealthiness of the attack. As the use of deep learning based diagnostic systems proliferates in clinical practice, we also show how explainability is indispensable in this context, as it can identify spatially localized backdoors in inference time.
How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?
Authors: Li Zhong, Zhen Fang, Feng Liu, Jie Lu, Bo Yuan, Guangquan Zhang
Date: 2020-12-30 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) aims to train a target classifier with labeled samples from the source domain and unlabeled samples from the target domain. Classical UDA learning bounds show that target risk is upper bounded by three terms: source risk, distribution discrepancy, and combined risk. Based on the assumption that the combined risk is a small fixed value, methods based on this bound train a target classifier by only minimizing estimators of the source risk and the distribution discrepancy. However, the combined risk may increase when minimizing both estimators, which makes the target risk uncontrollable. Hence the target classifier cannot achieve ideal performance if we fail to control the combined risk. To control the combined risk, the key challenge takes root in the unavailability of the labeled samples in the target domain. To address this key challenge, we propose a method named E-MixNet. E-MixNet employs enhanced mixup, a generic vicinal distribution, on the labeled source samples and pseudo-labeled target samples to calculate a proxy of the combined risk. Experiments show that the proxy can effectively curb the increase of the combined risk when minimizing the source risk and distribution discrepancy. Furthermore, we show that if the proxy of the combined risk is added into loss functions of four representative UDA methods, their performance is also improved.
Towards Understanding the Behaviors of Optimal Deep Active Learning Algorithms
Authors: Yilun Zhou, Adithya Renduchintala, Xian Li, Sida Wang, Yashar Mehdad, Asish Ghoshal
Date: 2020-12-29 00:00:00
Tasks: Active Learning

Active learning (AL) algorithms may achieve better performance with fewer data because the model guides the data selection process. While many algorithms have been proposed, there is little study on what the optimal AL algorithm looks like, which would help researchers understand where their models fall short and iterate on the design. In this paper, we present a simulated annealing algorithm to search for this optimal oracle and analyze it for several tasks. We present qualitative and quantitative insights into the behaviors of this oracle, comparing and contrasting them with those of various heuristics. Moreover, we are able to consistently improve the heuristics using one particular insight. We hope that our findings can better inform future active learning research. The code is available at https://github.com/YilunZhou/optimal-active-learning.
Presenting a Dataset for Collaborator Recommending Systems in Academic Social Network: a Case Study on ReseachGate
Authors: Zahra Roozbahani, Jalal Rezaeenour, Roshan Shahrooei, Hanif Emamgholizadeh
Date: 2020-12-29 00:00:00
Tasks: Question Answering

Collaborator finding systems are a special type of expert finding models. There is a long-lasting challenge for research in the collaborator recommending research area, which is the lack of the structured dataset to be used by the researchers. We introduce two datasets to fill this gap. The first dataset is prepared for designing a consistent, collaborator finding system. The next one, called a co-author finding model, models an academic social network as a table that contains different relations between the pair of users. Both of them provide an opportunity for introducing potential collaborators to each other. These two models have been extracted from ResearchGate (RG) data set and are available publicly. RG dataset has been collected from Jan. 2019 to April 2019 and includes raw data of 3980 RG users. The dataset consists of almost complete information about users. In the preprocessing phase, the well-known Elmo was used for analyzing textual data. We call this as ResearchGate dataset for Recommending Systems (RGRS). For assessing the validity of data, we analyze each layer of data separately, and the results are reported. After preparing data and evaluating the collaborator finding models, we have done some assessments on RGRS. Some of these assessments are co-author, following-follower, and question answering relations. The outcomes indicate that it is the best relation in propagating knowledge in the network. To the best of our knowledge, there is no processed and analyzed dataset of this size.
Uncertainty-Wizard: Fast and User-Friendly Neural Network Uncertainty Quantification
Authors: Michael Weiss, Paolo Tonella
Date: 2020-12-29 00:00:00
Tasks: Deep Learning, Uncertainty Quantification

Uncertainty and confidence have been shown to be useful metrics in a wide variety of techniques proposed for deep learning testing, including test data selection and system supervision.We present uncertainty-wizard, a tool that allows to quantify such uncertainty and confidence in artificial neural networks. It is built on top of the industry-leading tf.keras deep learning API and it provides a near-transparent and easy to understand interface. At the same time, it includes major performance optimizations that we benchmarked on two different machines and different configurations.
Data driven Dirichlet sampling on manifolds
Authors: Luan S Prado, Thiago G Ritto
Date: 2020-12-29 00:00:00
Tasks: Stochastic Optimization

This article presents a novel method to sampling on manifolds based on the Dirichlet distribution. The proposed strategy allows to completely respect the underlying manifold around which data is observed, and to do massive samplings with low computational effort. This can be very helpful, for instance, in neural networks training process, as well as in uncertainty analysis and stochastic optimization. Due to its simplicity and efficiency, we believe that the new method has great potential. Three manifolds (two dimensional ring, Mobius strip and spider geometry) are considered to test the proposed methodology, and then it is employed to an engineering application, related to gas seal coefficients.
Deep Unsupervised Identification of Selected SNPs between Adapted Populations on Pool-seq Data
Authors: Julia Siekiera, Stefan Kramer
Date: 2020-12-28 00:00:00
Tasks: Diversity, Explainable artificial intelligence

The exploration of selected single nucleotide polymorphisms (SNPs) to identify genetic diversity between different sequencing population pools (Pool-seq) is a fundamental task in genetic research. As underlying sequence reads and their alignment are error-prone and univariate statistical solutions only take individual positions of the genome into account, the identification of selected SNPs remains a challenging process. Deep learning models like convolutional neural networks (CNNs) are able to consider large input areas in their decisions. We suggest an unsupervised pipeline to be independent of a rarely known ground truth. We train a supervised discriminator CNN to distinguish alignments from different populations and utilize the model for unsupervised SNP calling by applying explainable artificial intelligence methods. Our proposed multivariate method is based on two main assumptions: We assume (i) that instances having a high predictive certainty of being distinguishable are likely to contain genetic variants, and (ii) that selected SNPs are located at regions with input features having the highest influence on the model's decision process. We directly compare our method with statistical results on two different Pool-seq datasets and show that our solution is able to extend statistical results.
Automatic-differentiated Physics-Informed Echo State Network (API-ESN)
Authors: Alberto Racca, Luca Magri
Date: 2020-12-28 00:00:00

We propose the Automatic-differentiated Physics-Informed Echo State Network (API-ESN). The network is constrained by the physical equations through the reservoir's exact time-derivative, which is computed by automatic differentiation. As compared to the original Physics-Informed Echo State Network, the accuracy of the time-derivative is increased by up to seven orders of magnitude. This increased accuracy is key in chaotic dynamical systems, where errors grows exponentially in time. The network is showcased in the reconstruction of unmeasured (hidden) states of a chaotic system. The API-ESN eliminates a source of error, which is present in existing physics-informed echo state networks, in the computation of the time-derivative. This opens up new possibilities for an accurate reconstruction of chaotic dynamical states.
Advanced Machine Learning Techniques for Fake News (Online Disinformation) Detection: A Systematic Mapping Study
Authors: Michal Choras, Konstantinos Demestichas, Agata Gielczyk, Alvaro Herrero, Pawel Ksieniewicz, Konstantina Remoundou, Daniel Urda, Michal Wozniak
Date: 2020-12-28 00:00:00
Tasks: BIG-bench Machine Learning, Fake News Detection, Misinformation

Fake news has now grown into a big problem for societies and also a major challenge for people fighting disinformation. This phenomenon plagues democratic elections, reputations of individual persons or organizations, and has negatively impacted citizens, (e.g., during the COVID-19 pandemic in the US or Brazil). Hence, developing effective tools to fight this phenomenon by employing advanced Machine Learning (ML) methods poses a significant challenge. The following paper displays the present body of knowledge on the application of such intelligent tools in the fight against disinformation. It starts by showing the historical perspective and the current role of fake news in the information war. Proposed solutions based solely on the work of experts are analysed and the most important directions of the application of intelligent systems in the detection of misinformation sources are pointed out. Additionally, the paper presents some useful resources (mainly datasets useful when assessing ML solutions for fake news detection) and provides a short overview of the most important R&D projects related to this subject. The main purpose of this work is to analyse the current state of knowledge in detecting fake news; on the one hand to show possible solutions, and on the other hand to identify the main challenges and methodological gaps to motivate future research.
Scene Text Detection for Augmented Reality -- Character Bigram Approach to reduce False Positive Rate
Authors: Sagar Gubbi, Bharadwaj Amrutur
Date: 2020-12-26 00:00:00
Tasks: Scene Text Detection, Scene Understanding, Text Detection, Text Spotting

Natural scene text detection is an important aspect of scene understanding and could be a useful tool in building engaging augmented reality applications. In this work, we address the problem of false positives in text spotting. We propose improving the performace of sliding window text spotters by looking for character pairs (bigrams) rather than single characters. An efficient convolutional neural network is designed and trained to detect bigrams. The proposed detector reduces false positive rate by 28.16% on the ICDAR 2015 dataset. We demonstrate that detecting bigrams is a computationally inexpensive way to improve sliding window text spotters.