pwc-archive/papers-with-abstracts huggingface dataset:
Complementary Composite Minimization, Small Gradients in General Norms, and Applications
Authors: Jelena Diakonikolas, Cristóbal Guzmán
Date: 2021-01-26 00:00:00
Tasks: regression

Composite minimization is a powerful framework in large-scale convex optimization, based on decoupling of the objective function into terms with structurally different properties and allowing for more flexible algorithmic design. We introduce a new algorithmic framework for complementary composite minimization, where the objective function decouples into a (weakly) smooth and a uniformly convex term. This particular form of decoupling is pervasive in statistics and machine learning, due to its link to regularization. The main contributions of our work are summarized as follows. First, we introduce the problem of complementary composite minimization in general normed spaces; second, we provide a unified accelerated algorithmic framework to address broad classes of complementary composite minimization problems; and third, we prove that the algorithms resulting from our framework are near-optimal in most of the standard optimization settings. Additionally, we show that our algorithmic framework can be used to address the problem of making the gradients small in general normed spaces. As a concrete example, we obtain a nearly-optimal method for the standard $\ell_1$ setup (small gradients in the $\ell_{\infty}$ norm), essentially matching the bound of Nesterov (2012) that was previously known only for the Euclidean setup. Finally, we show that our composite methods are broadly applicable to a number of regression and other classes of optimization problems, where regularization plays a key role. Our methods lead to complexity bounds that are either new or match the best existing ones.
A Comparison of Approaches to Document-level Machine Translation
Authors: Zhiyi Ma, Sergey Edunov, Michael Auli
Date: 2021-01-26 00:00:00
Tasks: Document Level Machine Translation, Machine Translation, Translation

Document-level machine translation conditions on surrounding sentences to produce coherent translations. There has been much recent work in this area with the introduction of custom model architectures and decoding algorithms. This paper presents a systematic comparison of selected approaches from the literature on two benchmarks for which document-level phenomena evaluation suites exist. We find that a simple method based purely on back-translating monolingual document-level data performs as well as much more elaborate alternatives, both in terms of document-level metrics as well as human evaluation.
Muppet: Massive Multi-task Representations with Pre-Finetuning
Authors: Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, Sonal Gupta
Date: 2021-01-26 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Abstractive Text Summarization, Common Sense Reasoning, Language Modeling, Language Modelling, Multi-Task Learning, Natural Language Inference, Question Answering, Sentence, Sentence Completion, Sentiment Analysis, Text Summarization

We propose pre-finetuning, an additional large-scale learning stage between language model pre-training and fine-tuning. Pre-finetuning is massively multi-task learning (around 50 datasets, over 4.8 million total labeled examples), and is designed to encourage learning of representations that generalize better to many different tasks. We show that pre-finetuning consistently improves performance for pretrained discriminators (e.g.~RoBERTa) and generation models (e.g.~BART) on a wide range of tasks (sentence prediction, commonsense reasoning, MRC, etc.), while also significantly improving sample efficiency during fine-tuning. We also show that large-scale multi-tasking is crucial; pre-finetuning can hurt performance when few tasks are used up until a critical point (usually above 15) after which performance improves linearly in the number of tasks.
Average Localised Proximity: A new data descriptor with good default one-class classification performance
Authors: Oliver Urs Lenz, Daniel Peralta, Chris Cornelis
Date: 2021-01-26 00:00:00
Tasks: Classification, General Classification, One-Class Classification

One-class classification is a challenging subfield of machine learning in which so-called data descriptors are used to predict membership of a class based solely on positive examples of that class, and no counter-examples. A number of data descriptors that have been shown to perform well in previous studies of one-class classification, like the Support Vector Machine (SVM), require setting one or more hyperparameters. There has been no systematic attempt to date to determine optimal default values for these hyperparameters, which limits their ease of use, especially in comparison with hyperparameter-free proposals like the Isolation Forest (IF). We address this issue by determining optimal default hyperparameter values across a collection of 246 one-class classification problems derived from 50 different real-world datasets. In addition, we propose a new data descriptor, Average Localised Proximity (ALP) to address certain issues with existing approaches based on nearest neighbour distances. Finally, we evaluate classification performance using a leave-one-dataset-out procedure, and find strong evidence that ALP outperforms IF and a number of other data descriptors, as well as weak evidence that it outperforms SVM, making ALP a good default choice.
On formal concepts of random formal contexts
Authors: Taro Sakurai
Date: 2021-01-26 00:00:00

In formal concept analysis, it is well-known that the number of formal concepts can be exponential in the worst case. To analyze the average case, we introduce a probabilistic model for random formal contexts and prove that the average number of formal concepts has a superpolynomial asymptotic lower bound.
Supervised quantum machine learning models are kernel methods
Authors: Maria Schuld
Date: 2021-01-26 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning

With near-term quantum devices available and the race for fault-tolerant quantum computers in full swing, researchers became interested in the question of what happens if we replace a supervised machine learning model with a quantum circuit. While such "quantum models" are sometimes called "quantum neural networks", it has been repeatedly noted that their mathematical structure is actually much more closely related to kernel methods: they analyse data in high-dimensional Hilbert spaces to which we only have access through inner products revealed by measurements. This technical manuscript summarises and extends the idea of systematically rephrasing supervised quantum models as a kernel method. With this, a lot of near-term and fault-tolerant quantum models can be replaced by a general support vector machine whose kernel computes distances between data-encoding quantum states. Kernel-based training is then guaranteed to find better or equally good quantum models than variational circuit training. Overall, the kernel perspective of quantum machine learning tells us that the way that data is encoded into quantum states is the main ingredient that can potentially set quantum models apart from classical machine learning models.
Synthetic Generation of Three-Dimensional Cancer Cell Models from Histopathological Images
Authors: Yoav Alon, Xiang Yu, Huiyu Zhou
Date: 2021-01-26 00:00:00
Tasks: Decoder, Image Registration, Style Transfer

Synthetic generation of three-dimensional cell models from histopathological images aims to enhance understanding of cell mutation, and progression of cancer, necessary for clinical assessment and optimal treatment. Classical reconstruction algorithms based on image registration of consecutive slides of stained tissues are prone to errors and often not suitable for the training of three-dimensional segmentation algorithms. We propose a novel framework to generate synthetic three-dimensional histological models based on a generator-discriminator pattern optimizing constrained features that construct a 3D model via a Blender interface exploiting smooth shape continuity typical for biological specimens. To capture the spatial context of entire cell clusters we deploy a novel deep topology transformer that implements and attention mechanism on cell group images to extract features for the frozen feature decoder. The proposed algorithms achieves high quantitative and qualitative synthesis evident in comparative evaluation metrics such as a low Frechet-Inception scores.
FDApy: a Python package for functional data
Authors: Steven Golovkine
Date: 2021-01-26 00:00:00
Tasks: Clustering, Dimensionality Reduction

We introduce FDApy, an open-source Python package for the analysis of functional data. The package provides tools for the representation of (multivariate) functional data defined on different dimensional domains and for functional data that is irregularly sampled. Additionally, dimension reduction techniques are implemented for multivariate and/or multidimensional functional data that are regularly or irregularly sampled. A toolbox for generating functional datasets is also provided. The documentation includes installation and usage instructions, examples on simulated and real datasets and a complete description of the API. FDApy is released under the MIT license. The code and documentation are available at https://github.com/StevenGolovkine/FDApy.
Identification of brain states, transitions, and communities using functional MRI
Authors: Lingbin Bian, Tiangang Cui, B. T. Thomas Yeo, Alex Fornito, Adeel Razi, Jonathan Keith
Date: 2021-01-26 00:00:00
Tasks: Time Series, Time Series Analysis

Brain function relies on a precisely coordinated and dynamic balance between the functional integration and segregation of distinct neural systems. Characterizing the way in which neural systems reconfigure their interactions to give rise to distinct but hidden brain states remains an open challenge. In this paper, we propose a Bayesian model-based characterization of latent brain states and showcase a novel method based on posterior predictive discrepancy using the latent block model to detect transitions between latent brain states in blood oxygen level-dependent (BOLD) time series. The set of estimated parameters in the model includes a latent label vector that assigns network nodes to communities, and also block model parameters that reflect the weighted connectivity within and between communities. Besides extensive in-silico model evaluation, we also provide empirical validation (and replication) using the Human Connectome Project (HCP) dataset of 100 healthy adults. Our results obtained through an analysis of task-fMRI data during working memory performance show appropriate lags between external task demands and change-points between brain states, with distinctive community patterns distinguishing fixation, low-demand and high-demand task conditions.
Magnetic Resonance Spectroscopy Deep Learning Denoising Using Few In Vivo Data
Authors: Dicheng Chen, Wanqi Hu, Huiting Liu, Yirong Zhou, Tianyu Qiu, Yihui Huang, Zi Wang, Jiazheng Wang, Liangjie Lin, Zhigang Wu, Hao Chen, Xi Chen, Gen Yan, Di Guo, Jianzhong Lin, Xiaobo Qu
Date: 2021-01-26 00:00:00
Tasks: Deep Learning, Denoising

Magnetic Resonance Spectroscopy (MRS) is a noninvasive tool to reveal metabolic information. One challenge of 1H-MRS is the low Signal-Noise Ratio (SNR). To improve the SNR, a typical approach is to perform Signal Averaging (SA) with M repeated samples. The data acquisition time, however, is increased by M times accordingly, and a complete clinical MRS scan takes approximately 10 minutes at a common setting M=128. Recently, deep learning has been introduced to improve the SNR but most of them use the simulated data as the training set. This may hinder the MRS applications since some potential differences, such as acquisition system imperfections, and physiological and psychologic conditions may exist between the simulated and in vivo data. Here, we proposed a new scheme that purely used the repeated samples of realistic data. A deep learning model, Refusion Long Short-Term Memory (ReLSTM), was designed to learn the mapping from the low SNR time-domain data (24 SA) to the high SNR one (128 SA). Experiments on the in vivo brain spectra of 7 healthy subjects, 2 brain tumor patients and 1 cerebral infarction patient showed that only using 20% repeated samples, the denoised spectra by ReLSTM could provide comparable estimated concentrations of metabolites to 128 SA. Compared with the state-of-the-art low-rank denoising method, the ReLSTM achieved the lower relative error and the Cram\'er-Rao lower bounds in quantifying some important biomarkers. In summary, ReLSTM can perform high-fidelity denoising of the spectra under fast acquisition (24 SA), which would be valuable to MRS clinical studies.
Recent Trends in Named Entity Recognition (NER)
Authors: Arya Roy
Date: 2021-01-25 00:00:00
Tasks: Deep Learning, Entity Linking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

The availability of large amounts of computer-readable textual data and hardware that can process the data has shifted the focus of knowledge projects towards deep learning architecture. Natural Language Processing, particularly the task of Named Entity Recognition is no exception. The bulk of the learning methods that have produced state-of-the-art results have changed the deep learning model, the training method used, the training data itself or the encoding of the output of the NER system. In this paper, we review significant learning methods that have been employed for NER in the recent past and how they came about from the linear learning methods of the past. We also cover the progress of related tasks that are upstream or downstream to NER, e.g., sequence tagging, entity linking, etc., wherever the processes in question have also improved NER results.
Pseudo-Adaptive Penalization to Handle Constraints in Particle Swarm Optimizers
Authors: Mauro S. Innocente, Johann Sienz
Date: 2021-01-25 00:00:00

The penalization method is a popular technique to provide particle swarm optimizers with the ability to handle constraints. The downside is the need of penalization coefficients whose settings are problem-specific. While adaptive coefficients can be found in the literature, a different adaptive scheme is proposed in this paper, where coefficients are kept constant. A pseudo-adaptive relaxation of the tolerances for constraint violations while penalizing only violations beyond such tolerances results in a pseudo-adaptive penalization. A particle swarm optimizer is tested on a suite of benchmark problems for three types of tolerance relaxation: no relaxation; self-tuned initial relaxation with deterministic decrease; and self-tuned initial relaxation with pseudo-adaptive decrease. Other authors' results are offered as frames of reference.
Individual and Social Behaviour in Particle Swarm Optimizers
Authors: Johann Sienz, Mauro S. Innocente
Date: 2021-01-25 00:00:00

Three basic factors govern the individual behaviour of a particle: the inertia from its previous displacement; the attraction to its own best experience; and the attraction to a given neighbour's best experience. The importance awarded to each factor is controlled by three coefficients: the inertia; the individuality; and the sociality weights. The social behaviour is ruled by the structure of the social network, which defines the neighbours that are to inform of their experiences to a given particle. This paper presents a study of the influence of different settings of the coefficients as well as of the combined effect of different settings and different neighbourhood topologies on the speed and form of convergence.
Particle Swarm Optimization: Fundamental Study and its Application to Optimization and to Jetty Scheduling Problems
Authors: Johann Sienz, Mauro S. Innocente
Date: 2021-01-25 00:00:00
Tasks: Combinatorial Optimization, Evolutionary Algorithms, Scheduling

The advantages of evolutionary algorithms with respect to traditional methods have been greatly discussed in the literature. While particle swarm optimizers share such advantages, they outperform evolutionary algorithms in that they require lower computational cost and easier implementation, involving no operator design and few coefficients to be tuned. However, even marginal variations in the settings of these coefficients greatly influence the dynamics of the swarm. Since this paper does not intend to study their tuning, general-purpose settings are taken from previous studies, and virtually the same algorithm is used to optimize a variety of notably different problems. Thus, following a review of the paradigm, the algorithm is tested on a set of benchmark functions and engineering problems taken from the literature. Later, complementary lines of code are incorporated to adapt the method to combinatorial optimization as it occurs in scheduling problems, and a real case is solved using the same optimizer with the same settings. The aim is to show the flexibility and robustness of the approach, which can handle a wide variety of problems.
SkillNER: Mining and Mapping Soft Skills from any Text
Authors: Silvia Fareri, Nicola Melluso, Filippo Chiarello, Gualtiero Fantoni
Date: 2021-01-22 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

In today's digital world, there is an increasing focus on soft skills. On the one hand, they facilitate innovation at companies, but on the other, they are unlikely to be automated soon. Researchers struggle with accurately approaching quantitatively the study of soft skills due to the lack of data-driven methods to retrieve them. This limits the possibility for psychologists and HR managers to understand the relation between humans and digitalisation. This paper presents SkillNER, a novel data-driven method for automatically extracting soft skills from text. It is a named entity recognition (NER) system trained with a support vector machine (SVM) on a corpus of more than 5000 scientific papers. We developed this system by measuring the performance of our approach against different training models and validating the results together with a team of psychologists. Finally, SkillNER was tested in a real-world case study using the job descriptions of ESCO (European Skill/Competence Qualification and Occupation) as textual source. The system enabled the detection of communities of job profiles based on their shared soft skills and communities of soft skills based on their shared job profiles. This case study demonstrates that the tool can automatically retrieve soft skills from a large corpus in an efficient way, proving useful for firms, institutions, and workers. The tool is open and available online to foster quantitative methods for the study of soft skills.
Using Finite-State Machines to Automatically Scan Classical Greek Hexameter
Authors: Anne-Kathrin Schumann, Christoph Beierle, Norbert Blößner
Date: 2021-01-22 00:00:00
Tasks: valid

This paper presents a fully automatic approach to the scansion of Classical Greek hexameter verse. In particular, the paper describes an algorithm that uses deterministic finite-state automata and local linguistic rules to implement a targeted search for valid spondeus patterns and, in addition, a weighted finite-state transducer to correct and complete partial analyses and to reject invalid candidates. The paper also details the results of an empirical evaluation of the annotation quality resulting from this approach on hand-annotated data. It is shown that a finite-state approach provides quick and linguistically sound analyses of hexameter verses as well as an efficient formalisation of linguistic knowledge. The project code is available (see https://github.com/anetschka/greek_scansion).
Challenges Encountered in Turkish Natural Language Processing Studies
Authors: Kadir Tohma, Yakup Kutlu
Date: 2021-01-21 00:00:00
Tasks: Diversity

Natural language processing is a branch of computer science that combines artificial intelligence with linguistics. It aims to analyze a language element such as writing or speaking with software and convert it into information. Considering that each language has its own grammatical rules and vocabulary diversity, the complexity of the studies in this field is somewhat understandable. For instance, Turkish is a very interesting language in many ways. Examples of this are agglutinative word structure, consonant/vowel harmony, a large number of productive derivational morphemes (practically infinite vocabulary), derivation and syntactic relations, a complex emphasis on vocabulary and phonological rules. In this study, the interesting features of Turkish in terms of natural language processing are mentioned. In addition, summary info about natural language processing techniques, systems and various sources developed for Turkish are given.
Online LDA based brain-computer interface system to aid disabled people
Authors: Apdullah Yayık, Yakup Kutlu
Date: 2021-01-21 00:00:00
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG)

This paper aims to develop brain-computer interface system based on electroencephalography that can aid disabled people in daily life. The system relies on one of the most effective event-related potential wave, P300, which can be elicited by oddball paradigm. Developed application has a basic interaction tool that enables disabled people to convey their needs to other people selecting related objects. These objects pseudo-randomly flash in a visual interface on computer screen. The user must focus on related object to convey desired needs. The system can convey desired needs correctly by detecting P300 wave in acquired 14-channel EEG signal and classifying using linear discriminant analysis classifier just in 15 seconds. Experiments have been carried out on 19 volunteers to validate developed BCI system. As a result, accuracy rate of 90.83% is achieved in online performance
Analysis of Basic Emotions in Texts Based on BERT Vector Representation
Authors: A. Artemov, A. Veselovskiy, I. Khasenevich, I. Bolokhov
Date: 2021-01-21 00:00:00
Tasks: Emotion Recognition

In the following paper the authors present a GAN-type model and the most important stages of its development for the task of emotion recognition in text. In particular, we propose an approach for generating a synthetic dataset of all possible emotions combinations based on manually labelled incomplete data.
Exploratory Arabic Offensive Language Dataset Analysis
Authors: Fatemah Husain, Ozlem Uzuner
Date: 2021-01-20 00:00:00

This paper adding more insights towards resources and datasets used in Arabic offensive language research. The main goal of this paper is to guide researchers in Arabic offensive language in selecting appropriate datasets based on their content, and in creating new Arabic offensive language resources to support and complement the available ones.
A Taylor Based Sampling Scheme for Machine Learning in Computational Physics
Authors: Paul Novello, Gaël Poëtte, David Lugato, Pietro Congedo
Date: 2021-01-20 00:00:00
Tasks: BIG-bench Machine Learning, Physical Simulations

Machine Learning (ML) is increasingly used to construct surrogate models for physical simulations. We take advantage of the ability to generate data using numerical simulations programs to train ML models better and achieve accuracy gain with no performance cost. We elaborate a new data sampling scheme based on Taylor approximation to reduce the error of a Deep Neural Network (DNN) when learning the solution of an ordinary differential equations (ODE) system.