pwc-archive/papers-with-abstracts huggingface dataset:
Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input
Authors: Brooke Stephenson, Thomas Hueber, Laurent Girin, Laurent Besacier
Date: 2021-02-19 00:00:00
Tasks: Language Modeling, Language Modelling, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis

The prosody of a spoken word is determined by its surrounding context. In incremental text-to-speech synthesis, where the synthesizer produces an output before it has access to the complete input, the full context is often unknown which can result in a loss of naturalness in the synthesized speech. In this paper, we investigate whether the use of predicted future text can attenuate this loss. We compare several test conditions of next future word: (a) unknown (zero-word), (b) language model predicted, (c) randomly predicted and (d) ground-truth. We measure the prosodic features (pitch, energy and duration) and find that predicted text provides significant improvements over a zero-word lookahead, but only slight gains over random-word lookahead. We confirm these results with a perceptive test.
Back to Prior Knowledge: Joint Event Causality Extraction via Convolutional Semantic Infusion
Authors: Zijian Wang, Hao Wang, Xiangfeng Luo, Jianqi Gao
Date: 2021-02-19 00:00:00
Tasks: Information Retrieval, Retrieval

Joint event and causality extraction is a challenging yet essential task in information retrieval and data mining. Recently, pre-trained language models (e.g., BERT) yield state-of-the-art results and dominate in a variety of NLP tasks. However, these models are incapable of imposing external knowledge in domain-specific extraction. Considering the prior knowledge of frequent n-grams that represent cause/effect events may benefit both event and causality extraction, in this paper, we propose convolutional knowledge infusion for frequent n-grams with different windows of length within a joint extraction framework. Knowledge infusion during convolutional filter initialization not only helps the model capture both intra-event (i.e., features in an event cluster) and inter-event (i.e., associations across event clusters) features but also boosts training convergence. Experimental results on the benchmark datasets show that our model significantly outperforms the strong BERT+CSNN baseline.
Towards Emotion Recognition in Hindi-English Code-Mixed Data: A Transformer Based Approach
Authors: Anshul Wadhawan, Akshita Aggarwal
Date: 2021-02-19 00:00:00
Proceeding: EACL (WASSA) 2021 4
Tasks: Emotion Recognition, Word Embeddings

In the last few years, emotion detection in social-media text has become a popular problem due to its wide ranging application in better understanding the consumers, in psychology, in aiding human interaction with computers, designing smart systems etc. Because of the availability of huge amounts of data from social-media, which is regularly used for expressing sentiments and opinions, this problem has garnered great attention. In this paper, we present a Hinglish dataset labelled for emotion detection. We highlight a deep learning based approach for detecting emotions in Hindi-English code mixed tweets, using bilingual word embeddings derived from FastText and Word2Vec approaches, as well as transformer based models. We experiment with various deep learning models, including CNNs, LSTMs, Bi-directional LSTMs (with and without attention), along with transformers like BERT, RoBERTa, and ALBERT. The transformer based BERT model outperforms all other models giving the best performance with an accuracy of 71.43%.
Privacy-Preserving Graph Convolutional Networks for Text Classification
Authors: Timour Igamberdiev, Ivan Habernal
Date: 2021-02-10 00:00:00
Proceeding: LREC 2022 6
Tasks: Classification, General Classification, Privacy Preserving, Representation Learning, text-classification, Text Classification

Graph convolutional networks (GCNs) are a powerful architecture for representation learning on documents that naturally occur as graphs, e.g., citation or social networks. However, sensitive personal information, such as documents with people's profiles or relationships as edges, are prone to privacy leaks, as the trained model might reveal the original input. Although differential privacy (DP) offers a well-founded privacy-preserving framework, GCNs pose theoretical and practical challenges due to their training specifics. We address these challenges by adapting differentially-private gradient-based training to GCNs and conduct experiments using two optimizers on five NLP datasets in two languages. We propose a simple yet efficient method based on random graph splits that not only improves the baseline privacy bounds by a factor of 2.7 while retaining competitive F1 scores, but also provides strong privacy guarantees of epsilon = 1.0. We show that, under certain modeling choices, privacy-preserving GCNs perform up to 90% of their non-private variants, while formally guaranteeing strong privacy measures.
Scaling Creative Inspiration with Fine-Grained Functional Aspects of Ideas
Authors: Tom Hope, Ronen Tamari, Hyeonsu Kang, Daniel Hershcovich, Joel Chan, Aniket Kittur, Dafna Shahaf
Date: 2021-02-19 00:00:00

Large repositories of products, patents and scientific papers offer an opportunity for building systems that scour millions of ideas and help users discover inspirations. However, idea descriptions are typically in the form of unstructured text, lacking key structure that is required for supporting creative innovation interactions. Prior work has explored idea representations that were either limited in expressivity, required significant manual effort from users, or dependent on curated knowledge bases with poor coverage. We explore a novel representation that automatically breaks up products into fine-grained functional aspects capturing the purposes and mechanisms of ideas, and use it to support important creative innovation interactions: functional search for ideas, and exploration of the design space around a focal problem by viewing related problem perspectives pooled from across many products. In user studies, our approach boosts the quality of creative search and inspirations, substantially outperforming strong baselines by 50-60%.
Do End-to-End Speech Recognition Models Care About Context?
Authors: Lasse Borgholt, Jakob Drachmann Havtorn, Željko Agić, Anders Søgaard, Lars Maaløe, Christian Igel
Date: 2021-02-17 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

The two most common paradigms for end-to-end speech recognition are connectionist temporal classification (CTC) and attention-based encoder-decoder (AED) models. It has been argued that the latter is better suited for learning an implicit language model. We test this hypothesis by measuring temporal context sensitivity and evaluate how the models perform when we constrain the amount of contextual information in the audio input. We find that the AED model is indeed more context sensitive, but that the gap can be closed by adding self-attention to the CTC model. Furthermore, the two models perform similarly when contextual information is constrained. Finally, in contrast to previous research, our results show that the CTC model is highly competitive on WSJ and LibriSpeech without the help of an external language model.
An Enhanced Corpus for Arabic Newspapers Comments
Authors: Hichem Rahab, Abdelhafid Zitouni, Mahieddine Djoudi
Date: 2021-02-08 00:00:00
Tasks: Classification, General Classification

In this paper, we propose our enhanced approach to create a dedicated corpus for Algerian Arabic newspapers comments. The developed approach has to enhance an existing approach by the enrichment of the available corpus and the inclusion of the annotation step by following the Model Annotate Train Test Evaluate Revise (MATTER) approach. A corpus is created by collecting comments from web sites of three well know Algerian newspapers. Three classifiers, support vector machines, na{\"i}ve Bayes, and k-nearest neighbors, were used for classification of comments into positive and negative classes. To identify the influence of the stemming in the obtained results, the classification was tested with and without stemming. Obtained results show that stemming does not enhance considerably the classification due to the nature of Algerian comments tied to Algerian Arabic Dialect. The promising results constitute a motivation for us to improve our approach especially in dealing with non Arabic sentences, especially Dialectal and French ones.
Maximizing Marginal Fairness for Dynamic Learning to Rank
Authors: Tao Yang, Qingyao Ai
Date: 2021-02-18 00:00:00
Tasks: Fairness, Learning-To-Rank, Recommendation Systems

Rankings, especially those in search and recommendation systems, often determine how people access information and how information is exposed to people. Therefore, how to balance the relevance and fairness of information exposure is considered as one of the key problems for modern IR systems. As conventional ranking frameworks that myopically sorts documents with their relevance will inevitably introduce unfair result exposure, recent studies on ranking fairness mostly focus on dynamic ranking paradigms where result rankings can be adapted in real-time to support fairness in groups (i.e., races, genders, etc.). Existing studies on fairness in dynamic learning to rank, however, often achieve the overall fairness of document exposure in ranked lists by significantly sacrificing the performance of result relevance and fairness on the top results. To address this problem, we propose a fair and unbiased ranking method named Maximal Marginal Fairness (MMF). The algorithm integrates unbiased estimators for both relevance and merit-based fairness while providing an explicit controller that balances the selection of documents to maximize the marginal relevance and fairness in top-k results. Theoretical and empirical analysis shows that, with small compromises on long list fairness, our method achieves superior efficiency and effectiveness comparing to the state-of-the-art algorithms in both relevance and fairness for top-k rankings.
Identifying and Mapping the Global Research Output on Coronavirus Disease: A Scientometric Study
Authors: Muneer Ahmad, Dr. M Sadik Batcha
Date: 2021-02-19 00:00:00
Tasks: Articles

The paper explores and analyses the trend of world literature on "Coronavirus Disease" in terms of the output of research publications as indexed in the Science Citation Index Expanded (SCI-E) of Web of Science during the period from 2011 to 2020. The study found that 6071 research records have been published on Coronavirus Disease till March 20, 2020. The various scientometric components of the research records published in the study period were studied. The study reveals the various aspects of Coronavirus Disease literature such as year wise distribution, relative growth rate, doubling time of literature, geographical wise, organization wise, language wise, form wise , most prolific authors, and source wise. The highest number of articles was published in the year 2019, while lowest numbers of research article were reported in the year 2020. Further, the relative growth rate is gradually increases and on the other hand doubling time decreases. Most of the research publications are published in English language and most of the publications published in the form of research articles. USA is the highest contributor to the field of Coronavirus Disease literature.
Modeling epigenetic evolutionary algorithms: An approach based on the epigenetic regulation process
Authors: Alvarez Lifeth
Date: 2021-02-18 00:00:00
Tasks: Decoder, Evolutionary Algorithms

Many biological processes have been the source of inspiration for heuristic methods that generate high-quality solutions to solve optimization and search problems. This thesis presents an epigenetic technique for Evolutionary Algorithms, inspired by the epigenetic regulation process, a mechanism to better understand the ability of individuals to adapt and learn from the environment. Epigenetic regulation comprises biological mechanisms by which small molecules, also known as epigenetic tags, are attached to or removed from a particular gene, affecting the phenotype. Five fundamental elements form the basis of the designed technique: first, a metaphorical representation of Epigenetic Tags as binary strings; second, a layer on chromosome top structure used to bind the tags (the Epigenotype layer); third, a Marking Function to add, remove, and modify tags; fourth, an Epigenetic Growing Function that acts like an interpreter, or decoder of the tags located over the alleles, in such a way that the phenotypic variations can be reflected when evaluating the individuals; and fifth, a tags inheritance mechanism. A set of experiments are performed for determining the applicability of the proposed approach.
Training a Resilient Q-Network against Observational Interference
Authors: Chao-Han Huck Yang, I-Te Danny Hung, Yi Ouyang, Pin-Yu Chen
Date: 2021-02-18 00:00:00
Tasks: Causal Inference, Deep Reinforcement Learning

Deep reinforcement learning (DRL) has demonstrated impressive performance in various gaming simulators and real-world applications. In practice, however, a DRL agent may receive faulty observation by abrupt interferences such as black-out, frozen-screen, and adversarial perturbation. How to design a resilient DRL algorithm against these rare but mission-critical and safety-crucial scenarios is an essential yet challenging task. In this paper, we consider a deep q-network (DQN) framework training with an auxiliary task of observational interferences such as artificial noises. Inspired by causal inference for observational interference, we propose a causal inference based DQN algorithm called causal inference Q-network (CIQ). We evaluate the performance of CIQ in several benchmark DQN environments with different types of interferences as auxiliary labels. Our experimental results show that the proposed CIQ method could achieve higher performance and more resilience against observational interferences.
Training Neural Networks is $\exists\mathbb R$-complete
Authors: Mikkel Abrahamsen, Linda Kleist, Tillmann Miltzow
Date: 2021-02-19 00:00:00
Proceeding: NeurIPS 2021 12

Given a neural network, training data, and a threshold, it was known that it is NP-hard to find weights for the neural network such that the total error is below the threshold. We determine the algorithmic complexity of this fundamental problem precisely, by showing that it is $\exists\mathbb R$-complete. This means that the problem is equivalent, up to polynomial-time reductions, to deciding whether a system of polynomial equations and inequalities with integer coefficients and real unknowns has a solution. If, as widely expected, $\exists\mathbb R$ is strictly larger than NP, our work implies that the problem of training neural networks is not even in NP. Neural networks are usually trained using some variation of backpropagation. The result of this paper offers an explanation why techniques commonly used to solve big instances of NP-complete problems seem not to be of use for this task. Examples of such techniques are SAT solvers, IP solvers, local search, dynamic programming, to name a few general ones.
A bi-level encoding scheme for the clustered shortest-path tree problem in multifactorial optimization
Authors: Huynh Thi Thanh Binh, Ta Bao Thang, Nguyen Duc Thai, Pham Dinh Thanh
Date: 2021-02-12 00:00:00
Tasks: valid

The Clustered Shortest-Path Tree Problem (CluSPT) plays an important role in various types of optimization problems in real-life. Recently, some Multifactorial Evolutionary Algorithm (MFEA) have been introduced to deal with the CluSPT, however these researches still have some shortcomings such as evolution operators only perform on complete graphs, huge resource consumption for finding the solution on large search spaces. To overcome these limitations, this paper describes a MFEA-based approach to solve the CluSPT. The proposed algorithm utilizes Dijkstra's algorithm to construct the spanning trees in clusters while using evolutionary operators for building the spanning tree connecting clusters. This approach takes advantage of both exact and approximate algorithms so it enables the algorithm to function efficiently on complete and sparse graphs alike. Furthermore, evolutionary operators such as individual encoding and decoding methods are also designed with great consideration regarding performance and memory usage. We have included a proof on the repairing method's efficacy in ensuring all solutions are valid. We have conducted tests on various types of Euclidean instances to assess the effectiveness of the proposed algorithm and methods. Experiment results point out the effectiveness of the proposed algorithm existing heuristic algorithms in most of the test cases. The impact of the proposed MFEA was analyzed and a possible influential factor that may be useful for further study was also pointed out.
Implicit Regularization in Tensor Factorization
Authors: Noam Razin, Asaf Maman, Nadav Cohen
Date: 2021-02-19 00:00:00
Tasks: Deep Learning, Matrix Completion

Recent efforts to unravel the mystery of implicit regularization in deep learning have led to a theoretical focus on matrix factorization -- matrix completion via linear neural network. As a step further towards practical deep learning, we provide the first theoretical analysis of implicit regularization in tensor factorization -- tensor completion via certain type of non-linear neural network. We circumvent the notorious difficulty of tensor problems by adopting a dynamical systems perspective, and characterizing the evolution induced by gradient descent. The characterization suggests a form of greedy low tensor rank search, which we rigorously prove under certain conditions, and empirically demonstrate under others. Motivated by tensor rank capturing the implicit regularization of a non-linear neural network, we empirically explore it as a measure of complexity, and find that it captures the essence of datasets on which neural networks generalize. This leads us to believe that tensor rank may pave way to explaining both implicit regularization in deep learning, and the properties of real-world data translating this implicit regularization to generalization.
A Deep Embedded Refined Clustering Approach for Breast Cancer Distinction based on DNA Methylation
Authors: del Amor Rocío, Colomer Adrián, Monteagudo Carlos, Naranjo Valery
Date: 2021-02-18 00:00:00
Tasks: Cancer Classification, Clustering, Dimensionality Reduction, General Classification

Epigenetic alterations have an important role in the development of several types of cancer. Epigenetic studies generate a large amount of data, which makes it essential to develop novel models capable of dealing with large-scale data. In this work, we propose a deep embedded refined clustering method for breast cancer differentiation based on DNA methylation. In concrete, the deep learning system presented here uses the levels of CpG island methylation between 0 and 1. The proposed approach is composed of two main stages. The first stage consists in the dimensionality reduction of the methylation data based on an autoencoder. The second stage is a clustering algorithm based on the soft-assignment of the latent space provided by the autoencoder. The whole method is optimized through a weighted loss function composed of two terms: reconstruction and classification terms. To the best of the authors' knowledge, no previous studies have focused on the dimensionality reduction algorithms linked to classification trained end-to-end for DNA methylation analysis. The proposed method achieves an unsupervised clustering accuracy of 0.9927 and an error rate (%) of 0.73 on 137 breast tissue samples. After a second test of the deep-learning-based method using a different methylation database, an accuracy of 0.9343 and an error rate (%) of 6.57 on 45 breast tissue samples is obtained. Based on these results, the proposed algorithm outperforms other state-of-the-art methods evaluated under the same conditions for breast cancer classification based on DNA methylation data.
Interpretable Stability Bounds for Spectral Graph Filters
Authors: Henry Kenlay, Dorina Thanou, Xiaowen Dong
Date: 2021-02-18 00:00:00
Tasks: Anomaly Detection, Denoising

Graph-structured data arise in a variety of real-world context ranging from sensor and transportation to biological and social networks. As a ubiquitous tool to process graph-structured data, spectral graph filters have been used to solve common tasks such as denoising and anomaly detection, as well as design deep learning architectures such as graph neural networks. Despite being an important tool, there is a lack of theoretical understanding of the stability properties of spectral graph filters, which are important for designing robust machine learning models. In this paper, we study filter stability and provide a novel and interpretable upper bound on the change of filter output, where the bound is expressed in terms of the endpoint degrees of the deleted and newly added edges, as well as the spatial proximity of those edges. This upper bound allows us to reason, in terms of structural properties of the graph, when a spectral graph filter will be stable. We further perform extensive experiments to verify intuition that can be gained from the bound.
A Differential Geometry Perspective on Orthogonal Recurrent Models
Authors: Omri Azencot, N. Benjamin Erichson, Mirela Ben-Chen, Michael W. Mahoney
Date: 2021-02-18 00:00:00

Recently, orthogonal recurrent neural networks (RNNs) have emerged as state-of-the-art models for learning long-term dependencies. This class of models mitigates the exploding and vanishing gradients problem by design. In this work, we employ tools and insights from differential geometry to offer a novel perspective on orthogonal RNNs. We show that orthogonal RNNs may be viewed as optimizing in the space of divergence-free vector fields. Specifically, based on a well-known result in differential geometry that relates vector fields and linear operators, we prove that every divergence-free vector field is related to a skew-symmetric matrix. Motivated by this observation, we study a new recurrent model, which spans the entire space of vector fields. Our method parameterizes vector fields via the directional derivatives of scalar functions. This requires the construction of latent inner product, gradient, and divergence operators. In comparison to state-of-the-art orthogonal RNNs, our approach achieves comparable or better results on a variety of benchmark tasks.