pwc-archive/papers-with-abstracts huggingface dataset:
Extractive Summarization of Long Documents by Combining Global and Local Context
Authors: Wen Xiao, Giuseppe Carenini
Date: 2019-09-17 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Extractive Summarization, Text Summarization

In this paper, we propose a novel neural single document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic. We evaluate the model on two datasets of scientific papers, Pubmed and arXiv, where it outperforms previous work, both extractive and abstractive models, on ROUGE-1, ROUGE-2 and METEOR scores. We also show that, consistently with our goal, the benefits of our method become stronger as we apply it to longer documents. Rather surprisingly, an ablation study indicates that the benefits of our model seem to come exclusively from modeling the local context, even for the longest documents.
DOVER: A Method for Combining Diarization Outputs
Authors: Andreas Stolcke, Takuya Yoshioka
Date: 2019-09-17 00:00:00
Tasks: speech-recognition, Speech Recognition

Speech recognition and other natural language tasks have long benefited from voting-based algorithms as a method to aggregate outputs from several systems to achieve a higher accuracy than any of the individual systems. Diarization, the task of segmenting an audio stream into speaker-homogeneous and co-indexed regions, has so far not seen the benefit of this strategy because the structure of the task does not lend itself to a simple voting approach. This paper presents DOVER (diarization output voting error reduction), an algorithm for weighted voting among diarization hypotheses, in the spirit of the ROVER algorithm for combining speech recognition hypotheses. We evaluate the algorithm for diarization of meeting recordings with multiple microphones, and find that it consistently reduces diarization error rate over the average of results from individual channels, and often improves on the single best channel chosen by an oracle.
Simultaneous Speech Recognition and Speaker Diarization for Monaural Dialogue Recordings with Target-Speaker Acoustic Models
Authors: Naoyuki Kanda, Shota Horiguchi, Yusuke Fujita, Yawen Xue, Kenji Nagamatsu, Shinji Watanabe
Date: 2019-09-17 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Clustering, speaker-diarization, Speaker Diarization, speech-recognition, Speech Recognition

This paper investigates the use of target-speaker automatic speech recognition (TS-ASR) for simultaneous speech recognition and speaker diarization of single-channel dialogue recordings. TS-ASR is a technique to automatically extract and recognize only the speech of a target speaker given a short sample utterance of that speaker. One obvious drawback of TS-ASR is that it cannot be used when the speakers in the recordings are unknown because it requires a sample of the target speakers in advance of decoding. To remove this limitation, we propose an iterative method, in which (i) the estimation of speaker embeddings and (ii) TS-ASR based on the estimated speaker embeddings are alternately executed. We evaluated the proposed method by using very challenging dialogue recordings in which the speaker overlap ratio was over 20%. We confirmed that the proposed method significantly reduced both the word error rate (WER) and diarization error rate (DER). Our proposed method combined with i-vector speaker embeddings ultimately achieved a WER that differed by only 2.1 % from that of TS-ASR given oracle speaker embeddings. Furthermore, our method can solve speaker diarization simultaneously as a by-product and achieved better DER than that of the conventional clustering-based speaker diarization method based on i-vector.
SUPP.AI: Finding Evidence for Supplement-Drug Interactions
Authors: Lucy Lu Wang, Oyvind Tafjord, Arman Cohan, Sarthak Jain, Sam Skjonsberg, Carissa Schoenick, Nick Botner, Waleed Ammar
Date: 2019-09-17 00:00:00
Proceeding: ACL 2020 6
Tasks: Articles, General Classification, Language Modeling, Language Modelling

Dietary supplements are used by a large portion of the population, but information on their pharmacologic interactions is incomplete. To address this challenge, we present SUPP.AI, an application for browsing evidence of supplement-drug interactions (SDIs) extracted from the biomedical literature. We train a model to automatically extract supplement information and identify such interactions from the scientific literature. To address the lack of labeled data for SDI identification, we use labels of the closely related task of identifying drug-drug interactions (DDIs) for supervision. We fine-tune the contextualized word representations of the RoBERTa language model using labeled DDI data, and apply the fine-tuned model to identify supplement interactions. We extract 195k evidence sentences from 22M articles (P=0.82, R=0.58, F1=0.68) for 60k interactions. We create the SUPP.AI application for users to search evidence sentences extracted by our model. SUPP.AI is an attempt to close the information gap on dietary supplements by making up-to-date evidence on SDIs more discoverable for researchers, clinicians, and consumers.
Recursive Graphical Neural Networks for Text Classification
Authors: Wei Li, Shuheng Li, Shuming Ma, Yancheng He, Deli Chen, Xu sun
Date: 2019-09-18 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

The complicated syntax structure of natural language is hard to be explicitly modeled by sequence-based models. Graph is a natural structure to describe the complicated relation between tokens. The recent advance in Graph Neural Networks (GNN) provides a powerful tool to model graph structure data, but simple graph models such as Graph Convolutional Networks (GCN) suffer from over-smoothing problem, that is, when stacking multiple layers, all nodes will converge to the same value. In this paper, we propose a novel Recursive Graphical Neural Networks model (ReGNN) to represent text organized in the form of graph. In our proposed model, LSTM is used to dynamically decide which part of the aggregated neighbor information should be transmitted to upper layers thus alleviating the over-smoothing problem. Furthermore, to encourage the exchange between the local and global information, a global graph-level node is designed. We conduct experiments on both single and multiple label text classification tasks. Experiment results show that our ReGNN model surpasses the strong baselines significantly in most of the datasets and greatly alleviates the over-smoothing problem.
Modeling Conversation Structure and Temporal Dynamics for Jointly Predicting Rumor Stance and Veracity
Authors: Penghui Wei, Nan Xu, Wenji Mao
Date: 2019-09-18 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Multi-Task Learning, Stance Classification

Automatically verifying rumorous information has become an important and challenging task in natural language processing and social media analytics. Previous studies reveal that people's stances towards rumorous messages can provide indicative clues for identifying the veracity of rumors, and thus determining the stances of public reactions is a crucial preceding step for rumor veracity prediction. In this paper, we propose a hierarchical multi-task learning framework for jointly predicting rumor stance and veracity on Twitter, which consists of two components. The bottom component of our framework classifies the stances of tweets in a conversation discussing a rumor via modeling the structural property based on a novel graph convolutional network. The top component predicts the rumor veracity by exploiting the temporal dynamics of stance evolution. Experimental results on two benchmark datasets show that our method outperforms previous methods in both rumor stance classification and veracity prediction.
Improving Natural Language Inference with a Pretrained Parser
Authors: Deric Pang, Lucy H. Lin, Noah A. Smith
Date: 2019-09-18 00:00:00
Tasks: Natural Language Inference

We introduce a novel approach to incorporate syntax into natural language inference (NLI) models. Our method uses contextual token-level vector representations from a pretrained dependency parser. Like other contextual embedders, our method is broadly applicable to any neural model. We experiment with four strong NLI models (decomposable attention model, ESIM, BERT, and MT-DNN), and show consistent benefit to accuracy across three NLI benchmarks.
Pre-trained Language Model for Biomedical Question Answering
Authors: Wonjin Yoon, Jinhyuk Lee, Donghyeon Kim, Minbyul Jeong, Jaewoo Kang
Date: 2019-09-18 00:00:00
Tasks: Language Modeling, Language Modelling, model, Question Answering

The recent success of question answering systems is largely attributed to pre-trained language models. However, as language models are mostly pre-trained on general domain corpora such as Wikipedia, they often have difficulty in understanding biomedical questions. In this paper, we investigate the performance of BioBERT, a pre-trained biomedical language model, in answering biomedical questions including factoid, list, and yes/no type questions. BioBERT uses almost the same structure across various question types and achieved the best performance in the 7th BioASQ Challenge (Task 7b, Phase B). BioBERT pre-trained on SQuAD or SQuAD 2.0 easily outperformed previous state-of-the-art models. BioBERT obtains the best performance when it uses the appropriate pre-/post-processing strategies for questions, passages, and answers.
Text Length Adaptation in Sentiment Classification
Authors: Reinald Kim Amplayo, Seonjae Lim, Seung-won Hwang
Date: 2019-09-18 00:00:00
Tasks: Classification, General Classification, Sentiment Analysis, Sentiment Classification, Transfer Learning

Can a text classifier generalize well for datasets where the text length is different? For example, when short reviews are sentiment-labeled, can these transfer to predict the sentiment of long reviews (i.e., short to long transfer), or vice versa? While unsupervised transfer learning has been well-studied for cross domain/lingual transfer tasks, Cross Length Transfer (CLT) has not yet been explored. One reason is the assumption that length difference is trivially transferable in classification. We show that it is not, because short/long texts differ in context richness and word intensity. We devise new benchmark datasets from diverse domains and languages, and show that existing models from similar tasks cannot deal with the unique challenge of transferring across text lengths. We introduce a strong baseline model called BaggedCNN that treats long texts as bags containing short texts. We propose a state-of-the-art CLT model called Length Transfer Networks (LeTraNets) that introduces a two-way encoding scheme for short and long texts using multiple training mechanisms. We test our models and find that existing models perform worse than the BaggedCNN baseline, while LeTraNets outperforms all models.
Subword ELMo
Authors: Jiangtong Li, Hai Zhao, Zuchao Li, Wei Bi, Xiaojiang Liu
Date: 2019-09-18 00:00:00
Tasks: Dependency Parsing, Natural Language Inference, Semantic Role Labeling

Embedding from Language Models (ELMo) has shown to be effective for improving many natural language processing (NLP) tasks, and ELMo takes character information to compose word representation to train language models.However, the character is an insufficient and unnatural linguistic unit for word representation.Thus we introduce Embedding from Subword-aware Language Models (ESuLMo) which learns word representation from subwords using unsupervised segmentation over words.We show that ESuLMo can enhance four benchmark NLP tasks more effectively than ELMo, including syntactic dependency parsing, semantic role labeling, implicit discourse relation recognition and textual entailment, which brings a meaningful improvement over ELMo.
Using BERT for Word Sense Disambiguation
Authors: Jiaju Du, Fanchao Qi, Maosong Sun
Date: 2019-09-18 00:00:00
Tasks: Word Sense Disambiguation

Word Sense Disambiguation (WSD), which aims to identify the correct sense of a given polyseme, is a long-standing problem in NLP. In this paper, we propose to use BERT to extract better polyseme representations for WSD and explore several ways of combining BERT and the classifier. We also utilize sense definitions to train a unified classifier for all words, which enables the model to disambiguate unseen polysemes. Experiments show that our model achieves the state-of-the-art results on the standard English All-word WSD evaluation.
Hierarchical Meta-Embeddings for Code-Switching Named Entity Recognition
Authors: Genta Indra Winata, Zhaojiang Lin, Jamin Shin, Zihan Liu, Pascale Fung
Date: 2019-09-18 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Word Embeddings

In countries that speak multiple main languages, mixing up different languages within a conversation is commonly called code-switching. Previous works addressing this challenge mainly focused on word-level aspects such as word embeddings. However, in many cases, languages share common subwords, especially for closely related languages, but also for languages that are seemingly irrelevant. Therefore, we propose Hierarchical Meta-Embeddings (HME) that learn to combine multiple monolingual word-level and subword-level embeddings to create language-agnostic lexical representations. On the task of Named Entity Recognition for English-Spanish code-switching data, our model achieves the state-of-the-art performance in the multilingual settings. We also show that, in cross-lingual settings, our model not only leverages closely related languages, but also learns from languages with different roots. Finally, we show that combining different subunits are crucial for capturing code-switching entities.
Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences
Authors: Genta Indra Winata, Andrea Madotto, Chien-Sheng Wu, Pascale Fung
Date: 2019-09-18 00:00:00
Proceeding: CONLL 2019 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Training code-switched language models is difficult due to lack of data and complexity in the grammatical structure. Linguistic constraint theories have been used for decades to generate artificial code-switching sentences to cope with this issue. However, this require external word alignments or constituency parsers that create erroneous results on distant languages. We propose a sequence-to-sequence model using a copy mechanism to generate code-switching data by leveraging parallel monolingual translations from a limited source of code-switching data. The model learns how to combine words from parallel sentences and identifies when to switch one language to the other. Moreover, it captures code-switching constraints by attending and aligning the words in inputs, without requiring any external knowledge. Based on experimental results, the language model trained with the generated sentences achieves state-of-the-art performance and improves end-to-end automatic speech recognition.
Combining complex networks and data mining: why and how
Authors: M. Zanin, D. Papo, P. A. Sousa, E. Menasalvas, A. Nicchi, E. Kubik, S. Boccaletti
Date: 2016-04-29 00:00:00

The increasing power of computer technology does not dispense with the need to extract meaningful in- formation out of data sets of ever growing size, and indeed typically exacerbates the complexity of this task. To tackle this general problem, two methods have emerged, at chronologically different times, that are now commonly used in the scientific community: data mining and complex network theory. Not only do complex network analysis and data mining share the same general goal, that of extracting information from complex systems to ultimately create a new compact quantifiable representation, but they also often address similar problems too. In the face of that, a surprisingly low number of researchers turn out to resort to both methodologies. One may then be tempted to conclude that these two fields are either largely redundant or totally antithetic. The starting point of this review is that this state of affairs should be put down to contingent rather than conceptual differences, and that these two fields can in fact advantageously be used in a synergistic manner. An overview of both fields is first provided, some fundamental concepts of which are illustrated. A variety of contexts in which complex network theory and data mining have been used in a synergistic manner are then presented. Contexts in which the appropriate integration of complex network metrics can lead to improved classification rates with respect to classical data mining algorithms and, conversely, contexts in which data mining can be used to tackle important issues in complex network theory applications are illustrated. Finally, ways to achieve a tighter integration between complex networks and data mining, and open lines of research are discussed.
An expert system for detecting automobile insurance fraud using social network analysis
Authors: Lovro Šubelj, Štefan Furlan, Marko Bajec
Date: 2011-04-19 00:00:00

The article proposes an expert system for detection, and subsequent investigation, of groups of collaborating automobile insurance fraudsters. The system is described and examined in great detail, several technical difficulties in detecting fraud are also considered, for it to be applicable in practice. Opposed to many other approaches, the system uses networks for representation of data. Networks are the most natural representation of such a relational domain, allowing formulation and analysis of complex relations between entities. Fraudulent entities are found by employing a novel assessment algorithm, \textit{Iterative Assessment Algorithm} (\textit{IAA}), also presented in the article. Besides intrinsic attributes of entities, the algorithm explores also the relations between entities. The prototype was evaluated and rigorously analyzed on real world data. Results show that automobile insurance fraud can be efficiently detected with the proposed system and that appropriate data representation is vital.
Movie Recommendation System using Sentiment Analysis from Microblogging Data
Authors: Sudhanshu Kumar, Shirsendu Sukanta Halder, Kanjar De, Partha Pratim Roy
Date: 2018-11-27 00:00:00
Tasks: Collaborative Filtering, Movie Recommendation, Recommendation Systems, Sentiment Analysis

Recommendation systems are important intelligent systems that play a vital role in providing selective information to users. Traditional approaches in recommendation systems include collaborative filtering and content-based filtering. However, these approaches have certain limitations like the necessity of prior user history and habits for performing the task of recommendation. In order to reduce the effect of such dependencies, this paper proposes a hybrid recommendation system which combines the collaborative filtering, content-based filtering with sentiment analysis of movie tweets. The movie tweets have been collected from microblogging websites to understand the current trends and user response of the movie. Experiments conducted on public database produce promising results.
Prediction of overall survival and molecular markers in gliomas via analysis of digital pathology images using deep learning
Authors: Saima Rathore, Muhammad Aksam Iftikhar, Zissimos Mourelatos
Date: 2019-09-19 00:00:00
Tasks: Descriptive, Survival Prediction, whole slide images

Cancer histology reveals disease progression and associated molecular processes, and contains rich phenotypic information that is predictive of outcome. In this paper, we developed a computational approach based on deep learning to predict the overall survival and molecular subtypes of glioma patients from microscopic images of tissue biopsies, reflecting measures of microvascular proliferation, mitotic activity, nuclear atypia, and the presence of necrosis. Whole-slide images from 663 unique patients [IDH: 333 IDH-wildtype, 330 IDH-mutants, 1p/19q: 201 1p/19q non-codeleted, 129 1p/19q codeleted] were obtained from TCGA. Sub-images that were free of artifacts and that contained viable tumor with descriptive histologic characteristics were extracted, which were further used for training and testing a deep neural network. The output layer of the network was configured in two different ways: (i) a final Cox model layer to output a prediction of patient risk, and (ii) a final layer with sigmoid activation function, and stochastic gradient decent based optimization with binary cross-entropy loss. Both survival prediction and molecular subtype classification produced promising results using our model. The c-statistic was estimated to be 0.82 (p-value=4.8x10-5) between the risk scores of the proposed deep learning model and overall survival, while accuracies of 88% (area under the curve [AUC]=0.86) were achieved in the detection of IDH mutational status and 1p/19q codeletion. These findings suggest that the deep learning techniques can be applied to microscopic images for objective, accurate, and integrated prediction of outcome for glioma patients. The proposed marker may contribute to (i) stratification of patients into clinical trials, (ii) patient selection for targeted therapy, and (iii) personalized treatment planning.
Kinetic Song Comprehension: Deciphering Personal Listening Habits via Phone Vibrations
Authors: Richard Matovu, Isaac Griswold-Steiner, Abdul Serwadda
Date: 2019-09-19 00:00:00

Music is an expression of our identity, showing a significant correlation with other personal traits, beliefs, and habits. If accessed by a malicious entity, an individual's music listening habits could be used to make critical inferences about the user. In this paper, we showcase an attack in which the vibrations propagated through a user's phone while playing music via its speakers can be used to detect and classify songs. Our attack shows that known songs can be detected with an accuracy of just under 80%, while a corpus of 100 songs can be classified with an accuracy greater than 80%. We investigate such questions under a wide variety of experimental scenarios involving three surfaces and five phone speaker volumes. Although users can mitigate some of the risk by using a phone cover to dampen the vibrations, we show that a sophisticated attacker could adapt the attack to still classify songs with a decent accuracy. This paper demonstrates a new way in which motion sensor data can be leveraged to intrude on user music preferences without their express permission. Whether this information is leveraged for financial gain or political purposes, our research makes a case for why more rigorous methods of protecting user data should be utilized by companies, and if necessary, individuals.
Self-Training for End-to-End Speech Recognition
Authors: Jacob Kahn, Ann Lee, Awni Hannun
Date: 2019-09-19 00:00:00
Tasks: Diversity, Language Modeling, Language Modelling, Pseudo Label, speech-recognition, Speech Recognition

We revisit self-training in the context of end-to-end speech recognition. We demonstrate that training with pseudo-labels can substantially improve the accuracy of a baseline model. Key to our approach are a strong baseline acoustic and language model used to generate the pseudo-labels, filtering mechanisms tailored to common errors from sequence-to-sequence models, and a novel ensemble approach to increase pseudo-label diversity. Experiments on the LibriSpeech corpus show that with an ensemble of four models and label filtering, self-training yields a 33.9% relative improvement in WER compared with a baseline trained on 100 hours of labelled data in the noisy speech setting. In the clean speech setting, self-training recovers 59.3% of the gap between the baseline and an oracle model, which is at least 93.8% relatively higher than what previous approaches can achieve.