pwc-archive/papers-with-abstracts huggingface dataset:
NLP Tools for Predictive Maintenance Records in MaintNet
Authors: Farhad Akhbardeh, Travis Desell, Marcos Zampieri
Date: 2020-12-01 00:00:00
Proceeding: Asian Chapter of the Association for Computational Linguistics 2020
Tasks: Clustering, POS, POS Tagging

Processing maintenance logbook records is an important step in the development of predictive maintenance systems. Logbooks often include free text fields with domain specific terms, abbreviations, and non-standard spelling posing challenges to off-the-shelf NLP pipelines trained on standard contemporary corpora. Despite the importance of this data type, processing predictive maintenance data is still an under-explored topic in NLP. With the goal of providing more datasets and resources to the community, in this paper we present a number of new resources available in MaintNet, a collaborative open-source library and data repository of predictive maintenance language datasets. We describe novel annotated datasets from multiple domains such as aviation, automotive, and facility maintenance domains and new tools for segmentation, spell checking, POS tagging, clustering, and classification.
Measuring Semantic Similarity by Latent Relational Analysis
Authors: Peter D. Turney
Date: 2005-08-10 00:00:00
Tasks: Multiple-choice, Semantic Similarity, Semantic Textual Similarity

This paper introduces Latent Relational Analysis (LRA), a method for measuring semantic similarity. LRA measures similarity in the semantic relations between two pairs of words. When two pairs have a high degree of relational similarity, they are analogous. For example, the pair cat:meow is analogous to the pair dog:bark. There is evidence from cognitive science that relational similarity is fundamental to many cognitive and linguistic tasks (e.g., analogical reasoning). In the Vector Space Model (VSM) approach to measuring relational similarity, the similarity between two pairs is calculated by the cosine of the angle between the vectors that represent the two pairs. The elements in the vectors are based on the frequencies of manually constructed patterns in a large corpus. LRA extends the VSM approach in three ways: (1) patterns are derived automatically from the corpus, (2) Singular Value Decomposition is used to smooth the frequency data, and (3) synonyms are used to reformulate word pairs. This paper describes the LRA algorithm and experimentally compares LRA to VSM on two tasks, answering college-level multiple-choice word analogy questions and classifying semantic relations in noun-modifier expressions. LRA achieves state-of-the-art results, reaching human-level performance on the analogy questions and significantly exceeding VSM performance on both tasks.
Named Entity Recognition in Tweets: An Experimental Study
Authors: Alan Ritter, Sam Clark, Mausam Etzioni, Oren Etzioni
Date: 2011-07-01 00:00:00
Proceeding: Conference on Empirical Methods in Natural Language Processing 2011 7
Tasks: Chunking, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Part-Of-Speech Tagging

People tweet more than 100 Million times
daily, yielding a noisy, informal, but sometimes informative corpus of 140-character
messages that mirrors the zeitgeist in an unprecedented manner. The performance of
standard NLP tools is severely degraded on
tweets. This paper addresses this issue by
re-building the NLP pipeline beginning with
part-of-speech tagging, through chunking, to
named-entity recognition. Our novel T-NER
system doubles F1 score compared with the
Stanford NER system. T-NER leverages the
redundancy inherent in tweets to achieve this
performance, using LabeledLDA to exploit
Freebase dictionaries as a source of distant
supervision. LabeledLDA outperforms cotraining, increasing F1 by 25% over ten common entity types.
Our NLP tools are available at: http://
github.com/aritter/twitter_nlp
mgsohrab at WNUT 2020 Shared Task-1: Neural Exhaustive Approach for Entity and Relation Recognition Over Wet Lab Protocols
Authors: Mohammad Golam Sohrab, Anh-Khoa Duong Nguyen, Makoto Miwa, Hiroya Takamura
Date: 2020-11-01 00:00:00
Proceeding: Workshop on Noisy User-generated Text 2020 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Relation, Relation Extraction

We present a neural exhaustive approach that
addresses named entity recognition (NER) and
relation recognition (RE), for the entity and relation recognition over the wet-lab protocols
shared task. We introduce BERT-based neural
exhaustive approach that enumerates all possible spans as potential entity mentions and
classifies them into entity types or no entity
with deep neural networks to address NER.
To solve relation extraction task, based on the
NER predictions or given gold mentions we
create all possible trigger-argument pairs and
classify them into relation types or no relation.
In NER task, we achieved 76.60% in terms of
F-score as third rank system among the participated systems. In relation extraction task, we
achieved 80.46% in terms of F-score as the top
system in the relation extraction or recognition
task. Besides we compare our model based on
the wet lab protocols corpus (WLPC) with the
WLPC baseline and dynamic graph-based information extraction (DyGIE) systems.
Structural Bifurcation Analysis in Chemical Reaction Networks
Authors: Takashi Okada, Je-Chiang Tsai, Atsushi Mochizuki
Date: 2017-11-01 00:00:00

In living cells, chemical reactions form a complex network. Complicated dynamics arising from such networks are the origins of biological functions. We propose a novel mathematical method to analyze bifurcation behaviors of a reaction system from the network structure alone. The whole network is decomposed into subnetworks based on "buffering structures". For each subnetwork, the bifurcation condition is studied independently, and the parameters that can induce bifurcations and the chemicals that can exhibit bifurcations are determined. We demonstrate our theory using hypothetical and real networks.
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
Authors: Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei LI, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen
Date: 2021-06-15 00:00:00
Proceeding: ACL 2022 5
Tasks: Intent Classification, Medical Concept Normalization, Medical Relation Extraction, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Inference, Semantic Similarity, Sentence, Sentence Classification, Sentence-Pair Classification

Artificial Intelligence (AI), along with the recent progress in biomedical language understanding, is gradually changing medical practice. With the development of biomedical language understanding benchmarks, AI applications are widely used in the medical field. However, most benchmarks are limited to English, which makes it challenging to replicate many of the successes in English for other languages. To facilitate research in this direction, we collect real-world biomedical data and present the first Chinese Biomedical Language Understanding Evaluation (CBLUE) benchmark: a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification, and an associated online platform for model evaluation, comparison, and analysis. To establish evaluation on these tasks, we report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling. Our benchmark is released at \url{https://tianchi.aliyun.com/dataset/dataDetail?dataId=95414&lang=en-us}.
Topological Receptive Field Model for Human Retinotopic Mapping
Authors: Yanshuai Tu, Duyan Ta, Zhong-Lin Lu, Yalin Wang
Date: 2021-06-15 00:00:00
Tasks: model

The mapping between visual inputs on the retina and neuronal activations in the visual cortex, i.e., retinotopic map, is an essential topic in vision science and neuroscience. Human retinotopic maps can be revealed by analyzing the functional magnetic resonance imaging (fMRI) signal responses to designed visual stimuli in vivo. Neurophysiology studies summarized that visual areas are topological (i.e., nearby neurons have receptive fields at nearby locations in the image). However, conventional fMRI-based analyses frequently generate non-topological results because they process fMRI signals on a voxel-wise basis, without considering the neighbor relations on the surface. Here we propose a topological receptive field (tRF) model which imposes the topological condition when decoding retinotopic fMRI signals. More specifically, we parametrized the cortical surface to a unit disk, characterized the topological condition by tRF, and employed an efficient scheme to solve the tRF model. We tested our framework on both synthetic and human fMRI data. Experimental results showed that the tRF model could remove the topological violations, improve model explaining power, and generate biologically plausible retinotopic maps. The proposed framework is general and can be applied to other sensory maps.
Resilience and performance of the power grid with high penetration of renewable energy sources: the Balearic Islands as a case study
Authors: Benjamín A. Carreras, Pere Colet, José M. Reynolds-Barredo, Damià Gomila
Date: 2020-09-07 00:00:00

We analyze the dynamics of the power grid with a high penetration of renewable energy sources using the ORNL-PSERC-Alaska (OPA) model. In particular we consider the power grid of the Balearic Islands with a high share of solar photovoltaic power as a case study. Day-to-day fluctuations of the solar generation and the use of storage are included in the model. Resilience is analyzed through the blackout distribution and performance is measured as the average fraction of the demand covered by solar power generation. We find that with the present consumption patterns and moderate storage, solar generation can replace conventional power plants without compromising reliability up to $30\%$ of the total installed capacity. We also find that using source redundancy it is possible to cover up to $80\%$ or more of the demand with solar plants, while keeping the risk similar to that with full conventional generation. However this requires oversizing the installed solar power to be at least $2.5$ larger than the average demand. The potential of wind energy is also briefly discussed
Exact mean-field theory explains the dual role of electrical synapses in collective synchronization
Authors: Ernest Montbrió, Diego Pazó
Date: 2020-09-03 00:00:00

Electrical synapses play a major role in setting up neuronal synchronization, but the precise mechanisms whereby these synapses contribute to synchrony are subtle and remain elusive. To investigate these mechanisms mean-field theories for quadratic integrate-and-fire neurons with electrical synapses have been recently put forward. Still, the validity of these theories is controversial since they assume that the neurons produce unrealistic, symmetric spikes, ignoring the well-known impact of spike shape on synchronization. Here we show that the assumption of symmetric spikes can be relaxed in such theories. The resulting mean-field equations reveal a dual role of electrical synapses: First, they equalize membrane potentials favoring the emergence of synchrony. Second, electrical synapses act as "virtual chemical synapses", which can be either excitatory or inhibitory depending upon the spike shape. Our results offer a precise mathematical explanation of the intricate effect of electrical synapses in collective synchronization. This reconciles previous theoretical and numerical works, and confirms the suitability of recent low-dimensional mean-field theories to investigate electrically coupled neuronal networks.
Models of benthic bipedalism
Authors: F. Giardina, L. Mahadevan
Date: 2020-09-02 00:00:00

Walking is a common bipedal and quadrupedal gait and is often associated with terrestrial and aquatic organisms. Inspired by recent evidence of the neural underpinnings of primitive aquatic walking in the little skate Leucoraja erinacea, we introduce a theoretical model of aquatic walking that reveals robust and efficient gaits with modest requirements for body morphology and control. The model predicts undulatory behavior of the system body with a regular foot placement pattern which is also observed in the animal, and additionally predicts the existence of gait bistability between two states, one with a large energetic cost for locomotion and another associated with almost no energetic cost. We show that these can be discovered using a simple reinforcement learning scheme. To test these theoretical frameworks, we built a bipedal robot and show that its behaviors are similar to those of our minimal model: its gait is also periodic and exhibits bistability, with a low efficiency gait separated from a high efficiency gait by a "jump" transition. Overall, our study highlights the physical constraints on the evolution of walking and provides a guide for the design of efficient biomimetic robots.
Learning Incident Prediction Models Over Large Geographical Areas for Emergency Response Systems
Authors: Sayyed Mohsen Vazirizade, Ayan Mukhopadhyay, Geoffrey Pettet, Said El Said, Hiba Baroud, Abhishek Dubey
Date: 2021-06-15 00:00:00
Tasks: Decision Making, Management

Principled decision making in emergency response management necessitates the use of statistical models that predict the spatial-temporal likelihood of incident occurrence. These statistical models are then used for proactive stationing which allocates first responders across the spatial area in order to reduce overall response time. Traditional methods that simply aggregate past incidents over space and time fail to make useful short-term predictions when the spatial region is large and focused on fine-grained spatial entities like interstate highway networks. This is partially due to the sparsity of incidents with respect to the area in consideration. Further, accidents are affected by several covariates, and collecting, cleaning, and managing multiple streams of data from various sources is challenging for large spatial areas. In this paper, we highlight how this problem is being solved for the state of Tennessee, a state in the USA with a total area of over 100,000 sq. km. Our pipeline, based on a combination of synthetic resampling, non-spatial clustering, and learning from data can efficiently forecast the spatial and temporal dynamics of accident occurrence, even under sparse conditions. In the paper, we describe our pipeline that uses data related to roadway geometry, weather, historical accidents, and real-time traffic congestion to aid accident forecasting. To understand how our forecasting model can affect allocation and dispatch, we improve upon a classical resource allocation approach. Experimental results show that our approach can significantly reduce response times in the field in comparison with current approaches followed by first responders.
Improving Paraphrase Detection with the Adversarial Paraphrasing Task
Authors: Animesh Nighojkar, John Licato
Date: 2021-06-14 00:00:00
Proceeding: ACL 2021 5
Tasks: Dataset Generation, Paraphrase Identification, Sentence

If two sentences have the same meaning, it should follow that they are equivalent in their inferential properties, i.e., each sentence should textually entail the other. However, many paraphrase datasets currently in widespread use rely on a sense of paraphrase based on word overlap and syntax. Can we teach them instead to identify paraphrases in a way that draws on the inferential properties of the sentences, and is not over-reliant on lexical and syntactic similarities of a sentence pair? We apply the adversarial paradigm to this question, and introduce a new adversarial method of dataset creation for paraphrase identification: the Adversarial Paraphrasing Task (APT), which asks participants to generate semantically equivalent (in the sense of mutually implicative) but lexically and syntactically disparate paraphrases. These sentence pairs can then be used both to test paraphrase identification models (which get barely random accuracy) and then improve their performance. To accelerate dataset generation, we explore automation of APT using T5, and show that the resulting dataset also improves accuracy. We discuss implications for paraphrase detection and release our dataset in the hope of making paraphrase detection models better able to detect sentence-level meaning equivalence.
Using heterogeneity in semi-supervised transcription hypotheses to improve code-switched speech recognition
Authors: Andrew Slottje, Shannon Wotherspoon, William Hartmann, Matthew Snover, Owen Kimball
Date: 2021-06-14 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Modeling code-switched speech is an important problem in automatic speech recognition (ASR). Labeled code-switched data are rare, so monolingual data are often used to model code-switched speech. These monolingual data may be more closely matched to one of the languages in the code-switch pair. We show that such asymmetry can bias prediction toward the better-matched language and degrade overall model performance. To address this issue, we propose a semi-supervised approach for code-switched ASR. We consider the case of English-Mandarin code-switching, and the problem of using monolingual data to build bilingual "transcription models'' for annotation of unlabeled code-switched data. We first build multiple transcription models so that their individual predictions are variously biased toward either English or Mandarin. We then combine these biased transcriptions using confidence-based selection. This strategy generates a superior transcript for semi-supervised training, and obtains a 19% relative improvement compared to a semi-supervised system that relies on a transcription model built with only the best-matched monolingual data.
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data
Authors: Han Guo, Bowen Tan, Zhengzhong Liu, Eric P. Xing, Zhiting Hu
Date: 2021-06-14 00:00:00
Tasks: Q-Learning, Reinforcement Learning (RL), Text Generation

Maximum likelihood estimation (MLE) is the predominant algorithm for training text generation models. This paradigm relies on direct supervision examples, which is not applicable to many emerging applications, such as generating adversarial attacks or generating prompts to control language models. Reinforcement learning (RL) on the other hand offers a more flexible solution by allowing users to plug in arbitrary task metrics as reward. Yet previous RL algorithms for text generation, such as policy gradient (on-policy RL) and Q-learning (off-policy RL), are often notoriously inefficient or unstable to train due to the large sequence space and the sparse reward received only at the end of sequences. In this paper, we introduce a new RL formulation for text generation from the soft Q-learning (SQL) perspective. It enables us to draw from the latest RL advances, such as path consistency learning, to combine the best of on-/off-policy updates, and learn effectively from sparse reward. We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation. Experiments show our approach consistently outperforms both task-specialized algorithms and the previous RL methods.
Overcoming Domain Mismatch in Low Resource Sequence-to-Sequence ASR Models using Hybrid Generated Pseudotranscripts
Authors: Chak-Fai Li, Francis Keith, William Hartmann, Matthew Snover, Owen Kimball
Date: 2021-06-14 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Domain Adaptation, Language Modeling, Language Modelling, speech-recognition, Speech Recognition

Sequence-to-sequence (seq2seq) models are competitive with hybrid models for automatic speech recognition (ASR) tasks when large amounts of training data are available. However, data sparsity and domain adaptation are more problematic for seq2seq models than their hybrid counterparts. We examine corpora of five languages from the IARPA MATERIAL program where the transcribed data is conversational telephone speech (CTS) and evaluation data is broadcast news (BN). We show that there is a sizable initial gap in such a data condition between hybrid and seq2seq models, and the hybrid model is able to further improve through the use of additional language model (LM) data. We use an additional set of untranscribed data primarily in the BN domain for semisupervised training. In semisupervised training, a seed model trained on transcribed data generates hypothesized transcripts for unlabeled domain-matched data for further training. By using a hybrid model with an expanded language model for pseudotranscription, we are able to improve our seq2seq model from an average word error rate (WER) of 66.7% across all five languages to 29.0% WER. While this puts the seq2seq model at a competitive operating point, hybrid models are still able to use additional LM data to maintain an advantage.
Unbiased Sentence Encoder For Large-Scale Multi-lingual Search Engines
Authors: Mahdi Hajiaghayi, Monir Hajiaghayi, Mark Bolin
Date: 2021-03-01 00:00:00
Tasks: Document Ranking, Language Modelling, Semantic Similarity, Semantic Textual Similarity, Sentence

In this paper, we present a multi-lingual sentence encoder that can be used in search engines as a query and document encoder. This embedding enables a semantic similarity score between queries and documents that can be an important feature in document ranking and relevancy. To train such a customized sentence encoder, it is beneficial to leverage users search data in the form of query-document clicked pairs however, we must avoid relying too much on search click data as it is biased and does not cover many unseen cases. The search data is heavily skewed towards short queries and for long queries is small and often noisy. The goal is to design a universal multi-lingual encoder that works for all cases and covers both short and long queries. We select a number of public NLI datasets in different languages and translation data and together with user search data we train a language model using a multi-task approach. A challenge is that these datasets are not homogeneous in terms of content, size and the balance ratio. While the public NLI datasets are usually two-sentence based with the same portion of positive and negative pairs, the user search data can contain multi-sentence documents and only positive pairs. We show how multi-task training enables us to leverage all these datasets and exploit knowledge sharing across these tasks.