pwc-archive/papers-with-abstracts huggingface dataset:
Author as Character and Narrator: Deconstructing Personal Narratives from the r/AmITheAsshole Reddit Community
Authors: Salvatore Giorgi, Ke Zhao, Alexander H. Feng, Lara J. Martin
Date: 2023-01-19 00:00:00
Tasks: ARC

In the r/AmITheAsshole subreddit, people anonymously share first person narratives that contain some moral dilemma or conflict and ask the community to judge who is at fault (i.e., who is "the asshole"). In general, first person narratives are a unique storytelling domain where the author is the narrator (the person telling the story) but can also be a character (the person living the story) and, thus, the author has two distinct voices presented in the story. In this study, we identify linguistic and narrative features associated with the author as the character or as a narrator. We use these features to answer the following questions: (1) what makes an asshole character and (2) what makes an asshole narrator? We extract both Author-as-Character features (e.g., demographics, narrative event chain, and emotional arc) and Author-as-Narrator features (i.e., the style and emotion of the story as a whole) in order to identify which aspects of the narrative are correlated with the final moral judgment. Our work shows that "assholes" as Characters frame themselves as lacking agency with a more positive personal arc, while "assholes" as Narrators will tell emotional and opinionated stories.
Keyword Embeddings for Query Suggestion
Authors: Jorge Gabín, M. Eduardo Ares, Javier Parapar
Date: 2023-01-19 00:00:00
Tasks: Retrieval, Sentence, Sentence Embedding, Sentence-Embedding

Nowadays, search engine users commonly rely on query suggestions to improve their initial inputs. Current systems are very good at recommending lexical adaptations or spelling corrections to users' queries. However, they often struggle to suggest semantically related keywords given a user's query. The construction of a detailed query is crucial in some tasks, such as legal retrieval or academic search. In these scenarios, keyword suggestion methods are critical to guide the user during the query formulation. This paper proposes two novel models for the keyword suggestion task trained on scientific literature. Our techniques adapt the architecture of Word2Vec and FastText to generate keyword embeddings by leveraging documents' keyword co-occurrence. Along with these models, we also present a specially tailored negative sampling approach that exploits how keywords appear in academic publications. We devise a ranking-based evaluation methodology following both known-item and ad-hoc search scenarios. Finally, we evaluate our proposals against the state-of-the-art word and sentence embedding models showing considerable improvements over the baselines for the tasks.
Semantic-aware Contrastive Learning for More Accurate Semantic Parsing
Authors: Shan Wu, Chunlei Xin, Bo Chen, Xianpei Han, Le Sun
Date: 2023-01-19 00:00:00
Tasks: Contrastive Learning, Semantic Parsing

Since the meaning representations are detailed and accurate annotations which express fine-grained sequence-level semtantics, it is usually hard to train discriminative semantic parsers via Maximum Likelihood Estimation (MLE) in an autoregressive fashion. In this paper, we propose a semantic-aware contrastive learning algorithm, which can learn to distinguish fine-grained meaning representations and take the overall sequence-level semantic into consideration. Specifically, a multi-level online sampling algorithm is proposed to sample confusing and diverse instances. Three semantic-aware similarity functions are designed to accurately measure the distance between meaning representations as a whole. And a ranked contrastive loss is proposed to pull the representations of the semantic-identical instances together and push negative instances away. Experiments on two standard datasets show that our approach achieves significant improvements over MLE baselines and gets state-of-the-art performances by simply applying semantic-aware contrastive learning on a vanilla Seq2Seq model.
Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection
Authors: Weijia Xu, Sweta Agrawal, Eleftheria Briakou, Marianna J. Martindale, Marine Carpuat
Date: 2023-01-18 00:00:00
Tasks: Hallucination, Machine Translation, Translation

Neural sequence generation models are known to "hallucinate", by producing outputs that are unrelated to the source text. These hallucinations are potentially harmful, yet it remains unclear in what conditions they arise and how to mitigate their impact. In this work, we first identify internal model symptoms of hallucinations by analyzing the relative token contributions to the generation in contrastive hallucinated vs. non-hallucinated outputs generated via source perturbations. We then show that these symptoms are reliable indicators of natural hallucinations, by using them to design a lightweight hallucination detector which outperforms both model-free baselines and strong classifiers based on quality estimation or large pre-trained models on manually annotated English-Chinese and German-English translation test beds.
What's happening in your neighborhood? A Weakly Supervised Approach to Detect Local News
Authors: Deven Santosh Shah, Shiying He, Gosuddin Kamaruddin Siddiqi, Radhika Bansal
Date: 2023-01-15 00:00:00
Tasks: Articles, NER, News Recommendation

Local news articles are a subset of news that impact users in a geographical area, such as a city, county, or state. Detecting local news (Step 1) and subsequently deciding its geographical location as well as radius of impact (Step 2) are two important steps towards accurate local news recommendation. Naive rule-based methods, such as detecting city names from the news title, tend to give erroneous results due to lack of understanding of the news content. Empowered by the latest development in natural language processing, we develop an integrated pipeline that enables automatic local news detection and content-based local news recommendations. In this paper, we focus on Step 1 of the pipeline, which highlights: (1) a weakly supervised framework incorporated with domain knowledge and auto data processing, and (2) scalability to multi-lingual settings. Compared with Stanford CoreNLP NER model, our pipeline has higher precision and recall evaluated on a real-world and human-labeled dataset. This pipeline has potential to more precise local news to users, helps local businesses get more exposure, and gives people more information about their neighborhood safety.
Music Playlist Title Generation Using Artist Information
Authors: Haven Kim, Seungheon Doh, Junwon Lee, Juhan Nam
Date: 2023-01-14 00:00:00
Tasks: Decoder, Diversity, Music Recommendation

Automatically generating or captioning music playlist titles given a set of tracks is of significant interest in music streaming services as customized playlists are widely used in personalized music recommendation, and well-composed text titles attract users and help their music discovery. We present an encoder-decoder model that generates a playlist title from a sequence of music tracks. While previous work takes track IDs as tokenized input for playlist title generation, we use artist IDs corresponding to the tracks to mitigate the issue from the long-tail distribution of tracks included in the playlist dataset. Also, we introduce a chronological data split method to deal with newly-released tracks in real-world scenarios. Comparing the track IDs and artist IDs as input sequences, we show that the artist-based approach significantly enhances the performance in terms of word overlap, semantic relevance, and diversity.
A Cohesive Distillation Architecture for Neural Language Models
Authors: Jan Philip Wahle
Date: 2023-01-12 00:00:00
Tasks: Knowledge Distillation, Language Modeling, Language Modelling, Masked Language Modeling, Natural Language Understanding, Word Sense Disambiguation

A recent trend in Natural Language Processing is the exponential growth in Language Model (LM) size, which prevents research groups without a necessary hardware infrastructure from participating in the development process. This study investigates methods for Knowledge Distillation (KD) to provide efficient alternatives to large-scale models. In this context, KD means extracting information about language encoded in a Neural Network and Lexical Knowledge Databases. We developed two methods to test our hypothesis that efficient architectures can gain knowledge from LMs and extract valuable information from lexical sources. First, we present a technique to learn confident probability distribution for Masked Language Modeling by prediction weighting of multiple teacher networks. Second, we propose a method for Word Sense Disambiguation (WSD) and lexical KD that is general enough to be adapted to many LMs. Our results show that KD with multiple teachers leads to improved training convergence. When using our lexical pre-training method, LM characteristics are not lost, leading to increased performance in Natural Language Understanding (NLU) tasks over the state-of-the-art while adding no parameters. Moreover, the improved semantic understanding of our model increased the task performance beyond WSD and NLU in a real-problem scenario (Plagiarism Detection). This study suggests that sophisticated training methods and network architectures can be superior over scaling trainable parameters. On this basis, we suggest the research area should encourage the development and use of efficient models and rate impacts resulting from growing LM size equally against task performance.
AI Insights into Theoretical Physics and the Swampland Program: A Journey Through the Cosmos with ChatGPT
Authors: Kay Lehnert
Date: 2023-01-10 00:00:00

In this case study, we explore the capabilities and limitations of ChatGPT, a natural language processing model developed by OpenAI, in the field of string theoretical swampland conjectures. We find that it is effective at paraphrasing and explaining concepts in a variety of styles, but not at genuinely connecting concepts. It will provide false information with full confidence and make up statements when necessary. However, its ingenious use of language can be fruitful for identifying analogies and describing visual representations of abstract concepts.
Towards Rigorous Understanding of Neural Networks via Semantics-preserving Transformations
Authors: Maximilian Schlüter, Gerrit Nolte, Alnis Murtovi, Bernhard Steffen
Date: 2023-01-19 00:00:00
Tasks: Diagnostic

In this paper we present an algebraic approach to the precise and global verification and explanation of Rectifier Neural Networks, a subclass of Piece-wise Linear Neural Networks (PLNNs), i.e., networks that semantically represent piece-wise affine functions. Key to our approach is the symbolic execution of these networks that allows the construction of semantically equivalent Typed Affine Decision Structures (TADS). Due to their deterministic and sequential nature, TADS can, similarly to decision trees, be considered as white-box models and therefore as precise solutions to the model and outcome explanation problem. TADS are linear algebras which allows one to elegantly compare Rectifier Networks for equivalence or similarity, both with precise diagnostic information in case of failure, and to characterize their classification potential by precisely characterizing the set of inputs that are specifically classified or the set of inputs where two network-based classifiers differ. All phenomena are illustrated along a detailed discussion of a minimal, illustrative example: the continuous XOR function.
Fully Elman Neural Network: A Novel Deep Recurrent Neural Network Optimized by an Improved Harris Hawks Algorithm for Classification of Pulmonary Arterial Wedge Pressure
Authors: Masoud Fetanat, Michael Stevens, Pankaj Jain, Christopher Hayward, Erik Meijering, Nigel H. Lovell
Date: 2023-01-16 00:00:00

Heart failure (HF) is one of the most prevalent life-threatening cardiovascular diseases in which 6.5 million people are suffering in the USA and more than 23 million worldwide. Mechanical circulatory support of HF patients can be achieved by implanting a left ventricular assist device (LVAD) into HF patients as a bridge to transplant, recovery or destination therapy and can be controlled by measurement of normal and abnormal pulmonary arterial wedge pressure (PAWP). While there are no commercial long-term implantable pressure sensors to measure PAWP, real-time non-invasive estimation of abnormal and normal PAWP becomes vital. In this work, first an improved Harris Hawks optimizer algorithm called HHO+ is presented and tested on 24 unimodal and multimodal benchmark functions. Second, a novel fully Elman neural network (FENN) is proposed to improve the classification performance. Finally, four novel 18-layer deep learning methods of convolutional neural networks (CNNs) with multi-layer perceptron (CNN-MLP), CNN with Elman neural networks (CNN-ENN), CNN with fully Elman neural networks (CNN-FENN), and CNN with fully Elman neural networks optimized by HHO+ algorithm (CNN-FENN-HHO+) for classification of abnormal and normal PAWP using estimated HVAD pump flow were developed and compared. The estimated pump flow was derived by a non-invasive method embedded into the commercial HVAD controller. The proposed methods are evaluated on an imbalanced clinical dataset using 5-fold cross-validation. The proposed CNN-FENN-HHO+ method outperforms the proposed CNN-MLP, CNN-ENN and CNN-FENN methods and improved the classification performance metrics across 5-fold cross-validation. The proposed methods can reduce the likelihood of hazardous events like pulmonary congestion and ventricular suction for HF patients and notify identified abnormal cases to the hospital, clinician and cardiologist.
Semiparametric inference using fractional posteriors
Authors: Alice L'Huillier, Luke Travis, Ismaël Castillo, Kolyan Ray
Date: 2023-01-19 00:00:00
Tasks: Uncertainty Quantification

We establish a general Bernstein--von Mises theorem for approximately linear semiparametric functionals of fractional posterior distributions based on nonparametric priors. This is illustrated in a number of nonparametric settings and for different classes of prior distributions, including Gaussian process priors. We show that fractional posterior credible sets can provide reliable semiparametric uncertainty quantification, but have inflated size. To remedy this, we further propose a \textit{shifted-and-rescaled} fractional posterior set that is an efficient confidence set having optimal size under regularity conditions. As part of our proofs, we also refine existing contraction rate results for fractional posteriors by sharpening the dependence of the rate on the fractional exponent.
Robust Gaussian Process Regression with Huber Likelihood
Authors: Pooja Algikar, Lamine Mili
Date: 2023-01-19 00:00:00
Tasks: Bayesian Inference, regression

Gaussian process regression in its most simplified form assumes normal homoscedastic noise and utilizes analytically tractable mean and covariance functions of predictive posterior distribution using Gaussian conditioning. Its hyperparameters are estimated by maximizing the evidence, commonly known as type II maximum likelihood estimation. Unfortunately, Bayesian inference based on Gaussian likelihood is not robust to outliers, which are often present in the observational training data sets. To overcome this problem, we propose a robust process model in the Gaussian process framework with the likelihood of observed data expressed as the Huber probability distribution. The proposed model employs weights based on projection statistics to scale residuals and bound the influence of vertical outliers and bad leverage points on the latent functions estimates while exhibiting a high statistical efficiency at the Gaussian and thick tailed noise distributions. The proposed method is demonstrated by two real world problems and two numerical examples using datasets with additive errors following thick tailed distributions such as Students t, Laplace, and Cauchy distribution.
New Metrics to Encourage Innovation and Diversity in Information Retrieval Approaches
Authors: Mehmet Deniz Türkmen, Matthew Lease, Mucahid Kutlu
Date: 2023-01-19 00:00:00
Tasks: Diversity, Information Retrieval, Retrieval

In evaluation campaigns, participants often explore variations of popular, state-of-the-art baselines as a low-risk strategy to achieve competitive results. While effective, this can lead to local "hill climbing" rather than more radical and innovative departure from standard methods. Moreover, if many participants build on similar baselines, the overall diversity of approaches considered may be limited. In this work, we propose a new class of IR evaluation metrics intended to promote greater diversity of approaches in evaluation campaigns. Whereas traditional IR metrics focus on user experience, our two "innovation" metrics instead reward exploration of more divergent, higher-risk strategies finding relevant documents missed by other systems. Experiments on four TREC collections show that our metrics do change system rankings by rewarding systems that find such rare, relevant documents. This result is further supported by a controlled, synthetic data experiment, and a qualitative analysis. In addition, we show that our metrics achieve higher evaluation stability and discriminative power than the standard metrics we modify. To support reproducibility, we share our source code.
From 10 Blue Links Pages to Feature-Full Search Engine Results Pages -- Analysis of the Temporal Evolution of SERP Features
Authors: B. Oliveira, C. T. Lopes
Date: 2023-01-19 00:00:00

Web Search Engine Results Pages (SERP) are one of the most well-known and used web pages. These pages have started as simple ``10 blue links'' pages, but the information in SERP currently goes way beyond these links. Several features have been included in these pages to complement organic and sponsored results and attempt to provide answers to the query instead of just pointing to websites that might deliver that information. In this work, we analyze the appearance and evolution of SERP features in the two leading web search engines, Google Search and Microsoft Bing. Using a sample of SERP from the Internet Archive, we analyzed the appearance and evolution of these features. We found that SERP are becoming more diverse in terms of elements, aggregating content from different verticals and including more features that provide direct answers.
DynInt: Dynamic Interaction Modeling for Large-scale Click-Through Rate Prediction
Authors: YaChen Yan, Liubo Li
Date: 2023-01-03 00:00:00
Tasks: Click-Through Rate Prediction, Recommendation Systems

Learning feature interactions is the key to success for the large-scale CTR prediction in Ads ranking and recommender systems. In industry, deep neural network-based models are widely adopted for modeling such problems. Researchers proposed various neural network architectures for searching and modeling the feature interactions in an end-to-end fashion. However, most methods only learn static feature interactions and have not fully leveraged deep CTR models' representation capacity. In this paper, we propose a new model: DynInt. By extending Polynomial-Interaction-Network (PIN), which learns higher-order interactions recursively to be dynamic and data-dependent, DynInt further derived two modes for modeling dynamic higher-order interactions: dynamic activation and dynamic parameter. In dynamic activation mode, we adaptively adjust the strength of learned interactions by instance-aware activation gating networks. In dynamic parameter mode, we re-parameterize the parameters by different formulations and dynamically generate the parameters by instance-aware parameter generation networks. Through instance-aware gating mechanism and dynamic parameter generation, we enable the PIN to model dynamic interaction for potential industry applications. We implement the proposed model and evaluate the model performance on real-world datasets. Extensive experiment results demonstrate the efficiency and effectiveness of DynInt over state-of-the-art models.
DATSING: Data Augmented Time Series Forecasting with Adversarial Domain Adaptation
Authors: Hailin Hu, Mingjian Tang, Chengcheng Bai
Date: 2020-10-19 00:00:00
Proceeding: ACM International Conference on Information & Knowledge Management 2020 10
Tasks: Domain Adaptation, Time Series, Time Series Analysis, Time Series Forecasting, Transfer Learning, Univariate Time Series Forecasting

Due to the high temporal uncertainty and low signal-to-noise ratio, transfer learning for univariate time series forecasting remains a challenging task. In addition, data scarcity, which is commonly encountered in business forecasting, further limits the application of conventional transfer learning protocols. In this work, we have developed, DATSING, a transfer learning-based framework that effectively leverages cross-domain time series latent representations to augment target domain forecasting. In particular, we aim to transfer domain-invariant feature representations from a pre-trained stacked deep residual network to the target domains, so as to assist the prediction of each target time series. To effectively avoid noisy feature representations, we propose a two-phased framework which first clusters similar mixed domains time series data and then performs a fine-tuning procedure with domain adversarial regularization to achieve better out-of-sample generalization. Extensive experiments with real-world datasets have demonstrated that our method significantly improves the forecasting performance of the pre-trained model. DATSING has the unique potential to empower forecasting practitioners to unleash the power of cross-domain time series data.
Scale-Prior Deformable Convolution for Exemplar-Guided Class-Agnostic Counting
Authors: Wei Lin, Kunlin Yang, Xinzhu Ma, Junyu Gao, Lingbo Liu, Shinan Liu, Jun Hou, Shuai Yi, Antoni B. Chan
Date: 2022-12-21 00:00:00
Proceeding: Conference 2022 12
Tasks: Object Counting

Class-agnostic counting has recently emerged as a more practical counting task, which aims to predict the number and distribution of any exemplar objects, instead of counting specific categories like pedestrians or cars. However, recent methods are developed by designing suitable similarity matching rules between exemplars and query images, but ignoring the robustness of extracted features. To address this issue, we propose a scale-prior deformable convolution by integrating exemplars' information, \eg, scale, into the counting network backbone. As a result, the proposed counting network can extract semantic features of objects similar to the given exemplars and effectively filter irrelevant backgrounds. Besides, we find that traditional L2 and generalized loss are not suitable for class-agnostic counting due to the variety of object scales in different samples. Here we propose a scale-sensitive generalized loss to tackle this problem. It can adjust the cost function formulation according to the given exemplars, making the difference between prediction and ground truth more prominent. Extensive experiments show that our model obtains remarkable improvement and achieves state-of-the-art performance on a public class-agnostic counting benchmark. the source code is available at https://github.com/Elin24/SPDCN-CAC.