pwc-archive/papers-with-abstracts huggingface dataset:
ET5: A Novel End-to-end Framework for Conversational Machine Reading Comprehension
Authors: Xiao Zhang, Heyan Huang, Zewen Chi, Xian-Ling Mao
Date: 2022-09-23 00:00:00
Proceeding: COLING 2022 10
Tasks: Decision Making, Machine Reading Comprehension, Reading Comprehension

Conversational machine reading comprehension (CMRC) aims to assist computers to understand an natural language text and thereafter engage in a multi-turn conversation to answer questions related to the text. Existing methods typically require three steps: (1) decision making based on entailment reasoning; (2) span extraction if required by the above decision; (3) question rephrasing based on the extracted span. However, for nearly all these methods, the span extraction and question rephrasing steps cannot fully exploit the fine-grained entailment reasoning information in decision making step because of their relative independence, which will further enlarge the information gap between decision making and question phrasing. Thus, to tackle this problem, we propose a novel end-to-end framework for conversational machine reading comprehension based on shared parameter mechanism, called entailment reasoning T5 (ET5). Despite the lightweight of our proposed framework, experimental results show that the proposed ET5 achieves new state-of-the-art results on the ShARC leaderboard with the BLEU-4 score of 55.2. Our model and code are publicly available at https://github.com/Yottaxx/ET5.
News Category Dataset
Authors: Rishabh Misra
Date: 2022-09-23 00:00:00
Tasks: Articles

People rely on news to know what is happening around the world and inform their daily lives. In today's world, when the proliferation of fake news is rampant, having a large-scale and high-quality source of authentic news articles with the published category information is valuable to learning authentic news' Natural Language syntax and semantics. As part of this work, we present a News Category Dataset that contains around 210k news headlines from the year 2012 to 2022 obtained from HuffPost, along with useful metadata to enable various NLP tasks. In this paper, we also produce some novel insights from the dataset and describe various existing and potential applications of our dataset.
Zero-shot Domain Adaptation for Neural Machine Translation with Retrieved Phrase-level Prompts
Authors: Zewei Sun, Qingnan Jiang, ShuJian Huang, Jun Cao, Shanbo Cheng, Mingxuan Wang
Date: 2022-09-23 00:00:00
Tasks: Domain Adaptation, Machine Translation, Translation

Domain adaptation is an important challenge for neural machine translation. However, the traditional fine-tuning solution requires multiple extra training and yields a high cost. In this paper, we propose a non-tuning paradigm, resolving domain adaptation with a prompt-based method. Specifically, we construct a bilingual phrase-level database and retrieve relevant pairs from it as a prompt for the input sentences. By utilizing Retrieved Phrase-level Prompts (RePP), we effectively boost the translation quality. Experiments show that our method improves domain-specific machine translation for 6.2 BLEU scores and improves translation constraints for 11.5% accuracy without additional training.
Extending Word-Level Quality Estimation for Post-Editing Assistance
Authors: Yizhen Wei, Takehito Utsuro, Masaaki Nagata
Date: 2022-09-23 00:00:00
Tasks: Word Alignment, XLM-R

We define a novel concept called extended word alignment in order to improve post-editing assistance efficiency. Based on extended word alignment, we further propose a novel task called refined word-level QE that outputs refined tags and word-level correspondences. Compared to original word-level QE, the new task is able to directly point out editing operations, thus improves efficiency. To extract extended word alignment, we adopt a supervised method based on mBERT. To solve refined word-level QE, we firstly predict original QE tags by training a regression model for sequence tagging based on mBERT and XLM-R. Then, we refine original word tags with extended word alignment. In addition, we extract source-gap correspondences, meanwhile, obtaining gap tags. Experiments on two language pairs show the feasibility of our method and give us inspirations for further improvement.
Towards Faithful Model Explanation in NLP: A Survey
Authors: Qing Lyu, Marianna Apidianaki, Chris Callison-Burch
Date: 2022-09-22 00:00:00
Tasks: counterfactual, model, Survey

End-to-end neural Natural Language Processing (NLP) models are notoriously difficult to understand. This has given rise to numerous efforts towards model explainability in recent years. One desideratum of model explanation is faithfulness, i.e. an explanation should accurately represent the reasoning process behind the model's prediction. In this survey, we review over 110 model explanation methods in NLP through the lens of faithfulness. We first discuss the definition and evaluation of faithfulness, as well as its significance for explainability. We then introduce recent advances in faithful explanation, grouping existing approaches into five categories: similarity-based methods, analysis of model-internal structures, backpropagation-based methods, counterfactual intervention, and self-explanatory models. For each category, we synthesize its representative studies, strengths, and weaknesses. Finally, we summarize their common virtues and remaining challenges, and reflect on future work directions towards faithful explainability in NLP.
XF2T: Cross-lingual Fact-to-Text Generation for Low-Resource Languages
Authors: Shivprasad Sagare, Tushar Abhishek, Bhavyajeet Singh, Anubhav Sharma, Manish Gupta, Vasudeva Varma
Date: 2022-09-22 00:00:00
Tasks: Data-to-Text Generation, Descriptive, Question Answering, Text Generation

Multiple business scenarios require an automated generation of descriptive human-readable text from structured input data. Hence, fact-to-text generation systems have been developed for various downstream tasks like generating soccer reports, weather and financial reports, medical reports, person biographies, etc. Unfortunately, previous work on fact-to-text (F2T) generation has focused primarily on English mainly due to the high availability of relevant datasets. Only recently, the problem of cross-lingual fact-to-text (XF2T) was proposed for generation across multiple languages alongwith a dataset, XALIGN for eight languages. However, there has been no rigorous work on the actual XF2T generation problem. We extend XALIGN dataset with annotated data for four more languages: Punjabi, Malayalam, Assamese and Oriya. We conduct an extensive study using popular Transformer-based text generation models on our extended multi-lingual dataset, which we call XALIGNV2. Further, we investigate the performance of different text generation strategies: multiple variations of pretraining, fact-aware embeddings and structure-aware input encoding. Our extensive experiments show that a multi-lingual mT5 model which uses fact-aware embeddings with structure-aware input encoding leads to best results on average across the twelve languages. We make our code, dataset and model publicly available, and hope that this will help advance further research in this critical area.
Quantile-constrained Wasserstein projections for robust interpretability of numerical and machine learning models
Authors: Marouane Il Idrissi, Nicolas Bousquet, Fabrice Gamboa, Bertrand Iooss, Jean-Michel Loubes
Date: 2022-09-23 00:00:00
Tasks: Uncertainty Quantification

Robustness studies of black-box models is recognized as a necessary task for numerical models based on structural equations and predictive models learned from data. These studies must assess the model's robustness to possible misspecification of regarding its inputs (e.g., covariate shift). The study of black-box models, through the prism of uncertainty quantification (UQ), is often based on sensitivity analysis involving a probabilistic structure imposed on the inputs, while ML models are solely constructed from observed data. Our work aim at unifying the UQ and ML interpretability approaches, by providing relevant and easy-to-use tools for both paradigms. To provide a generic and understandable framework for robustness studies, we define perturbations of input information relying on quantile constraints and projections with respect to the Wasserstein distance between probability measures, while preserving their dependence structure. We show that this perturbation problem can be analytically solved. Ensuring regularity constraints by means of isotonic polynomial approximations leads to smoother perturbations, which can be more suitable in practice. Numerical experiments on real case studies, from the UQ and ML fields, highlight the computational feasibility of such studies and provide local and global insights on the robustness of black-box models to input perturbations.
Feature selection in stratification estimators of causal effects: lessons from potential outcomes, causal diagrams, and structural equations
Authors: P. Richard Hahn, Andrew Herren
Date: 2022-09-23 00:00:00
Tasks: feature selection, regression

What is the ideal regression (if any) for estimating average causal effects? We study this question in the setting of discrete covariates, deriving expressions for the finite-sample variance of various stratification estimators. This approach clarifies the fundamental statistical phenomena underlying many widely-cited results. Our exposition combines insights from three distinct methodological traditions for studying causal effect estimation: potential outcomes, causal diagrams, and structural models with additive errors.
A Unified Generative Framework based on Prompt Learning for Various Information Extraction Tasks
Authors: Zhigang Kan, Linhui Feng, Zhangyue Yin, Linbo Qiao, Xipeng Qiu, Dongsheng Li
Date: 2022-09-23 00:00:00
Tasks: Prompt Learning, Relation Extraction

Prompt learning is an effective paradigm that bridges gaps between the pre-training tasks and the corresponding downstream applications. Approaches based on this paradigm have achieved great transcendent results in various applications. However, it still needs to be answered how to design a unified framework based on the prompt learning paradigm for various information extraction tasks. In this paper, we propose a novel composable prompt-based generative framework, which could be applied to a wide range of tasks in the field of Information Extraction. Specifically, we reformulate information extraction tasks into the form of filling slots in pre-designed type-specific prompts, which consist of one or multiple sub-prompts. A strategy of constructing composable prompts is proposed to enhance the generalization ability to extract events in data-scarce scenarios. Furthermore, to fit this framework, we transform Relation Extraction into the task of determining semantic consistency in prompts. The experimental results demonstrate that our approach surpasses compared baselines on real-world datasets in data-abundant and data-scarce scenarios. Further analysis of the proposed framework is presented, as well as numerical experiments conducted to investigate impact factors of performance on various tasks.
Modeling and Leveraging Prerequisite Context in Recommendation
Authors: Hengchang Hu, Liangming Pan, Yiding Ran, Min-Yen Kan
Date: 2022-09-23 00:00:00
Tasks: Decision Making, Recommendation Systems

Prerequisites can play a crucial role in users' decision-making yet recommendation systems have not fully utilized such contextual background knowledge. Traditional recommendation systems (RS) mostly enrich user-item interactions where the context consists of static user profiles and item descriptions, ignoring the contextual logic and constraints that underlie them. For example, an RS may recommend an item on the condition that the user has interacted with another item as its prerequisite. Modeling prerequisite context from conceptual side information can overcome this weakness. We propose Prerequisite Driven Recommendation (PDR), a generic context-aware framework where prerequisite context is explicitly modeled to facilitate recommendation. We first design a Prerequisite Knowledge Linking (PKL) algorithm, to curate datasets facilitating PDR research. Employing it, we build a 75k+ high-quality prerequisite concept dataset which spans three domains. We then contribute PDRS, a neural instantiation of PDR. By jointly optimizing both the prerequisite learning and recommendation tasks through multi-layer perceptrons, we find PDRS consistently outperforms baseline models in all three domains, by an average margin of 7.41%. Importantly, PDRS performs especially well in cold-start scenarios with improvements of up to 17.65%.
Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation
Authors: Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, Yang Wang
Date: 2022-09-23 00:00:00
Tasks: Collaborative Filtering, Contrastive Learning, Session-Based Recommendations

Session-based recommendation (SBR) systems aim to utilize the user's short-term behavior sequence to predict the next item without the detailed user profile. Most recent works try to model the user preference by treating the sessions as between-item transition graphs and utilize various graph neural networks (GNNs) to encode the representations of pair-wise relations among items and their neighbors. Some of the existing GNN-based models mainly focus on aggregating information from the view of spatial graph structure, which ignores the temporal relations within neighbors of an item during message passing and the information loss results in a sub-optimal problem. Other works embrace this challenge by incorporating additional temporal information but lack sufficient interaction between the spatial and temporal patterns. To address this issue, inspired by the uniformity and alignment properties of contrastive learning techniques, we propose a novel framework called Session-based Recommendation with Spatio-Temporal Contrastive Learning Enhanced GNNs (RESTC). The idea is to supplement the GNN-based main supervised recommendation task with the temporal representation via an auxiliary cross-view contrastive learning mechanism. Furthermore, a novel global collaborative filtering graph (CFG) embedding is leveraged to enhance the spatial view in the main task. Extensive experiments demonstrate the significant performance of RESTC compared with the state-of-the-art baselines e.g., with an improvement as much as 27.08% gain on HR@20 and 20.10% gain on MRR@20.
A Trio-Method for Retinal Vessel Segmentation using Image Processing
Authors: Mahendra Kumar Gourisaria, Vinayak Singh, Manoj Sahni
Date: 2022-09-19 00:00:00
Tasks: Edge Detection, Retinal Vessel Segmentation, Segmentation

Inner Retinal neurons are a most essential part of the retina and they are supplied with blood via retinal vessels. This paper primarily focuses on the segmentation of retinal vessels using a triple preprocessing approach. DRIVE database was taken into consideration and preprocessed by Gabor Filtering, Gaussian Blur, and Edge Detection by Sobel and Pruning. Segmentation was driven out by 2 proposed U-Net architectures. Both the architectures were compared in terms of all the standard performance metrics. Preprocessing generated varied interesting results which impacted the results shown by the UNet architectures for segmentation. This real-time deployment can help in the efficient pre-processing of images with better segmentation and detection.
Adversarial Catoptric Light: An Effective, Stealthy and Robust Physical-World Attack to DNNs
Authors: Chengyin Hu, Weiwen Shi
Date: 2022-09-19 00:00:00

Deep neural networks (DNNs) have demonstrated exceptional success across various tasks, underscoring the need to evaluate the robustness of advanced DNNs. However, traditional methods using stickers as physical perturbations to deceive classifiers present challenges in achieving stealthiness and suffer from printing loss. Recent advancements in physical attacks have utilized light beams such as lasers and projectors to perform attacks, where the optical patterns generated are artificial rather than natural. In this study, we introduce a novel physical attack, adversarial catoptric light (AdvCL), where adversarial perturbations are generated using a common natural phenomenon, catoptric light, to achieve stealthy and naturalistic adversarial attacks against advanced DNNs in a black-box setting. We evaluate the proposed method in three aspects: effectiveness, stealthiness, and robustness. Quantitative results obtained in simulated environments demonstrate the effectiveness of the proposed method, and in physical scenarios, we achieve an attack success rate of 83.5%, surpassing the baseline. We use common catoptric light as a perturbation to enhance the stealthiness of the method and make physical samples appear more natural. Robustness is validated by successfully attacking advanced and robust DNNs with a success rate over 80% in all cases. Additionally, we discuss defense strategy against AdvCL and put forward some light-based physical attacks.
GSP-Based MAP Estimation of Graph Signals
Authors: Guy Sagi, Tirza Routtenberg
Date: 2022-09-23 00:00:00
Tasks: State Estimation

In this paper, we consider the problem of recovering random graph signals from nonlinear measurements. We formulate the maximum a-posteriori probability (MAP) estimator, which results in a nonconvex optimization problem. Conventional iterative methods for minimizing nonconvex problems are sensitive to the initialization, have high computational complexity, and do not utilize the underlying graph structure behind the data. In this paper we propose two new estimators that are both based on the Gauss-Newton method: 1) the elementwise graph-frequency-domain MAP (eGFD-MAP) estimator; and 2) the graph signal processing MAP (GSP-MAP) estimator. At each iteration, these estimators are updated by the outputs of two graph filters, with the previous state estimator and the residual as the input graph signals. The eGFD-MAP estimator is an ad-hoc method that minimizes the MAP objective function in the graph frequency domain and neglects mixed-derivatives of different graph frequencies in the Jacobian matrix as well as off-diagonal elements in the covariance matrices. Consequently, it updates the elements of the graph signal independently, which reduces the computational complexity compared to the conventional MAP estimator. The GSP-MAP estimator is based on optimizing the graph filters at each iteration of the Gauss-Newton algorithm. We state conditions under which the eGFD-MAP and GSP- MAP estimators coincide with the MAP estimator, in the case of an observation model with orthogonal graph frequencies. We evaluate the performance of the estimators for nonlinear graph signal recovery tasks with synthetic data and with the real-world problem of state estimation in power systems. These simulations show the advantages of the proposed estimators in terms of computational complexity, mean-squared-error, and robustness to the initialization of the iterative algorithms.
The Kriston AI System for the VoxCeleb Speaker Recognition Challenge 2022
Authors: Qutang Cai, Guoqiang Hong, Zhijian Ye, Ximin Li, Haizhou Li
Date: 2022-09-23 00:00:00
Tasks: Action Detection, Activity Detection, Clustering, Speaker Recognition

This technical report describes our system for track 1, 2 and 4 of the VoxCeleb Speaker Recognition Challenge 2022 (VoxSRC-22). By combining several ResNet variants, our submission for track 1 attained a minDCF of 0:090 with EER 1:401%. By further incorporating three fine-tuned pre-trained models, our submission for track 2 achieved a minDCF of 0:072 with EER 1:119%. For track 4, our system consisted of voice activity detection (VAD), speaker embedding extraction, agglomerative hierarchical clustering (AHC) followed by a re-clustering step based on a Bayesian hidden Markov model and overlapped speech detection and handling. Our submission for track 4 achieved a diarisation error rate (DER) of 4.86%. The submissions all ranked the 2nd places for the corresponding tracks.
Robust adaptive MPC using control contraction metrics
Authors: András Sasfi, Melanie N. Zeilinger, Johannes Köhler
Date: 2022-09-23 00:00:00
Tasks: Model Predictive Control

We present a robust adaptive model predictive control (MPC) framework for nonlinear continuous-time systems with bounded parametric uncertainty and additive disturbance. We utilize general control contraction metrics (CCMs) to parameterize a homothetic tube around a nominal prediction that contains all uncertain trajectories. Furthermore, we incorporate model adaptation using set-membership estimation. As a result, the proposed MPC formulation is applicable to a large class of nonlinear systems, reduces conservatism during online operation, and guarantees robust constraint satisfaction and convergence to a neighborhood of the desired setpoint. One of the main technical contributions is the derivation of corresponding tube dynamics based on CCMs that account for the state and input dependent nature of the model mismatch. Furthermore, we online optimize over the nominal parameter, which enables general set-membership updates for the parametric uncertainty in the MPC. Benefits of the proposed homothetic tube MPC and online adaptation are demonstrated using a numerical example involving a planar quadrotor.
Feedforward Control in the Presence of Input Nonlinearities: A Learning-based Approach
Authors: Jilles van Hulst, Maurice Poot, Dragan Kostić, Kai Wa Yan, Jim Portegies, Tom Oomen
Date: 2022-09-23 00:00:00

Advanced feedforward control methods enable mechatronic systems to perform varying motion tasks with extreme accuracy and throughput. The aim of this paper is to develop a data-driven feedforward controller that addresses input nonlinearities, which are common in typical applications such as semiconductor back-end equipment. The developed method consists of parametric inverse-model feedforward that is optimized for tracking error reduction by exploiting ideas from iterative learning control. Results on a simulated set-up indicate improved performance over existing identification methods for systems with nonlinearities at the input.
Treatment Effects with Multidimensional Unobserved Heterogeneity: Identification of the Marginal Treatment Effect
Authors: Toshiki Tsuda
Date: 2022-09-23 00:00:00
Tasks: Multiple-choice

This paper establishes sufficient conditions for the identification of the marginal treatment effects with multivalued treatments. Our model is based on a multinomial choice model with utility maximization. Our MTE generalizes the MTE defined in Heckman and Vytlacil (2005) in binary treatment models. As in the binary case, we can interpret the MTE as the treatment effect for persons who are indifferent between two treatments at a particular level. Our MTE enables one to obtain the treatment effects of those with specific preference orders over the choice set. Further, our results can identify other parameters such as the marginal distribution of potential outcomes.
Determinants Of Migration: Linear regression Analysis in Indian Context
Authors: Soumik Ghosh, Arpan Chakraborty
Date: 2022-09-23 00:00:00
Tasks: regression

Environmental degradation, global pandemic and severing natural resource related problems cater to increase demand resulting from migration is nightmare for all of us. Huge flocks of people are rushing towards to earn, to live and to lead a better life. This they do for their own development often ignoring the environmental cost. With existing model, this paper looks at out migration (interstate) within India focusing on the various proximate and fundamental causes relating to migration. The author deploys OLS to see those fundamental causes. Obviously, these are not exhaustive cause, but definitely plays a role in migration decision of individual. Finally, this paper advocates for some policy prescription to cope with this problem.