pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Infer from Unlabeled Data: A Semi-supervised Learning Approach for Robust Natural Language Inference
Authors: Mobashir Sadat, Cornelia Caragea
Date: 2022-11-05 00:00:00
Tasks: Language Modelling, Natural Language Inference, RTE, Sentence, Sentence Classification

Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE) aims at predicting the relation between a pair of sentences (premise and hypothesis) as entailment, contradiction or semantic independence. Although deep learning models have shown promising performance for NLI in recent years, they rely on large scale expensive human-annotated datasets. Semi-supervised learning (SSL) is a popular technique for reducing the reliance on human annotation by leveraging unlabeled data for training. However, despite its substantial success on single sentence classification tasks where the challenge in making use of unlabeled data is to assign "good enough" pseudo-labels, for NLI tasks, the nature of unlabeled data is more complex: one of the sentences in the pair (usually the hypothesis) along with the class label are missing from the data and require human annotations, which makes SSL for NLI more challenging. In this paper, we propose a novel way to incorporate unlabeled data in SSL for NLI where we use a conditional language model, BART to generate the hypotheses for the unlabeled sentences (used as premises). Our experiments show that our SSL framework successfully exploits unlabeled data and substantially improves the performance of four NLI datasets in low-resource settings. We release our code at: https://github.com/msadat3/SSL_for_NLI.
Privacy-Preserving Models for Legal Natural Language Processing
Authors: Ying Yin, Ivan Habernal
Date: 2022-11-05 00:00:00
Tasks: Domain Adaptation, Privacy Preserving, Self-Supervised Learning

Pre-training large transformer models with in-domain data improves domain adaptation and helps gain performance on the domain-specific downstream tasks. However, sharing models pre-trained on potentially sensitive data is prone to adversarial privacy attacks. In this paper, we asked to which extent we can guarantee privacy of pre-training data and, at the same time, achieve better downstream performance on legal tasks without the need of additional labeled data. We extensively experiment with scalable self-supervised learning of transformer models under the formal paradigm of differential privacy and show that under specific training configurations we can improve downstream performance without sacrifying privacy protection for the in-domain data. Our main contribution is utilizing differential privacy for large-scale pre-training of transformer language models in the legal NLP domain, which, to the best of our knowledge, has not been addressed before.
The Legal Argument Reasoning Task in Civil Procedure
Authors: Leonard Bongard, Lena Held, Ivan Habernal
Date: 2022-11-05 00:00:00
Tasks: Benchmarking

We present a new NLP task and dataset from the domain of the U.S. civil procedure. Each instance of the dataset consists of a general introduction to the case, a particular question, and a possible solution argument, accompanied by a detailed analysis of why the argument applies in that case. Since the dataset is based on a book aimed at law students, we believe that it represents a truly complex task for benchmarking modern legal language models. Our baseline evaluation shows that fine-tuning a legal transformer provides some advantage over random baseline models, but our analysis reveals that the actual ability to infer legal arguments remains a challenging open research question.
HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models
Authors: Yizhi Li, Ge Zhang, Bohao Yang, Chenghua Lin, Shi Wang, Anton Ragni, Jie Fu
Date: 2022-11-05 00:00:00
Tasks: Fairness

Fairness has become a trending topic in natural language processing (NLP), which addresses biases targeting certain social groups such as genders and religions. However, regional bias in language models (LMs), a long-standing global discrimination problem, still remains unexplored. This paper bridges the gap by analysing the regional bias learned by the pre-trained language models that are broadly used in NLP tasks. In addition to verifying the existence of regional bias in LMs, we find that the biases on regional groups can be strongly influenced by the geographical clustering of the groups. We accordingly propose a HiErarchical Regional Bias evaluation method (HERB) utilising the information from the sub-region clusters to quantify the bias in pre-trained LMs. Experiments show that our hierarchical metric can effectively evaluate the regional bias with respect to comprehensive topics and measure the potential regional bias that can be propagated to downstream tasks. Our codes are available at https://github.com/Bernard-Yang/HERB.
BEKG: A Built Environment Knowledge Graph
Authors: Xiaojun Yang, Haoyu Zhong, Penglin Du, Keyi Zhou, Xingjin Lai, Zhengdong Wang, Yik Lun Lau, Yangqiu Song, Liyaning Tang
Date: 2022-11-05 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation Extraction, Retrieval

Practices in the built environment have become more digitalized with the rapid development of modern design and construction technologies. However, the requirement of practitioners or scholars to gather complicated professional knowledge in the built environment has not been satisfied yet. In this paper, more than 80,000 paper abstracts in the built environment field were obtained to build a knowledge graph, a knowledge base storing entities and their connective relations in a graph-structured data model. To ensure the retrieval accuracy of the entities and relations in the knowledge graph, two well-annotated datasets have been created, containing 2,000 instances and 1,450 instances each in 29 relations for the named entity recognition task and relation extraction task respectively. These two tasks were solved by two BERT-based models trained on the proposed dataset. Both models attained an accuracy above 85% on these two tasks. More than 200,000 high-quality relations and entities were obtained using these models to extract all abstract data. Finally, this knowledge graph is presented as a self-developed visualization system to reveal relations between various entities in the domain. Both the source code and the annotated dataset can be found here: https://github.com/HKUST-KnowComp/BEKG.
Aligning Recommendation and Conversation via Dual Imitation
Authors: Jinfeng Zhou, Bo wang, Minlie Huang, Dongming Zhao, Kun Huang, Ruifang He, Yuexian Hou
Date: 2022-11-05 00:00:00
Tasks: Conversational Recommendation, Recommendation Systems

Human conversations of recommendation naturally involve the shift of interests which can align the recommendation actions and conversation process to make accurate recommendations with rich explanations. However, existing conversational recommendation systems (CRS) ignore the advantage of user interest shift in connecting recommendation and conversation, which leads to an ineffective loose coupling structure of CRS. To address this issue, by modeling the recommendation actions as recommendation paths in a knowledge graph (KG), we propose DICR (Dual Imitation for Conversational Recommendation), which designs a dual imitation to explicitly align the recommendation paths and user interest shift paths in a recommendation module and a conversation module, respectively. By exchanging alignment signals, DICR achieves bidirectional promotion between recommendation and conversation modules and generates high-quality responses with accurate recommendations and coherent explanations. Experiments demonstrate that DICR outperforms the state-of-the-art models on recommendation and conversation performance with automatic, human, and novel explainability metrics.
Evaluation of Automated Speech Recognition Systems for Conversational Speech: A Linguistic Perspective
Authors: Hannaneh B. Pasandi, Haniyeh B. Pasandi
Date: 2022-11-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Automatic speech recognition (ASR) meets more informal and free-form input data as voice user interfaces and conversational agents such as the voice assistants such as Alexa, Google Home, etc., gain popularity. Conversational speech is both the most difficult and environmentally relevant sort of data for speech recognition. In this paper, we take a linguistic perspective, and take the French language as a case study toward disambiguation of the French homophones. Our contribution aims to provide more insight into human speech transcription accuracy in conditions to reproduce those of state-of-the-art ASR systems, although in a much focused situation. We investigate a case study involving the most common errors encountered in the automatic transcription of French language.
Hierarchical Multi-Label Classification of Scientific Documents
Authors: Mobashir Sadat, Cornelia Caragea
Date: 2022-11-05 00:00:00
Tasks: Classification, Hierarchical Multi-label Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Multi-Label Text Classification, Multi-Task Learning, Text Classification, Topic Classification

Automatic topic classification has been studied extensively to assist managing and indexing scientific documents in a digital collection. With the large number of topics being available in recent years, it has become necessary to arrange them in a hierarchy. Therefore, the automatic classification systems need to be able to classify the documents hierarchically. In addition, each paper is often assigned to more than one relevant topic. For example, a paper can be assigned to several topics in a hierarchy tree. In this paper, we introduce a new dataset for hierarchical multi-label text classification (HMLTC) of scientific papers called SciHTC, which contains 186,160 papers and 1,233 categories from the ACM CCS tree. We establish strong baselines for HMLTC and propose a multi-task learning approach for topic classification with keyword labeling as an auxiliary task. Our best model achieves a Macro-F1 score of 34.57% which shows that this dataset provides significant research opportunities on hierarchical scientific topic classification. We make our dataset and code available on Github.
LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducers
Authors: Peidong Wang, Eric Sun, Jian Xue, Yu Wu, Long Zhou, Yashesh Gaur, Shujie Liu, Jinyu Li
Date: 2022-11-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Identification, speech-recognition, Speech Recognition, Text Generation, Translation

Automatic speech recognition (ASR) and speech translation (ST) can both use neural transducers as the model structure. It is thus possible to use a single transducer model to perform both tasks. In real-world applications, such joint ASR and ST models may need to be streaming and do not require source language identification (i.e. language-agnostic). In this paper, we propose LAMASSU, a streaming language-agnostic multilingual speech recognition and translation model using neural transducers. Based on the transducer model structure, we propose four methods, a unified joint and prediction network for multilingual output, a clustered multilingual encoder, target language identification for encoder, and connectionist temporal classification regularization. Experimental results show that LAMASSU not only drastically reduces the model size but also reaches the performances of monolingual ASR and bilingual ST models.
1Cademy @ Causal News Corpus 2022: Leveraging Self-Training in Causality Classification of Socio-Political Event Data
Authors: Adam Nik, Ge Zhang, Xingran Chen, Mingyu Li, Jie Fu
Date: 2022-11-04 00:00:00

This paper details our participation in the Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE) workshop @ EMNLP 2022, where we take part in Subtask 1 of Shared Task 3. We approach the given task of event causality detection by proposing a self-training pipeline that follows a teacher-student classifier method. More specifically, we initially train a teacher model on the true, original task data, and use that teacher model to self-label data to be used in the training of a separate student model for the final task prediction. We test how restricting the number of positive or negative self-labeled examples in the self-training process affects classification performance. Our final results show that using self-training produces a comprehensive performance improvement across all models and self-labeled training sets tested within the task of event causality sequence classification. On top of that, we find that self-training performance did not diminish even when restricting either positive/negative examples used in training. Our code is be publicly available at https://github.com/Gzhang-umich/1CademyTeamOfCASE.
An approach to standardize, automate omni-channel and AI transactional digital service creation
Authors: Antoine Aamarcha, Martin Caussanel, Hadrien Lanneau, Kevin Mege, Florian Peyron
Date: 2022-11-04 00:00:00

Our work is at the crossroads of two categories of technologies. On the one hand, omnichannel digit services, to address the needs of users in the most seamless way. On the other hand, low code approaches, to build simply even complex software applications. In this twofold context, we propose DSUL (Digital Service Universal Language). It allows to build omnichannel services with minimal work from their designers. We describe precisely how DSUL operates, and its innovation in regard to the state of the art. We also consider the various methods to evaluate this framework.
A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environments
Authors: Ke Li, Renzhi Chen, Xin Yao
Date: 2022-11-05 00:00:00
Tasks: Transfer Learning

Many real-world problems are usually computationally costly and the objective functions evolve over time. Data-driven, a.k.a. surrogate-assisted, evolutionary optimization has been recognized as an effective approach for tackling expensive black-box optimization problems in a static environment whereas it has rarely been studied under dynamic environments. This paper proposes a simple but effective transfer learning framework to empower data-driven evolutionary optimization to solve dynamic optimization problems. Specifically, it applies a hierarchical multi-output Gaussian process to capture the correlation between data collected from different time steps with a linearly increased number of hyperparameters. Furthermore, an adaptive source task selection along with a bespoke warm staring initialization mechanisms are proposed to better leverage the knowledge extracted from previous optimization exercises. By doing so, the data-driven evolutionary optimization can jump start the optimization in the new environment with a strictly limited computational budget. Experiments on synthetic benchmark test problems and a real-world case study demonstrate the effectiveness of our proposed algorithm against nine state-of-the-art peer algorithms.
Beyond Conjugacy for Chain Event Graph Model Selection
Authors: Aditi Shenvi, Silvia Liverani
Date: 2022-11-07 00:00:00
Tasks: model, Model Selection

Chain event graphs are a family of probabilistic graphical models that generalise Bayesian networks and have been successfully applied to a wide range of domains. Unlike Bayesian networks, these models can encode context-specific conditional independencies as well as asymmetric developments within the evolution of a process. More recently, new model classes belonging to the chain event graph family have been developed for modelling time-to-event data to study the temporal dynamics of a process. However, existing model selection algorithms for chain event graphs and its variants rely on all parameters having conjugate priors. This is unrealistic for many real-world applications. In this paper, we propose a mixture modelling approach to model selection in chain event graphs that does not rely on conjugacy. Moreover, we also show that this methodology is more amenable to being robustly scaled than the existing model selection algorithms used for this family. We demonstrate our techniques on simulated datasets.
Multi-Fidelity Cost-Aware Bayesian Optimization
Authors: Zahra Zanjani Foumani, Mehdi Shishehbor, Amin Yousefpour, Ramin Bostanabad
Date: 2022-11-04 00:00:00
Tasks: Bayesian Optimization, Drug Discovery

Bayesian optimization (BO) is increasingly employed in critical applications such as materials design and drug discovery. An increasingly popular strategy in BO is to forgo the sole reliance on high-fidelity data and instead use an ensemble of information sources which provide inexpensive low-fidelity data. The overall premise of this strategy is to reduce the overall sampling costs by querying inexpensive low-fidelity sources whose data are correlated with high-fidelity samples. Here, we propose a multi-fidelity cost-aware BO framework that dramatically outperforms the state-of-the-art technologies in terms of efficiency, consistency, and robustness. We demonstrate the advantages of our framework on analytic and engineering problems and argue that these benefits stem from our two main contributions: (1) we develop a novel acquisition function for multi-fidelity cost-aware BO that safeguards the convergence against the biases of low-fidelity data, and (2) we tailor a newly developed emulator for multi-fidelity BO which enables us to not only simultaneously learn from an ensemble of multi-fidelity datasets, but also identify the severely biased low-fidelity sources that should be excluded from BO.
Debiasing Graph Transfer Learning via Item Semantic Clustering for Cross-Domain Recommendations
Authors: Zhi Li, Daichi Amagata, Yihong Zhang, Takahiro Hara, Shuichiro Haruta, Kei Yonekawa, Mori Kurokawa
Date: 2022-11-07 00:00:00
Tasks: Clustering, Recommendation Systems, Transfer Learning

Deep learning-based recommender systems may lead to over-fitting when lacking training interaction data. This over-fitting significantly degrades recommendation performances. To address this data sparsity problem, cross-domain recommender systems (CDRSs) exploit the data from an auxiliary source domain to facilitate the recommendation on the sparse target domain. Most existing CDRSs rely on overlapping users or items to connect domains and transfer knowledge. However, matching users is an arduous task and may involve privacy issues when data comes from different companies, resulting in a limited application for the above CDRSs. Some studies develop CDRSs that require no overlapping users and items by transferring learned user interaction patterns. However, they ignore the bias in user interaction patterns between domains and hence suffer from an inferior performance compared with single-domain recommender systems. In this paper, based on the above findings, we propose a novel CDRS, namely semantic clustering enhanced debiasing graph neural recommender system (SCDGN), that requires no overlapping users and items and can handle the domain bias. More precisely, SCDGN semantically clusters items from both domains and constructs a cross-domain bipartite graph generated from item clusters and users. Then, the knowledge is transferred via this cross-domain user-cluster graph from the source to the target. Furthermore, we design a debiasing graph convolutional layer for SCDGN to extract unbiased structural knowledge from the cross-domain user-cluster graph. Our Experimental results on three public datasets and a pair of proprietary datasets verify the effectiveness of SCDGN over state-of-the-art models in terms of cross-domain recommendations.
Two-Server Oblivious Transfer for Quantum Messages
Authors: Masahito Hayashi, Seunghoan Song
Date: 2022-11-07 00:00:00
Tasks: Vocal Bursts Valence Prediction

Oblivious transfer is considered as a cryptographic primitive task for quantum information processing over quantum network. Although it is possible with two servers, any existing protocol works only with classical messages. We propose two-server oblivious transfer protocols for quantum messages.
Deep Factorization Model for Robust Recommendation
Authors: Li Wang, Qiang Zhao, Wei Wang
Date: 2022-11-05 00:00:00
Tasks: model, Recommendation Systems

Recently, malevolent user hacking has become a huge problem for real-world companies. In order to learn predictive models for recommender systems, factorization techniques have been developed to deal with user-item ratings. In this paper, we suggest a broad architecture of a factorization model with adversarial training to get over these issues. The effectiveness of our systems is demonstrated by experimental findings on real-world datasets.
One Person, One Model--Learning Compound Router for Sequential Recommendation
Authors: Zhiding Liu, Mingyue Cheng, Zhi Li, Qi Liu, Enhong Chen
Date: 2022-11-05 00:00:00
Tasks: Sequential Recommendation

Deep learning has brought significant breakthroughs in sequential recommendation (SR) for capturing dynamic user interests. A series of recent research revealed that models with more parameters usually achieve optimal performance for SR tasks, inevitably resulting in great challenges for deploying them in real systems. Following the simple assumption that light networks might already suffice for certain users, in this work, we propose CANet, a conceptually simple yet very scalable framework for assigning adaptive network architecture in an input-dependent manner to reduce unnecessary computation. The core idea of CANet is to route the input user behaviors with a light-weighted router module. Specifically, we first construct the routing space with various submodels parameterized in terms of multiple model dimensions such as the number of layers, hidden size and embedding size. To avoid extra storage overhead of the routing space, we employ a weight-slicing schema to maintain all the submodels in exactly one network. Furthermore, we leverage several solutions to solve the discrete optimization issues caused by the router module. Thanks to them, CANet could adaptively adjust its network architecture for each input in an end-to-end manner, in which the user preference can be effectively captured. To evaluate our work, we conduct extensive experiments on benchmark datasets. Experimental results show that CANet reduces computation by 55 ~ 65% while preserving the accuracy of the original model. Our codes are available at https://github.com/icantnamemyself/CANet.
Arabic Text Mining
Authors: Sumaia Mohammed AL-Ghuribi, Shahrul Azman Mohd Noah
Date: 2022-11-04 00:00:00
Tasks: Retrieval, text-classification, Text Classification, Text Retrieval

The rapid growth of the internet has increased the number of online texts. This led to the rapid growth of the number of online texts in the Arabic language. The enormous amount of text must be organized into classes to make the analysis process and text retrieval easier. Text classification is, therefore, a key component of text mining. There are numerous systems and approaches for categorizing literature in English, European (French, German, Spanish), and Asian (Chinese, Japanese). In contrast, there are relatively few studies on categorizing Arabic literature due to the difficulty of the Arabic language. In this work, a brief explanation of key ideas relevant to Arabic text mining are introduced then a new classification system for the Arabic language is presented using light stemming and Classifier Na\"ive Bayesian (CNB). Texts from two classes: politics and sports, are included in our corpus. Some texts are added to the system, and the system correctly classified them, demonstrating the effectiveness of the system.