pwc-archive/papers-with-abstracts huggingface dataset:
Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model
Authors: Zhuoer Wang, Yicheng Wang, Ziwei Zhu, James Caverlee
Date: 2023-10-19 00:00:00
Tasks: Data Augmentation, Question Generation, Question-Generation

Question generation is a widely used data augmentation approach with extensive applications, and extracting qualified candidate answers from context passages is a critical step for most question generation systems. However, existing methods for candidate answer extraction are reliant on linguistic rules or annotated data that face the partial annotation issue and challenges in generalization. To overcome these limitations, we propose a novel unsupervised candidate answer extraction approach that leverages the inherent structure of context passages through a Differentiable Masker-Reconstructor (DMR) Model with the enforcement of self-consistency for picking up salient information tokens. We curated two datasets with exhaustively-annotated answers and benchmark a comprehensive set of supervised and unsupervised candidate answer extraction methods. We demonstrate the effectiveness of the DMR model by showing its performance is superior among unsupervised methods and comparable to supervised methods.
Do Language Models Learn about Legal Entity Types during Pretraining?
Authors: Claire Barale, Michael Rovatsos, Nehal Bhuta
Date: 2023-10-19 00:00:00
Tasks: Decoder, Entity Typing, Reading Comprehension

Language Models (LMs) have proven their ability to acquire diverse linguistic knowledge during the pretraining phase, potentially serving as a valuable source of incidental supervision for downstream tasks. However, there has been limited research conducted on the retrieval of domain-specific knowledge, and specifically legal knowledge. We propose to explore the task of Entity Typing, serving as a proxy for evaluating legal knowledge as an essential aspect of text comprehension, and a foundational task to numerous downstream legal NLP applications. Through systematic evaluation and analysis and two types of prompting (cloze sentences and QA-based templates) and to clarify the nature of these acquired cues, we compare diverse types and lengths of entities both general and domain-specific entities, semantics or syntax signals, and different LM pretraining corpus (generic and legal-oriented) and architectures (encoder BERT-based and decoder-only with Llama2). We show that (1) Llama2 performs well on certain entities and exhibits potential for substantial improvement with optimized prompt templates, (2) law-oriented LMs show inconsistent performance, possibly due to variations in their training corpus, (3) LMs demonstrate the ability to type entities even in the case of multi-token entities, (4) all models struggle with entities belonging to sub-domains of the law (5) Llama2 appears to frequently overlook syntactic cues, a shortcoming less present in BERT-based architectures.
GARI: Graph Attention for Relative Isomorphism of Arabic Word Embeddings
Authors: Muhammad Asif Ali, Maha Alshmrani, Jianbin Qin, Yan Hu, Di Wang
Date: 2023-10-19 00:00:00
Tasks: Graph Attention, Word Embeddings

Bilingual Lexical Induction (BLI) is a core challenge in NLP, it relies on the relative isomorphism of individual embedding spaces. Existing attempts aimed at controlling the relative isomorphism of different embedding spaces fail to incorporate the impact of semantically related words in the model training objective. To address this, we propose GARI that combines the distributional training objectives with multiple isomorphism losses guided by the graph attention network. GARI considers the impact of semantical variations of words in order to define the relative isomorphism of the embedding spaces. Experimental evaluation using the Arabic language data set shows that GARI outperforms the existing research by improving the average P@1 by a relative score of up to 40.95% and 76.80% for in-domain and domain mismatch settings respectively. We release the codes for GARI at https://github.com/asif6827/GARI.
An Improved Artificial Fish Swarm Algorithm for Solving the Problem of Investigation Path Planning
Authors: Qian Huang, Weiwen Qian, Chang Li, Xuan Ding
Date: 2023-10-20 00:00:00
Tasks: Decision Making, Traveling Salesman Problem

Informationization is a prevailing trend in today's world. The increasing demand for information in decision-making processes poses significant challenges for investigation activities, particularly in terms of effectively allocating limited resources to plan investigation programs. This paper addresses the investigation path planning problem by formulating it as a multi-traveling salesman problem (MTSP). Our objective is to minimize costs, and to achieve this, we propose a chaotic artificial fish swarm algorithm based on multiple population differential evolution (DE-CAFSA). To overcome the limitations of the artificial fish swarm algorithm, such as low optimization accuracy and the inability to consider global and local information, we incorporate adaptive field of view and step size adjustments, replace random behavior with the 2-opt operation, and introduce chaos theory and sub-optimal solutions to enhance optimization accuracy and search performance. Additionally, we integrate the differential evolution algorithm to create a hybrid algorithm that leverages the complementary advantages of both approaches. Experimental results demonstrate that DE-CAFSA outperforms other algorithms on various public datasets of different sizes, as well as showcasing excellent performance on the examples proposed in this study.
A Study of Fitness Gains in Evolving Finite State Machines
Authors: Gabor Zoltai, Yue Xie, Frank Neumann
Date: 2023-10-20 00:00:00

Among the wide variety of evolutionary computing models, Finite State Machines (FSMs) have several attractions for fundamental research. They are easy to understand in concept and can be visualised clearly in simple cases. They have a ready fitness criterion through their relationship with Regular Languages. They have also been shown to be tractably evolvable, even up to exhibiting evidence of open-ended evolution in specific scenarios. In addition to theoretical attraction, they also have industrial applications, as a paradigm of both automated and user-initiated control. Improving the understanding of the factors affecting FSM evolution has relevance to both computer science and practical optimisation of control. We investigate an evolutionary scenario of FSMs adapting to recognise one of a family of Regular Languages by categorising positive and negative samples, while also being under a counteracting selection pressure that favours fewer states. The results appear to indicate that longer strings provided as samples reduce the speed of fitness gain, when fitness is measured against a fixed number of sample strings. We draw the inference that additional information from longer strings is not sufficient to compensate for sparser coverage of the combinatorial space of positive and negative sample strings.
Interaction Screening and Pseudolikelihood Approaches for Tensor Learning in Ising Models
Authors: Tianyu Liu, Somabha Mukherjee
Date: 2023-10-20 00:00:00

In this paper, we study two well known methods of Ising structure learning, namely the pseudolikelihood approach and the interaction screening approach, in the context of tensor recovery in $k$-spin Ising models. We show that both these approaches, with proper regularization, retrieve the underlying hypernetwork structure using a sample size logarithmic in the number of network nodes, and exponential in the maximum interaction strength and maximum node-degree. We also track down the exact dependence of the rate of tensor recovery on the interaction order $k$, that is allowed to grow with the number of samples and nodes, for both the approaches. We then provide a comparative discussion of the performance of the two approaches based on simulation studies, which also demonstrates the exponential dependence of the tensor recovery rate on the maximum coupling strength. Our tensor recovery methods are then applied on gene data taken from the Curated Microarray Database (CuMiDa), where we focus on understanding the important genes related to hepatocellular carcinoma.
Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language
Authors: Zekai Qu, Ruobing Xie, Chaojun Xiao, Yuan YAO, Zhiyuan Liu, Fengzong Lian, Zhanhui Kang, Jie zhou
Date: 2023-10-20 00:00:00
Tasks: Informativeness, Language Modeling, Language Modelling, Sequential Recommendation

With the thriving of pre-trained language model (PLM) widely verified in various of NLP tasks, pioneer efforts attempt to explore the possible cooperation of the general textual information in PLM with the personalized behavioral information in user historical behavior sequences to enhance sequential recommendation (SR). However, despite the commonalities of input format and task goal, there are huge gaps between the behavioral and textual information, which obstruct thoroughly modeling SR as language modeling via PLM. To bridge the gap, we propose a novel Unified pre-trained language model enhanced sequential recommendation (UPSR), aiming to build a unified pre-trained recommendation model for multi-domain recommendation tasks. We formally design five key indicators, namely naturalness, domain consistency, informativeness, noise & ambiguity, and text length, to guide the text-item adaptation and behavior sequence-text sequence adaptation differently for pre-training and fine-tuning stages, which are essential but under-explored by previous works. In experiments, we conduct extensive evaluations on seven datasets with both tuning and zero-shot settings and achieve the overall best performance. Comprehensive model analyses also provide valuable insights for behavior modeling via PLM, shedding light on large pre-trained recommendation models. The source codes will be released in the future.
Towards Multi-Subsession Conversational Recommendation
Authors: Yu Ji, Qi Shen, Shixuan Zhu, Hang Yu, Yiming Zhang, Chuan Cui, Zhihua Wei
Date: 2023-10-20 00:00:00
Tasks: Attribute, Conversational Recommendation, Recommendation Systems

Conversational recommendation systems (CRS) could acquire dynamic user preferences towards desired items through multi-round interactive dialogue. Previous CRS mainly focuses on the single conversation (subsession) that user quits after a successful recommendation, neglecting the common scenario where user has multiple conversations (multi-subsession) over a short period. Therefore, we propose a novel conversational recommendation scenario named Multi-Subsession Multi-round Conversational Recommendation (MSMCR), where user would still resort to CRS after several subsessions and might preserve vague interests, and system would proactively ask attributes to activate user interests in the current subsession. To fill the gap in this new CRS scenario, we devise a novel framework called Multi-Subsession Conversational Recommender with Activation Attributes (MSCAA). Specifically, we first develop a context-aware recommendation module, comprehensively modeling user interests from historical interactions, previous subsessions, and feedback in the current subsession. Furthermore, an attribute selection policy module is proposed to learn a flexible strategy for asking appropriate attributes to elicit user interests. Finally, we design a conversation policy module to manage the above two modules to decide actions between asking and recommending. Extensive experiments on four datasets verify the effectiveness of our MSCAA framework for the MSMCR setting.
Motif-Based Prompt Learning for Universal Cross-Domain Recommendation
Authors: Bowen Hao, Chaoqun Yang, Lei Guo, Junliang Yu, Hongzhi Yin
Date: 2023-10-20 00:00:00
Tasks: General Knowledge, Multi-Task Learning, Prompt Learning

Cross-Domain Recommendation (CDR) stands as a pivotal technology addressing issues of data sparsity and cold start by transferring general knowledge from the source to the target domain. However, existing CDR models suffer limitations in adaptability across various scenarios due to their inherent complexity. To tackle this challenge, recent advancements introduce universal CDR models that leverage shared embeddings to capture general knowledge across domains and transfer it through "Multi-task Learning" or "Pre-train, Fine-tune" paradigms. However, these models often overlook the broader structural topology that spans domains and fail to align training objectives, potentially leading to negative transfer. To address these issues, we propose a motif-based prompt learning framework, MOP, which introduces motif-based shared embeddings to encapsulate generalized domain knowledge, catering to both intra-domain and inter-domain CDR tasks. Specifically, we devise three typical motifs: butterfly, triangle, and random walk, and encode them through a Motif-based Encoder to obtain motif-based shared embeddings. Moreover, we train MOP under the "Pre-training \& Prompt Tuning" paradigm. By unifying pre-training and recommendation tasks as a common motif-based similarity learning task and integrating adaptable prompt parameters to guide the model in downstream recommendation tasks, MOP excels in transferring domain knowledge effectively. Experimental results on four distinct CDR tasks demonstrate the effectiveness of MOP than the state-of-the-art models.
VR PreM+ : An Immersive Pre-learning Branching Visualization System for Museum Tours
Authors: Ze Gao, Xiang Li, Changkun Liu, Xian Wang, Anqi Wang, Liang Yang, Yuyang Wang, Pan Hui, Tristan Braud
Date: 2023-10-20 00:00:00
Tasks: Information Retrieval, Retrieval

We present VR PreM+, an innovative VR system designed to enhance web exploration beyond traditional computer screens. Unlike static 2D displays, VR PreM+ leverages 3D environments to create an immersive pre-learning experience. Using keyword-based information retrieval allows users to manage and connect various content sources in a dynamic 3D space, improving communication and data comparison. We conducted preliminary and user studies that demonstrated efficient information retrieval, increased user engagement, and a greater sense of presence. These findings yielded three design guidelines for future VR information systems: display, interaction, and user-centric design. VR PreM+ bridges the gap between traditional web browsing and immersive VR, offering an interactive and comprehensive approach to information acquisition. It holds promise for research, education, and beyond.
A Data-Centric Multi-Objective Learning Framework for Responsible Recommendation Systems
Authors: Xu Huang, Jianxun Lian, Hao Wang, Defu Lian, Xing Xie
Date: 2023-10-20 00:00:00
Tasks: Fairness, Recommendation Systems

Recommendation systems effectively guide users in locating their desired information within extensive content repositories. Generally, a recommendation model is optimized to enhance accuracy metrics from a user utility standpoint, such as click-through rate or matching relevance. However, a responsible industrial recommendation system must address not only user utility (responsibility to users) but also other objectives, including increasing platform revenue (responsibility to platforms), ensuring fairness (responsibility to content creators), and maintaining unbiasedness (responsibility to long-term healthy development). Multi-objective learning is a potent approach for achieving responsible recommendation systems. Nevertheless, current methods encounter two challenges: difficulty in scaling to heterogeneous objectives within a unified framework, and inadequate controllability over objective priority during optimization, leading to uncontrollable solutions. In this paper, we present a data-centric optimization framework, MoRec, which unifies the learning of diverse objectives. MoRec is a tri-level framework: the outer level manages the balance between different objectives, utilizing a proportional-integral-derivative (PID)-based controller to ensure a preset regularization on the primary objective. The middle level transforms objective-aware optimization into data sampling weights using sign gradients. The inner level employs a standard optimizer to update model parameters with the sampled data. Consequently, MoRec can flexibly support various objectives while maintaining the original model intact. Comprehensive experiments on two public datasets and one industrial dataset showcase the effectiveness, controllability, flexibility, and Pareto efficiency of MoRec, making it highly suitable for real-world implementation.
Software Metadata Classification based on Generative Artificial Intelligence
Authors: Seetharam Killivalavan, Durairaj Thenmozhi
Date: 2023-10-14 00:00:00
Tasks: Classification, Language Modeling, Language Modelling, Large Language Model

This paper presents a novel approach to enhance the performance of binary code comment quality classification models through the application of Generative Artificial Intelligence (AI). By leveraging the OpenAI API, a dataset comprising 1239 newly generated code-comment pairs, extracted from various GitHub repositories and open-source projects, has been labelled as "Useful" or "Not Useful", and integrated into the existing corpus of 9048 pairs in the C programming language. Employing a cutting-edge Large Language Model Architecture, the generated dataset demonstrates notable improvements in model accuracy. Specifically, when incorporated into the Support Vector Machine (SVM) model, a 6% increase in precision is observed, rising from 0.79 to 0.85. Additionally, the Artificial Neural Network (ANN) model exhibits a 1.5% increase in recall, climbing from 0.731 to 0.746. This paper sheds light on the potential of Generative AI in augmenting code comment quality classification models. The results affirm the effectiveness of this methodology, indicating its applicability in broader contexts within software development and quality assurance domains. The findings underscore the significance of integrating generative techniques to advance the accuracy and efficacy of machine learning models in practical software engineering scenarios.
Conversational Factor Information Retrieval Model (ConFIRM)
Authors: Stephen Choi, William Gazeley, Siu Ho Wong, TingTing Li
Date: 2023-10-06 00:00:00
Tasks: GPU, Information Retrieval, intent-classification, Intent Classification, model, parameter-efficient fine-tuning, Retrieval

This paper introduces the Conversational Factor Information Retrieval Method (ConFIRM), a novel approach to fine-tuning large language models (LLMs) for domain-specific retrieval tasks. ConFIRM leverages the Five-Factor Model of personality to generate synthetic datasets that accurately reflect target population characteristics, addressing data scarcity in specialized domains. We demonstrate ConFIRM's effectiveness through a case study in the finance sector, fine-tuning a Llama-2-7b model using personality-aligned data from the PolyU-Asklora Fintech Adoption Index. The resulting model achieved 91% accuracy in classifying financial queries, with an average inference time of 0.61 seconds on an NVIDIA A100 GPU. ConFIRM shows promise for creating more accurate and personalized AI-driven information retrieval systems across various domains, potentially mitigating issues of hallucinations and outdated information in LLMs deployed
Document-Level Relation Extraction with Relation Correlation Enhancement
Authors: Yusheng Huang, Zhouhan Lin
Date: 2023-10-06 00:00:00
Tasks: Document-level Relation Extraction, Graph Attention, Relation, Relation Extraction

Document-level relation extraction (DocRE) is a task that focuses on identifying relations between entities within a document. However, existing DocRE models often overlook the correlation between relations and lack a quantitative analysis of relation correlations. To address this limitation and effectively capture relation correlations in DocRE, we propose a relation graph method, which aims to explicitly exploit the interdependency among relations. Firstly, we construct a relation graph that models relation correlations using statistical co-occurrence information derived from prior relation knowledge. Secondly, we employ a re-weighting scheme to create an effective relation correlation matrix to guide the propagation of relation information. Furthermore, we leverage graph attention networks to aggregate relation embeddings. Importantly, our method can be seamlessly integrated as a plug-and-play module into existing models. Experimental results demonstrate that our approach can enhance the performance of multi-relation extraction, highlighting the effectiveness of considering relation correlations in DocRE.
SplitGNN: Spectral Graph Neural Network for Fraud Detection against Heterophily
Authors: Bin Wu, Xinyu Yao, Boyan Zhang, Kuo-Ming Chao, Yinsheng Li
Date: 2023-10-21 00:00:00
Proceeding: CIKM 2023 10
Tasks: Fraud Detection, Graph Neural Network, Node Classification

Fraudsters in the real world frequently add more legitimate links while concealing their direct ones with other fraudsters, leading to heterophily in fraud graphs, which is a problem that most GNN-based techniques are not built to solve. Several works have been proposed to tackle the issue from the spatial domain. However, researches on addressing the heterophily problem in the spectral domain are still limited due to a lack of understanding of spectral energy distribution in graphs with heterophily. In this paper, we analyze the spectral distribution with different heterophily degrees and observe that the heterophily of fraud nodes leads to the spectral energy moving from low-frequency to high-frequency. Further, we verify that splitting graphs using heterophilic and homophilic edges can obtain more significant expressions of signals in different frequency bands. The observation drives us to propose the spectral graph neural network, SplitGNN, to capture signals for fraud detection against heterophily. SplitGNN uses an edge classifier to split the original graph and adopts flexible band-pass graph filters to learn representations. Extensive experiments on real-world datasets demonstrate the effectiveness of our proposed method. The code and data are available at https://github.com/Split-GNN/SplitGNN.
Fairness and Effectiveness in Federated Learning on Non-independent and Identically Distributed Data
Authors: Wentao Pan；Hui Zhou
Date: 2023-01-01 00:00:00
Proceeding: EEE 3rd International Conference on Computer Communication and Artificial Intelligence 2023 1
Tasks: Fairness, Federated Learning

Federated learning is a distributed machine learning method that protects privacy by allowing participants to train models locally rather than uploading data. However, federated learning has a significant barrier because of the nonindependent and identically distributed (Non-IID) nature of each participant's local data. FedFE, a novel fair and effective federated optimization algorithm, is presented in this paper. FedFE introduces momentum gradient descent in the federated training process and proposes a fair weighting strategy based on participant performance in training to eliminate the unfairness caused by the preference for some participants in the federated aggregation process. Experiments on a large number of Non-IID datasets have demonstrated that the proposed algorithm improves on existing baseline algorithms in terms of fairness, effectiveness, and convergence speed.