pwc-archive/papers-with-abstracts huggingface dataset:
Stance Detection: A Practical Guide to Classifying Political Beliefs in Text
Authors: Michael Burnham
Date: 2023-05-02 00:00:00
Tasks: In-Context Learning, Natural Language Inference, Sentiment Analysis, Stance Detection, Zero-Shot Learning, Zero-Shot Stance Detection

Stance detection is identifying expressed beliefs in a document. While researchers widely use sentiment analysis for this, recent research demonstrates that sentiment and stance are distinct. This paper advances text analysis methods by precisely defining stance detection and presenting three distinct approaches: supervised classification, natural language inference, and in-context learning with generative language models. I discuss how document context and trade-offs between resources and workload should inform your methods. For all three approaches I provide guidance on application and validation techniques, as well as coding tutorials for implementation. Finally, I demonstrate how newer classification approaches can replicate supervised classifiers.
Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner
Authors: Zhengxiang Shi, Aldo Lipani
Date: 2023-05-02 00:00:00
Tasks: Sentence, Unsupervised Pre-training

Language models (LMs) trained on vast quantities of unlabelled data have greatly advanced the field of natural language processing (NLP). In this study, we re-visit the widely accepted notion in NLP that continued pre-training LMs on task-related texts improves the performance of fine-tuning (FT) in downstream tasks. Through experiments on eight single-sentence tasks and eight sentence-pair tasks in both semi-supervised and fully-supervised settings, we find that conventional continued pre-training does not consistently provide benefits and can even be detrimental for sentence-pair tasks or when prompt-based FT is used. To tackle these issues, we propose Prompt-based Continued Pre-training (PCP), which combines the idea of instruction tuning with conventional continued pre-training. Our approach aims to improve the performance of prompt-based FT by presenting both task-related texts and prompt templates to LMs through unsupervised pre-training objectives before fine-tuning for the target task. Our empirical evaluations on 21 benchmarks demonstrate that the PCP consistently improves the performance of state-of-the-art prompt-based FT approaches (up to 20.1% absolute) in both semi-supervised and fully-supervised settings, even with only hundreds of unlabelled examples. Additionally, prompt-based FT with the PCP outperforms state-of-the-art semi-supervised approaches with greater simplicity, eliminating the need for an iterative process and extra data augmentation. Our further analysis explores the performance lower bound of the PCP and reveals that the advantages of PCP persist across different sizes of models and datasets.
From Stars to Insights: Exploration and Implementation of Unified Sentiment Analysis with Distant Supervision
Authors: Wenchang Li, John P. Lalor, Yixing Chen, Vamsi K. Kanuri
Date: 2023-05-02 00:00:00
Tasks: Aspect Category Detection, Aspect Category Sentiment Analysis, Sentiment Analysis

Sentiment analysis is integral to understanding the voice of the customer and informing businesses' strategic decisions. Conventional sentiment analysis involves three separate tasks: aspect-category detection (ACD), aspect-category sentiment analysis (ACSA), and rating prediction (RP). However, independently tackling these tasks can overlook their interdependencies and often requires expensive, fine-grained annotations. This paper introduces Unified Sentiment Analysis (Uni-SA), a novel learning paradigm that unifies ACD, ACSA, and RP into a coherent framework. To achieve this, we propose the Distantly Supervised Pyramid Network (DSPN), which employs a pyramid structure to capture sentiment at word, aspect, and document levels in a hierarchical manner. Evaluations on multi-aspect review datasets in English and Chinese show that DSPN, using only star rating labels for supervision, demonstrates significant efficiency advantages while performing comparably well to a variety of benchmark models. Additionally, DSPN's pyramid structure enables the interpretability of its outputs. Our findings validate DSPN's effectiveness and efficiency, establishing a robust, resource-efficient, unified framework for sentiment analysis.
Shotgun crystal structure prediction using machine-learned formation energies
Authors: Chang Liu, Hiromasa Tamaki, Tomoyasu Yokoyama, Kensuke Wakasugi, Satoshi Yotsuhashi, Minoru Kusaba, Artem R. Oganov, Ryo Yoshida
Date: 2023-05-03 00:00:00
Tasks: Prediction, Transfer Learning

Stable or metastable crystal structures of assembled atoms can be predicted by finding the global or local minima of the energy surface within a broad space of atomic configurations. Generally, this requires repeated first-principles energy calculations, which is often impractical for large crystalline systems. Here, we present significant progress toward solving the crystal structure prediction problem: we performed noniterative, single-shot screening using a large library of virtually created crystal structures with a machine-learning energy predictor. This shotgun method (ShotgunCSP) has two key technical components: transfer learning for accurate energy prediction of pre-relaxed crystalline states, and two generative models based on element substitution and symmetry-restricted structure generation to produce promising and diverse crystal structures. First-principles calculations were performed only to generate the training samples and to refine a few selected pre-relaxed crystal structures. The ShotunCSP method is computationally less intensive than conventional methods and exhibits exceptional prediction accuracy, reaching 93.3% in benchmark tests with 90 different crystal structures.
Generating Synthetic Documents for Cross-Encoder Re-Rankers: A Comparative Study of ChatGPT and Human Experts
Authors: Arian Askari, Mohammad Aliannejadi, Evangelos Kanoulas, Suzan Verberne
Date: 2023-05-03 00:00:00
Tasks: Re-Ranking, Retrieval

We investigate the usefulness of generative Large Language Models (LLMs) in generating training data for cross-encoder re-rankers in a novel direction: generating synthetic documents instead of synthetic queries. We introduce a new dataset, ChatGPT-RetrievalQA, and compare the effectiveness of models fine-tuned on LLM-generated and human-generated data. Data generated with generative LLMs can be used to augment training data, especially in domains with smaller amounts of labeled data. We build ChatGPT-RetrievalQA based on an existing dataset, human ChatGPT Comparison Corpus (HC3), consisting of public question collections with human responses and answers from ChatGPT. We fine-tune a range of cross-encoder re-rankers on either human-generated or ChatGPT-generated data. Our evaluation on MS MARCO DEV, TREC DL'19, and TREC DL'20 demonstrates that cross-encoder re-ranking models trained on ChatGPT responses are statistically significantly more effective zero-shot re-rankers than those trained on human responses. In a supervised setting, the human-trained re-rankers outperform the LLM-trained re-rankers. Our novel findings suggest that generative LLMs have high potential in generating training data for neural retrieval models. Further work is needed to determine the effect of factually wrong information in the generated responses and test our findings' generalizability with open-source LLMs. We release our data, code, and cross-encoders checkpoints for future work.
Uncovering ChatGPT's Capabilities in Recommender Systems
Authors: Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongxiang Sun, Xiao Zhang, Jun Xu
Date: 2023-05-03 00:00:00
Tasks: Explainable Recommendation, Information Retrieval, Recommendation Systems, Retrieval

The debut of ChatGPT has recently attracted the attention of the natural language processing (NLP) community and beyond. Existing studies have demonstrated that ChatGPT shows significant improvement in a range of downstream NLP tasks, but the capabilities and limitations of ChatGPT in terms of recommendations remain unclear. In this study, we aim to conduct an empirical analysis of ChatGPT's recommendation ability from an Information Retrieval (IR) perspective, including point-wise, pair-wise, and list-wise ranking. To achieve this goal, we re-formulate the above three recommendation policies into a domain-specific prompt format. Through extensive experiments on four datasets from different domains, we demonstrate that ChatGPT outperforms other large language models across all three ranking policies. Based on the analysis of unit cost improvements, we identify that ChatGPT with list-wise ranking achieves the best trade-off between cost and performance compared to point-wise and pair-wise ranking. Moreover, ChatGPT shows the potential for mitigating the cold start problem and explainable recommendation. To facilitate further explorations in this area, the full code and detailed original results are open-sourced at https://github.com/rainym00d/LLM4RS.
Understanding Differential Search Index for Text Retrieval
Authors: Xiaoyang Chen, Yanjiang Liu, Ben He, Le Sun, Yingfei Sun
Date: 2023-05-03 00:00:00
Tasks: Information Retrieval, Retrieval, Text Retrieval

The Differentiable Search Index (DSI) is a novel information retrieval (IR) framework that utilizes a differentiable function to generate a sorted list of document identifiers in response to a given query. However, due to the black-box nature of the end-to-end neural architecture, it remains to be understood to what extent DSI possesses the basic indexing and retrieval abilities. To mitigate this gap, in this study, we define and examine three important abilities that a functioning IR framework should possess, namely, exclusivity, completeness, and relevance ordering. Our analytical experimentation shows that while DSI demonstrates proficiency in memorizing the unidirectional mapping from pseudo queries to document identifiers, it falls short in distinguishing relevant documents from random ones, thereby negatively impacting its retrieval effectiveness. To address this issue, we propose a multi-task distillation approach to enhance the retrieval quality without altering the structure of the model and successfully endow it with improved indexing abilities. Through experiments conducted on various datasets, we demonstrate that our proposed method outperforms previous DSI baselines.
Denoising Multi-modal Sequential Recommenders with Contrastive Learning
Authors: Dong Yao, Shengyu Zhang, Zhou Zhao, Jieming Zhu, Wenqiao Zhang, Rui Zhang, Xiaofei He, Fei Wu
Date: 2023-05-03 00:00:00
Tasks: Contrastive Learning, Denoising, point of interests, Recommendation Systems

There is a rapidly-growing research interest in engaging users with multi-modal data for accurate user modeling on recommender systems. Existing multimedia recommenders have achieved substantial improvements by incorporating various modalities and devising delicate modules. However, when users decide to interact with items, most of them do not fully read the content of all modalities. We refer to modalities that directly cause users' behaviors as point-of-interests, which are important aspects to capture users' interests. In contrast, modalities that do not cause users' behaviors are potential noises and might mislead the learning of a recommendation model. Not surprisingly, little research in the literature has been devoted to denoising such potential noises due to the inaccessibility of users' explicit feedback on their point-of-interests. To bridge the gap, we propose a weakly-supervised framework based on contrastive learning for denoising multi-modal recommenders (dubbed Demure). In a weakly-supervised manner, Demure circumvents the requirement of users' explicit feedback and identifies the noises by analyzing the modalities of all interacted items from a given user.
Towards Imperceptible Document Manipulations against Neural Ranking Models
Authors: Xuanang Chen, Ben He, Zheng Ye, Le Sun, Yingfei Sun
Date: 2023-05-03 00:00:00
Tasks: Adversarial Text, Language Modeling, Language Modelling, Text Generation

Adversarial attacks have gained traction in order to identify potential vulnerabilities in neural ranking models (NRMs), but current attack methods often introduce grammatical errors, nonsensical expressions, or incoherent text fragments, which can be easily detected. Additionally, current methods rely heavily on the use of a well-imitated surrogate NRM to guarantee the attack effect, which makes them difficult to use in practice. To address these issues, we propose a framework called Imperceptible DocumEnt Manipulation (IDEM) to produce adversarial documents that are less noticeable to both algorithms and humans. IDEM instructs a well-established generative language model, such as BART, to generate connection sentences without introducing easy-to-detect errors, and employs a separate position-wise merging strategy to balance relevance and coherence of the perturbed text. Experimental results on the popular MS MARCO benchmark demonstrate that IDEM can outperform strong baselines while preserving fluency and correctness of the target documents as evidenced by automatic and human evaluations. Furthermore, the separation of adversarial text generation from the surrogate NRM makes IDEM more robust and less affected by the quality of the surrogate NRM.
H2CGL: Modeling Dynamics of Citation Network for Impact Prediction
Authors: Guoxiu He, Zhikai Xue, Zhuoren Jiang, Yangyang Kang, Star Zhao, Wei Lu
Date: 2023-04-16 00:00:00
Tasks: Contrastive Learning, Graph Learning, Graph Neural Network

The potential impact of a paper is often quantified by how many citations it will receive. However, most commonly used models may underestimate the influence of newly published papers over time, and fail to encapsulate this dynamics of citation network into the graph. In this study, we construct hierarchical and heterogeneous graphs for target papers with an annual perspective. The constructed graphs can record the annual dynamics of target papers' scientific context information. Then, a novel graph neural network, Hierarchical and Heterogeneous Contrastive Graph Learning Model (H2CGL), is proposed to incorporate heterogeneity and dynamics of the citation network. H2CGL separately aggregates the heterogeneous information for each year and prioritizes the highly-cited papers and relationships among references, citations, and the target paper. It then employs a weighted GIN to capture dynamics between heterogeneous subgraphs over years. Moreover, it leverages contrastive learning to make the graph representations more sensitive to potential citations. Particularly, co-cited or co-citing papers of the target paper with large citation gap are taken as hard negative samples, while randomly dropping low-cited papers could generate positive samples. Extensive experimental results on two scholarly datasets demonstrate that the proposed H2CGL significantly outperforms a series of baseline approaches for both previously and freshly published papers. Additional analyses highlight the significance of the proposed modules. Our codes and settings have been released on Github (https://github.com/ECNU-Text-Computing/H2CGL)
Deep Reinforcement Learning-Based Dynamic Resource Management for Mobile Edge Computing in Industrial Internet of Things
Authors: Ying Chen, Zhiyong Liu, Yongchao Zhang, Yuan Wu, Xin Chen, Lian Zhao
Date: 2020-10-06 00:00:00
Proceeding: IEEE 2020 10
Tasks: Deep Reinforcement Learning, Edge-computing, Management

Nowadays, driven by the rapid development ofsmart mobile equipments and 5G network technologies, theapplication scenarios of Internet of Things (IoT) technologyare becoming increasingly widespread. The integration ofIoT and industrial manufacturing systems forms the industrial IoT (IIoT). Because of the limitation of resources,such as the computation unit and battery capacity in theIIoT equipments (IIEs), computation-intensive tasks need tobe executed in the mobile edge computing (MEC) server.However, the dynamics and continuity of task generationlead to a severe challenge to the management of limitedresources in IIoT. In this article, we investigate the dynamicresource management problem of joint power control andcomputing resource allocation for MEC in IIoT. In order to minimize the long-term average delay of the tasks, theoriginal problem is transformed into a Markov decision process (MDP). Considering the dynamics and continuity oftask generation, we propose a deep reinforcement learningbased dynamic resource management (DDRM) algorithm to
solve the formulated MDP problem. Our DDRM algorithmexploits the deep deterministic policy gradient and can deal
with the high-dimensional continuity of the action and state spaces. Extensive simulation results demonstrate that the
DDRM can reduce the long-term average delay of the taskseffectively.
Resource allocation algorithm for MEC based on Deep Reinforcement Learning
Authors: Yijie Wang, Xin Chen, Ying Chen, Shougang Du
Date: 2022-01-20 00:00:00
Proceeding: IEEE 2022 1
Tasks: Deep Reinforcement Learning, Edge-computing, reinforcement-learning, Reinforcement Learning

In recent years, driven by the commercialization of the 6th Generation Communication Technology (6G), an increasing number of 6G devices connected to mobile networks produces computation-intensive tasks such as ultra-high-resolution video streaming, inter-active visual reality (VR) gaming, augmented reality (AR). However, the computing capacity and the capacity of battery of the 6G devices are limited. The technology of computation offloading would offload the tasks from the IoT devices to the edge network in the scenario of mobile edge computing (MEC). Not only can solve the shortage of mobile user device in energy effciency, but also deal with the tasks in low latency. IoT devices can offload computing tasks or execute them locally to finish the work. In order to find the optimal allocation rate of local computing tasks and offloading tasks, a resource allocation policy gradient (RAPG) based DDPG is considered. Finally we analyze the performance of RAPG by contrasts with different resource allocation algorithms. Numerial simulation results showed that the RAPG can achieve the best allocate rate between the BS and local, also can reduce the overall system delay of task combination with minimum energy consumption.
D3PG: Dirichlet DDPG for Task Partitioning and Offloading With Constrained Hybrid Action Space in Mobile-Edge Computing
Authors: Laha Ale, Scott A. King, Ning Zhang, Abdul Rahman Sattar, Janahan Skandaraniyam
Date: 2022-04-14 00:00:00
Proceeding: IEEE 2022 4
Tasks: Deep Reinforcement Learning, Edge-computing, General Reinforcement Learning, Multiobjective Optimization, reinforcement-learning, Reinforcement Learning

Mobile-edge computing (MEC) has been regarded as a promising paradigm to reduce service latency for data processing in the Internet of Things (IoT) by provisioning computing resources at the network edges. In this work, we jointly optimize the task partitioning and computational power allocation for computation offloading in a dynamic environment with multiple IoT devices and multiple edge servers. We formulate the problem as a Markov decision process with constrained hybrid action space, which cannot be well handled by existing deep reinforcement learning (DRL) algorithms. Therefore, we develop a novel DRL called Dirichlet deep deterministic policy gradient (D3PG), which is built on deep deterministic policy gradient (DDPG) to solve the problem. The developed model can learn to solve multiobjective optimization, including maximizing the number of tasks processed before deadlines and minimizing the energy cost and service latency. More importantly, D3PG can effectively deal with a constrained distribution-continuous hybrid action spaces, where the distribution variables are for the task partitioning and offloading, while the continuous variables are for computational frequency control. Moreover, the D3PG can address many similar issues in MEC and general reinforcement learning problems. Extensive simulation results show that the proposed D3PG outperforms the state-of-the-art methods.
Multi-Channel Vision Transformer for Epileptic Seizure Prediction
Authors: Ramy Hussein, Soojin Lee, Rabab Ward
Date: 2022-06-29 00:00:00
Proceeding: Biomedicines 2022 6
Tasks: EEG, Prediction, Seizure prediction, Time Series

Epilepsy is a neurological disorder that causes recurrent seizures and sometimes loss of awareness. Around 30% of epileptic patients continue to have seizures despite taking anti-seizure medication. The ability to predict the future occurrence of seizures would enable the patients to take precautions against probable injuries and administer timely treatment to abort or control impending seizures. In this study, we introduce a Transformer-based approach called Multi-channel Vision Transformer (MViT) for automated and simultaneous learning of the spatio-temporal-spectral features in multi-channel EEG data. Continuous wavelet transform, a simple yet efficient pre-processing approach, is first used for turning the time-series EEG signals into image-like time-frequency representations named Scalograms. Each scalogram is split into a sequence of fixed-size non-overlapping patches, which are then fed as inputs to the MViT for EEG classification. Extensive experiments on three benchmark EEG datasets demonstrate the superiority of the proposed MViT algorithm over the state-of-the-art seizure prediction methods, achieving an average prediction sensitivity of 99.80% for surface EEG and 90.28–91.15% for invasive EEG data.
RCP-RF: A Comprehensive Road-car-pedestrian Risk Management Framework based on Driving Risk Potential Field
Authors: Shuhang Tan, Zhiling Wang, Yan Zhong
Date: 2023-05-04 00:00:00
Tasks: Management

Recent years have witnessed the proliferation of traffic accidents, which led wide researches on Automated Vehicle (AV) technologies to reduce vehicle accidents, especially on risk assessment framework of AV technologies. However, existing time-based frameworks can not handle complex traffic scenarios and ignore the motion tendency influence of each moving objects on the risk distribution, leading to performance degradation. To address this problem, we novelly propose a comprehensive driving risk management framework named RCP-RF based on potential field theory under Connected and Automated Vehicles (CAV) environment, where the pedestrian risk metric are combined into a unified road-vehicle driving risk management framework. Different from existing algorithms, the motion tendency between ego and obstacle cars and the pedestrian factor are legitimately considered in the proposed framework, which can improve the performance of the driving risk model. Moreover, it requires only O(N 2) of time complexity in the proposed method. Empirical studies validate the superiority of our proposed framework against state-of-the-art methods on real-world dataset NGSIM and real AV platform.