pwc-archive/papers-with-abstracts huggingface dataset:
COPR -- Efficient, large-scale log storage and retrieval
Authors: Julian Reichinger, Thomas Krismayer, Jan Rellermeyer
Date: 2024-02-28 00:00:00
Tasks: Retrieval

Modern, large scale monitoring systems have to process and store vast amounts of log data in near real-time. At query time the systems have to find relevant logs based on the content of the log message using support structures that can scale to these amounts of data while still being efficient to use. We present our novel Compressed Probabilistic Retrieval algorithm (COPR), capable of answering Multi-Set Multi-Membership-Queries, that can be used as an alternative to existing indexing structures for streamed log data. In our experiments, COPR required up to 93% less storage space than the tested state-of-the-art inverted index and had up to four orders of magnitude less false-positives than the tested state-of-the-art membership sketch. Additionally, COPR achieved up to 250 times higher query throughput than the tested inverted index and up to 240 times higher query throughput than the tested membership sketch.
Detecting Anti-vaccine Content on Twitter using Multiple Message-Based Network Representations
Authors: James R. Ashford
Date: 2024-02-28 00:00:00
Tasks: Binary Classification

Social media platforms such as Twitter have a fundamental role in facilitating the spread and discussion of ideas online through the concept of retweeting and replying. However, these features also contribute to the spread of mis/disinformation during the vaccine rollout of the COVID-19 pandemic. Using COVID-19 vaccines as a case study, we analyse multiple social network representation derived from three message-based interactions on Twitter (quote retweets, mentions and replies) based upon a set of known anti-vax hashtags and keywords. Each network represents a certain hashtag or keyword which were labelled as "controversial" and "non-controversial" according to a small group of participants. For each network, we extract a combination of global and local network-based metrics which are used as feature vectors for binary classification. Our results suggest that it is possible to detect controversial from non-controversial terms with high accuracy using simple network-based metrics. Furthermore, these results demonstrate the potential of network representations as language-agnostic models for detecting mis/disinformation at scale, irrespective of content and across multiple social media platforms.
Sequence-level Semantic Representation Fusion for Recommender Systems
Authors: Lanling Xu, Zhen Tian, Bingqian Li, Junjie Zhang, Jinpeng Wang, Mingchen Cai, Wayne Xin Zhao
Date: 2024-02-28 00:00:00
Tasks: Mixture-of-Experts, Recommendation Systems, Sequential Recommendation

With the rapid development of recommender systems, there is increasing side information that can be employed to improve the recommendation performance. Specially, we focus on the utilization of the associated \emph{textual data} of items (eg product title) and study how text features can be effectively fused with ID features in sequential recommendation. However, there exists distinct data characteristics for the two kinds of item features, making a direct fusion method (eg adding text and ID embeddings as item representation) become less effective. To address this issue, we propose a novel {\ul \emph{Te}}xt-I{\ul \emph{D}} semantic fusion approach for sequential {\ul \emph{Rec}}ommendation, namely \textbf{\our}. The core idea of our approach is to conduct a sequence-level semantic fusion approach by better integrating global contexts. The key strategy lies in that we transform the text embeddings and ID embeddings by Fourier Transform from \emph{time domain} to \emph{frequency domain}. In the frequency domain, the global sequential characteristics of the original sequences are inherently aggregated into the transformed representations, so that we can employ simple multiplicative operations to effectively fuse the two kinds of item features. Our fusion approach can be proved to have the same effects of contextual convolution, so as to achieving sequence-level semantic fusion. In order to further improve the fusion performance, we propose to enhance the discriminability of the text embeddings from the text encoder, by adaptively injecting positional information via a mixture-of-experts~(MoE) modulation method. Our implementation is available at this repository: \textcolor{magenta}{\url{https://github.com/RUCAIBox/TedRec}}.
[RE] Modeling Personalized Item Frequency Information for Next-basket Recommendation
Authors: Sławomir Garcarz, Avik Pal, Pim Praat
Date: 2024-02-27 00:00:00
Tasks: Fairness, Next-basket recommendation

This paper focuses on reproducing and extending the results of the paper: "Modeling Personalized Item Frequency Information for Next-basket Recommendation" which introduced the TIFU-KNN model and proposed to utilize Personalized Item Frequency (PIF) for Next Basket Recommendation (NBR). We utilized publicly available grocery shopping datasets used in the original paper and incorporated additional datasets to assess the generalizability of the findings. We evaluated the performance of the models using metrics such as Recall@K, NDCG@K, personalized-hit ratio (PHR), and Mean Reciprocal Rank (MRR). Furthermore, we conducted a thorough examination of fairness by considering user characteristics such as average basket size, item popularity, and novelty. Lastly, we introduced novel $\beta$-VAE architecture to model NBR. The experimental results confirmed that the reproduced model, TIFU-KNN, outperforms the baseline model, Personal Top Frequency, on various datasets and metrics. The findings also highlight the challenges posed by smaller basket sizes in some datasets and suggest avenues for future research to improve NBR performance.
A Language Model based Framework for New Concept Placement in Ontologies
Authors: Hang Dong, Jiaoyan Chen, Yuan He, Yongsheng Gao, Ian Horrocks
Date: 2024-02-27 00:00:00
Tasks: Contrastive Learning, Entity Linking, Language Modeling, Language Modelling

We investigate the task of inserting new concepts extracted from texts into an ontology using language models. We explore an approach with three steps: edge search which is to find a set of candidate locations to insert (i.e., subsumptions between concepts), edge formation and enrichment which leverages the ontological structure to produce and enhance the edge candidates, and edge selection which eventually locates the edge to be placed into. In all steps, we propose to leverage neural methods, where we apply embedding-based methods and contrastive learning with Pre-trained Language Models (PLMs) such as BERT for edge search, and adapt a BERT fine-tuning-based multi-label Edge-Cross-encoder, and Large Language Models (LLMs) such as GPT series, FLAN-T5, and Llama 2, for edge selection. We evaluate the methods on recent datasets created using the SNOMED CT ontology and the MedMentions entity linking benchmark. The best settings in our framework use fine-tuned PLM for search and a multi-label Cross-encoder for selection. Zero-shot prompting of LLMs is still not adequate for the task, and we propose explainable instruction tuning of LLMs for improved performance. Our study shows the advantages of PLMs and highlights the encouraging performance of LLMs that motivates future studies.
JMLR: Joint Medical LLM and Retrieval Training for Enhancing Reasoning and Professional Question Answering Capability
Authors: Junda Wang, Zhichao Yang, Zonghai Yao, Hong Yu
Date: 2024-02-27 00:00:00
Tasks: GPU, Information Retrieval, Medical Question Answering, Question Answering, RAG, Retrieval, Retrieval-augmented Generation

Large Language Models (LLMs) have demonstrated a remarkable potential in medical knowledge acquisition and question-answering. However, LLMs can potentially hallucinate and yield factually incorrect outcomes, even with domain-specific pretraining. Previously, retrieval augmented generation (RAG) has limited success in addressing hallucinations. Unlike previous methods in RAG where the retrieval model was trained separately from the LLM, we introduce JMLR (for Jointly trains LLM and information Retrieval) during the fine-tuning phase. The synchronized training mechanism enhances JMLR's ability to retrieve clinical guidelines and leverage medical knowledge to reason and answer questions and reduces the demand for computational resources. We evaluated JMLR on the important medical question-answering application. Our experimental results demonstrate that JMLR-13B (70.5%) outperforms a previous state-of-the-art open-source model using conventional pre-training and fine-tuning Meditron-70B (68.9%) and Llama2-13B with RAG (67.7%) on a medical question-answering dataset. Comprehensive evaluations reveal JMLR-13B enhances reasoning quality and reduces hallucinations better than Claude3-Opus. Additionally, JMLR-13B (148 GPU hours) also trains much faster than Meditron-70B (42630 GPU hours). Through this work, we provide a new and efficient knowledge enhancement method for healthcare, demonstrating the potential of integrating retrieval and LLM training for medical question-answering systems.
BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset
Authors: Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Ce Bian, Chi Zhang, Ruiyang Sun, Yizhou Wang, Yaodong Yang
Date: 2023-07-10 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Question Answering, Safety Alignment

In this paper, we introduce the BeaverTails dataset, aimed at fostering research on safety alignment in large language models (LLMs). This dataset uniquely separates annotations of helpfulness and harmlessness for question-answering pairs, thus offering distinct perspectives on these crucial attributes. In total, we have gathered safety meta-labels for 333,963 question-answer (QA) pairs and 361,903 pairs of expert comparison data for both the helpfulness and harmlessness metrics. We further showcase applications of BeaverTails in content moderation and reinforcement learning with human feedback (RLHF), emphasizing its potential for practical safety measures in LLMs. We believe this dataset provides vital resources for the community, contributing towards the safe development and deployment of LLMs. Our project page is available at the following URL: https://sites.google.com/view/pku-beavertails.
Faithful Abstractive Summarization via Fact-aware Consistency-constrained Transformer
Authors: Yuanjie Lyu, Chen Zhu, Tong Xu, Zikai Yin, Enhong Chen
Date: 2022-10-01 00:00:00
Proceeding: CIKM 2022 10
Tasks: Abstractive Text Summarization, Text Generation

Abstractive summarization is a classic task in Natural Language Generation (NLG), which aims to produce a concise summary of the original document. Recently, great efforts have been made on sequence-to-sequence neural networks to generate abstractive summaries with a high level of fluency. However, prior arts mainly focus on the optimization of token-level likelihood, while the rich semantic information in documents has been largely ignored. In this way, the summarization results could be vulnerable to hallucinations, i.e., the semantic-level inconsistency between a summary and corresponding original document. To deal with this challenge, in this paper, we propose a novel fact-aware abstractive summarization model, named Entity-Relation Pointer Generator Network (ERPGN). Specially, we attempt to formalize the facts in original document as a factual knowledge graph, and then generate the high-quality summary via directly modeling consistency between summary and the factual knowledge graph. To that end, we first leverage two pointer network structures to capture the fact in original documents. Then, to enhance the traditional token-level likelihood loss, we design two extra semantic-level losses to measure the disagreement between a summary and facts from its original document. Extensive experiments on public datasets demonstrate that our ERPGN framework could outperform both classic abstractive summarization models and the state-of-the-art fact-aware baseline methods, with significant improvement in terms of faithfulness.
Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey
Authors: Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Scott Nickleach, Diego Socolinsky, Srinivasan Sengamedu, Christos Faloutsos
Date: 2024-02-27 00:00:00
Tasks: Language Modeling, Language Modelling, Navigate, Question Answering, Survey

Recent breakthroughs in large language modeling have facilitated rigorous exploration of their application in diverse tasks related to tabular data modeling, such as prediction, tabular data synthesis, question answering, and table understanding. Each task presents unique challenges and opportunities. However, there is currently a lack of comprehensive review that summarizes and compares the key techniques, metrics, datasets, models, and optimization approaches in this research domain. This survey aims to address this gap by consolidating recent progress in these areas, offering a thorough survey and taxonomy of the datasets, metrics, and methodologies utilized. It identifies strengths, limitations, unexplored territories, and gaps in the existing literature, while providing some insights for future research directions in this vital and rapidly evolving field. It also provides relevant code and datasets references. Through this comprehensive review, we hope to provide interested readers with pertinent references and insightful perspectives, empowering them with the necessary tools and knowledge to effectively navigate and address the prevailing challenges in the field.
TuckerDNCaching: high-quality negative sampling with tucker decomposition
Authors: Tiroshan Madushanka, Ryutaro Ichise
Date: 2023-06-20 00:00:00
Proceeding: Journal of Intelligent Information Systems 2023 6
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graphs, Link Prediction

Knowledge Graph Embedding (KGE) translates entities and relations of knowledge graphs (KGs) into a low-dimensional vector space, enabling an efficient way of predicting missing facts. Generally, KGE models are trained with positive and negative examples, discriminating positives against negatives. Nevertheless, KGs contain only positive facts; KGE training requires generating negatives from non-observed ones in KGs, referred to as negative sampling. Since KGE models are sensitive to inputs, negative sampling becomes crucial, and the quality of the negatives becomes critical in KGE training. Generative adversarial networks (GAN) and self-adversarial methods have recently been utilized in negative sampling to address the vanishing gradients observed with early negative sampling methods. However, they introduce the problem of false negatives with high probability. In this paper, we extend the idea of reducing false negatives by adopting a Tucker decomposition representation, i.e., TuckerDNCaching, to enhance the semantic soundness of latent relations among entities by introducing a relation feature space. TuckerDNCaching ensures the quality of generated negative samples, and the experimental results reflect that our proposed negative sampling method outperforms the existing state-of-the-art negative sampling methods.
Prosocial and Financial Incentives for Biodiversity Conservation: A Field Experiment Using a Smartphone App
Authors: Shusaku Sasaki, Takahiro Kubo, Shodai Kitano
Date: 2024-02-28 00:00:00

Ascertaining the number, type, and location of plant, insect, and animal species is essential for biodiversity conservation. However, comprehensively monitoring the situation only through public fixed-point surveys is challenging, and therefore information voluntarily provided by citizens assists in ascertaining the species distribution. To effectively encourage the citizens' data sharing behavior, this study proposed a prosocial incentive scheme in which, if they provide species information, donations are made to activities for saving endangered species. We conducted a field experiment with users (N=830) of a widely-used Japanese smartphone app to which they post species photos and measured the incentive's effect on their posting behavior. In addition, we measured the effect of a financial incentive scheme that provides monetary rewards for posting species photos and compared the two incentives' effects. The analyses revealed that while the prosocial incentive did not increase the number of posts on average, it did change the contents of posts, increasing the proportion of posts on rare species. On the contrary, the financial incentive statistically significantly increased the number of posts, in particular, on less rare and invasive species. Our findings suggest that the prosocial and financial incentives could stimulate different motivations and encourage different posting behaviors.
Cautious Decision-Making for Tree Ensembles
Authors: Haifei Zhang, Benjamin Quost, Marie-Hélène Masson
Date: 2023-11-19 00:00:00
Proceeding: ECSQARU 2023 2023 11
Tasks: Decision Making

Cautious classifiers are designed to make indeterminate decisions when the uncertainty on the input data or the model output is too high, so as to reduce the risk of making wrong decisions. In this paper, we propose two cautious decision-making procedures, by aggregating trees providing probability intervals constructed via the imprecise Dirichlet model. The trees are aggregated in the belief functions framework, by maximizing the lower expected discounted utility, so as to achieve a good compromise between model accuracy and determinacy. They can be regarded as generalizations of the two classical aggregation strategies for tree ensembles, i.e., averaging and voting. The efficiency and performance of the proposed procedures are tested on random forests and illustrated on three UCI datasets.
Learning Sets of Probabilities Through Ensemble Methods
Authors: Vu-Linh Nguyen, Haifei Zhang, Sébastien Destercke
Date: 2023-11-19 00:00:00
Proceeding: ECSQARU 2023 2023 11

A possible approach to obtain set-valued predictions is to learn for each query instance a probability set (a.k.a. credal set) representing its associated uncertainty. Theoretically founded decision rules extending classical expectation and inducing a partial order between predictions can be used to derive set-valued predictions. However, obtaining such a credal set by imprecisiating a given learning algorithm is usually computationally challenging, except for simple models such as decision trees or naive Bayes classifiers. In this paper, we propose a simple, easy-to-use quantile-based framework for estimating credal sets using the output of ensemble methods, that can also cope with complex types of data, such as images and mixed/multimodal data, etc. Experiments are conducted to highlight the usefulness of the proposed framework.
In-Person, Hybrid or Remote? Employers' Perspectives on the Future of Work Post-Pandemic
Authors: Divyakant Tahlyan, Hani Mahmassani, Amanda Stathopoulos, Maher Said, Susan Shaheen, Joan Walker, Breton Johnson
Date: 2024-02-28 00:00:00

We present an employer-side perspective on remote work through the pandemic using data from top executives of 129 employers in North America. Our analysis suggests that at least some of the pandemic-accelerated changes to the work location landscape will likely stick; with some form of hybrid work being the norm. However, the patterns will vary by department (HR/legal/sales/IT, etc.) and by sector of operations. Top three concerns among employers include: supervision and mentoring, reduction in innovation, and creativity; and the top three benefits include their ability to retain / recruit talent, positive impact on public image and their ability to compete. An Ordered Probit model of the expected April 2024 work location strategy revealed that those in transportation, warehousing, and manufacturing sectors, those with a fully in-person approach to work pre-COVID, and those with a negative outlook towards the impact of remote work are likely to be more in-person-centered, while those with fully remote work approach in April 2020 are likely to be less in-person-centered. Lastly, we present data on resumption of business travel, in-person client interactions and changes in office space reconfigurations that employers have made since the beginning of the pandemic.
Automated segmentation of lesions and organs at risk on [68Ga]Ga-PSMA-11 PET/CT images using self-supervised learning with Swin UNETR
Authors: Elmira Yazdani, Najme Karamzadeh-Ziarati, Seyyed Saeid Cheshmi, Mahdi Sadeghi, Parham Geramifar, Habibeh Vosoughi, Mahmood Kazemi Jahromi, Saeed Reza Kheradpisheh
Date: 2024-02-29 00:00:00
Proceeding: Cancer Imaging 2024 2
Tasks: Decoder, Medical Image Segmentation, Segmentation, Self-Supervised Learning

Prostate-specific membrane antigen (PSMA) PET/CT imaging is widely used for quantitative image analysis, especially in radioligand therapy (RLT) for metastatic castration-resistant prostate cancer (mCRPC). Unknown features influencing PSMA biodistribution can be explored by analyzing segmented organs at risk (OAR) and lesions. Manual segmentation is time-consuming and labor-intensive, so automated segmentation methods are desirable. Training deep-learning segmentation models is challenging due to the scarcity of high-quality annotated images. Addressing this, we developed shifted windows UNEt TRansformers (Swin UNETR) for fully automated segmentation. Within a self-supervised framework, the model’s encoder was pre-trained on unlabeled data. The entire model was fine-tuned, including its decoder, using labeled data.
Model Predictive Control with adaptive resilience for Denial-of-Service Attacks mitigation on a Regulated Dam
Authors: Raffaele Giuseppe Cestari, Stefano Longari, Stefano Zanero, Simone Formentin
Date: 2024-02-28 00:00:00
Tasks: Model Predictive Control

In recent years, SCADA (Supervisory Control and Data Acquisition) systems have increasingly become the target of cyber attacks. SCADAs are no longer isolated, as web-based applications expose strategic infrastructures to the outside world connection. In a cyber-warfare context, we propose a Model Predictive Control (MPC) architecture with adaptive resilience, capable of guaranteeing control performance in normal operating conditions and driving towards resilience against DoS (controller-actuator) attacks when needed. Since the attackers' goal is typically to maximize the system damage, we assume they solve an adversarial optimal control problem. An adaptive resilience factor is then designed as a function of the intensity function of a Hawkes process, a point process model estimating the occurrence of random events in time, trained on a moving window to estimate the return time of the next attack. We demonstrate the resulting MPC strategy's effectiveness in 2 attack scenarios on a real system with actual data, the regulated Olginate dam of Lake Como.
Modeling and Analysis of Crypto-Backed Over-Collateralized Stable Derivatives in DeFi
Authors: Zhenbang Feng, Hardhik Mohanty, Bhaskar Krishnamachari
Date: 2024-02-28 00:00:00

In decentralized finance (DeFi), stablecoins like DAI are designed to offer a stable value amidst the fluctuating nature of cryptocurrencies. We examine the class of crypto-backed stable derivatives, with a focus on mechanisms for price stabilization, which is exemplified by the well-known stablecoin DAI from MakerDAO. For simplicity, we focus on a single-collateral setting. We introduce a belief parameter to the simulation model of DAI in a previous work (DAISIM), reflecting market sentiments about the value and stability of DAI, and show that it better matches the expected behavior when this parameter is set to a sufficiently high value. We also propose a simple mathematical model of DAI price to explain its stability and dependency on ETH price. Finally, we analyze possible risk factors associated with these stable derivatives to provide valuable insights for stakeholders in the DeFi ecosystem.