pwc-archive/papers-with-abstracts huggingface dataset:
Fine-grained Contract NER using instruction based model
Authors: Hiranmai Sri Adibhatla, Pavan Baswani, Manish Shrivastava
Date: 2024-01-24 00:00:00
Tasks: Few-Shot Learning, Hallucination, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentence, Text Generation

Lately, instruction-based techniques have made significant strides in improving performance in few-shot learning scenarios. They achieve this by bridging the gap between pre-trained language models and fine-tuning for specific downstream tasks. Despite these advancements, the performance of Large Language Models (LLMs) in information extraction tasks like Named Entity Recognition (NER), using prompts or instructions, still falls short of supervised baselines. The reason for this performance gap can be attributed to the fundamental disparity between NER and LLMs. NER is inherently a sequence labeling task, where the model must assign entity-type labels to individual tokens within a sentence. In contrast, LLMs are designed as a text generation task. This distinction between semantic labeling and text generation leads to subpar performance. In this paper, we transform the NER task into a text-generation task that can be readily adapted by LLMs. This involves enhancing source sentences with task-specific instructions and answer choices, allowing for the identification of entities and their types within natural language. We harness the strength of LLMs by integrating supervised learning within them. The goal of this combined strategy is to boost the performance of LLMs in extraction tasks like NER while simultaneously addressing hallucination issues often observed in LLM-generated content. A novel corpus Contract NER comprising seven frequently observed contract categories, encompassing named entities associated with 18 distinct legal entity types is released along with our baseline models. Our models and dataset are available to the community for future research * .
TPRF: A Transformer-based Pseudo-Relevance Feedback Model for Efficient and Effective Retrieval
Authors: Hang Li, Chuting Yu, Ahmed Mourad, Bevan Koopman, Guido Zuccon
Date: 2024-01-24 00:00:00
Tasks: CPU, Retrieval

This paper considers Pseudo-Relevance Feedback (PRF) methods for dense retrievers in a resource constrained environment such as that of cheap cloud instances or embedded systems (e.g., smartphones and smartwatches), where memory and CPU are limited and GPUs are not present. For this, we propose a transformer-based PRF method (TPRF), which has a much smaller memory footprint and faster inference time compared to other deep language models that employ PRF mechanisms, with a marginal effectiveness loss. TPRF learns how to effectively combine the relevance feedback signals from dense passage representations. Specifically, TPRF provides a mechanism for modelling relationships and weights between the query and the relevance feedback signals. The method is agnostic to the specific dense representation used and thus can be generally applied to any dense retriever.
Decentralized Collaborative Learning with Adaptive Reference Data for On-Device POI Recommendation
Authors: Ruiqi Zheng, Liang Qu, Tong Chen, Lizhen Cui, Yuhui Shi, Hongzhi Yin
Date: 2024-01-24 00:00:00

In Location-based Social Networks, Point-of-Interest (POI) recommendation helps users discover interesting places. There is a trend to move from the cloud-based model to on-device recommendations for privacy protection and reduced server reliance. Due to the scarcity of local user-item interactions on individual devices, solely relying on local instances is not adequate. Collaborative Learning (CL) emerges to promote model sharing among users, where reference data is an intermediary that allows users to exchange their soft decisions without directly sharing their private data or parameters, ensuring privacy and benefiting from collaboration. However, existing CL-based recommendations typically use a single reference for all users. Reference data valuable for one user might be harmful to another, given diverse user preferences. Users may not offer meaningful soft decisions on items outside their interest scope. Consequently, using the same reference data for all collaborations can impede knowledge exchange and lead to sub-optimal performance. To address this gap, we introduce the Decentralized Collaborative Learning with Adaptive Reference Data (DARD) framework, which crafts adaptive reference data for effective user collaboration. It first generates a desensitized public reference data pool with transformation and probability data generation methods. For each user, the selection of adaptive reference data is executed in parallel by training loss tracking and influence function. Local models are trained with individual private data and collaboratively with the geographical and semantic neighbors. During the collaboration between two users, they exchange soft decisions based on a combined set of their adaptive reference data. Our evaluations across two real-world datasets highlight DARD's superiority in recommendation performance and addressing the scarcity of available reference data.
Query Exposure Prediction for Groups of Documents in Rankings
Authors: Thomas Jaenich, Graham McDonald, Iadh Ounis
Date: 2024-01-24 00:00:00
Tasks: Information Retrieval, Prediction, Re-Ranking, Retrieval

The main objective of an Information Retrieval system is to provide a user with the most relevant documents to the user's query. To do this, modern IR systems typically deploy a re-ranking pipeline in which a set of documents is retrieved by a lightweight first-stage retrieval process and then re-ranked by a more effective but expensive model. However, the success of a re-ranking pipeline is heavily dependent on the performance of the first stage retrieval, since new documents are not usually identified during the re-ranking stage. Moreover, this can impact the amount of exposure that a particular group of documents, such as documents from a particular demographic group, can receive in the final ranking. For example, the fair allocation of exposure becomes more challenging or impossible if the first stage retrieval returns too few documents from certain groups, since the number of group documents in the ranking affects the exposure more than the documents' positions. With this in mind, it is beneficial to predict the amount of exposure that a group of documents is likely to receive in the results of the first stage retrieval process, in order to ensure that there are a sufficient number of documents included from each of the groups. In this paper, we introduce the novel task of query exposure prediction (QEP). Specifically, we propose the first approach for predicting the distribution of exposure that groups of documents will receive for a given query. Our new approach, called GEP, uses lexical information from individual groups of documents to estimate the exposure the groups will receive in a ranking. Our experiments on the TREC 2021 and 2022 Fair Ranking Track test collections show that our proposed GEP approach results in exposure predictions that are up to 40 % more accurate than the predictions of adapted existing query performance prediction and resource allocation approaches.
Predicting IR Personalization Performance using Pre-retrieval Query Predictors
Authors: Eduardo Vicente-López, Luis M. de Campos, Juan M. Fernández-Luna, Juan F. Huete
Date: 2024-01-24 00:00:00
Tasks: Retrieval

Personalization generally improves the performance of queries but in a few cases it may also harms it. If we are able to predict and therefore to disable personalization for those situations, the overall performance will be higher and users will be more satisfied with personalized systems. We use some state-of-the-art pre-retrieval query performance predictors and propose some others including the user profile information for the previous purpose. We study the correlations among these predictors and the difference between the personalized and the original queries. We also use classification and regression techniques to improve the results and finally reach a bit more than one third of the maximum ideal performance. We think this is a good starting point within this research line, which certainly needs more effort and improvements.
A Big Data Architecture for Early Identification and Categorization of Dark Web Sites
Authors: Javier Pastor-Galindo, Hông-Ân Sandlin, Félix Gómez Mármol, Gérôme Bovet, Gregorio Martínez Pérez
Date: 2024-01-24 00:00:00
Tasks: Dimensionality Reduction

The dark web has become notorious for its association with illicit activities and there is a growing need for systems to automate the monitoring of this space. This paper proposes an end-to-end scalable architecture for the early identification of new Tor sites and the daily analysis of their content. The solution is built using an Open Source Big Data stack for data serving with Kubernetes, Kafka, Kubeflow, and MinIO, continuously discovering onion addresses in different sources (threat intelligence, code repositories, web-Tor gateways, and Tor repositories), downloading the HTML from Tor and deduplicating the content using MinHash LSH, and categorizing with the BERTopic modeling (SBERT embedding, UMAP dimensionality reduction, HDBSCAN document clustering and c-TF-IDF topic keywords). In 93 days, the system identified 80,049 onion services and characterized 90% of them, addressing the challenge of Tor volatility. A disproportionate amount of repeated content is found, with only 6.1% unique sites. From the HTML files of the dark sites, 31 different low-topics are extracted, manually labeled, and grouped into 11 high-level topics. The five most popular included sexual and violent content, repositories, search engines, carding, cryptocurrencies, and marketplaces. During the experiments, we identified 14 sites with 13,946 clones that shared a suspiciously similar mirroring rate per day, suggesting an extensive common phishing network. Among the related works, this study is the most representative characterization of onion services based on topics to date.
It's About Time: Incorporating Temporality in Retrieval Augmented Language Models
Authors: Anoushka Gade, Jorjeta Jetcheva
Date: 2024-01-24 00:00:00
Tasks: Few-Shot Learning, Hallucination, Information Retrieval, Language Modelling, Question Answering, Retrieval, Semantic Similarity, Semantic Textual Similarity

The web serves as a global repository of knowledge, used by billions of people to search for information. Ensuring that users receive the most relevant and up-to-date information, especially in the presence of multiple versions of web content from different time points remains a critical challenge for information retrieval. This challenge has recently been compounded by the increased use of question answering tools trained on Wikipedia or web content and powered by large language models (LLMs) which have been found to make up information (or hallucinate), and in addition have been shown to struggle with the temporal dimensions of information. Even Retriever Augmented Language Models (RALMs) which incorporate a document database to reduce LLM hallucination are unable to handle temporal queries correctly. This leads to instances where RALMs respond to queries such as "Who won the Wimbledon Championship?", by retrieving document passages related to Wimbledon but without the ability to differentiate between them based on how recent they are. In this paper, we propose and evaluate, TempRALM, a temporally-aware Retriever Augmented Language Model (RALM) with few-shot learning extensions, which takes into account both semantically and temporally relevant documents relative to a given query, rather than relying on semantic similarity alone. We show that our approach results in up to 74% improvement in performance over the baseline RALM model, without requiring model pre-training, recalculating or replacing the RALM document index, or adding other computationally intensive elements.
Into the crossfire: evaluating the use of a language model to crowdsource gun violence reports
Authors: Adriano Belisario, Scott Hale, Luc Rocher
Date: 2024-01-16 00:00:00
Tasks: Language Modeling, Language Modelling

Gun violence is a pressing and growing human rights issue that affects nearly every dimension of the social fabric, from healthcare and education to psychology and the economy. Reliable data on firearm events is paramount to developing more effective public policy and emergency responses. However, the lack of comprehensive databases and the risks of in-person surveys prevent human rights organizations from collecting needed data in most countries. Here, we partner with a Brazilian human rights organization to conduct a systematic evaluation of language models to assist with monitoring real-world firearm events from social media data. We propose a fine-tuned BERT-based model trained on Twitter (now X) texts to distinguish gun violence reports from ordinary Portuguese texts. Our model achieves a high AUC score of 0.97. We then incorporate our model into a web application and test it in a live intervention. We study and interview Brazilian analysts who continuously fact-check social media texts to identify new gun violence events. Qualitative assessments show that our solution helped all analysts use their time more efficiently and expanded their search capacities. Quantitative assessments show that the use of our model was associated with more analysts' interactions with online users reporting gun violence. Taken together, our findings suggest that modern Natural Language Processing techniques can help support the work of human rights organizations.
Simulation of Fractional Brownian Motion and Related Stochastic Processes in Practice: A Straightforward Approach
Authors: Yat Chun Chester Wong, Paul Alexander Bilokon
Date: 2024-01-25 00:00:00
Proceeding: SSRN 2024 1

Fractional Brownian motion, with its long-time correlated increments, has been applied in many fields in recent years. After volatility was shown to be rough by Gatheral, Jaisson, and Rosenbaum, fractional Brownian motion has gained popularity as a financial model. In this work, we revisit the definitions and properties of the univariate and multivariate fractional Brownian motions, and consider four simulation methods. We demonstrate the issues associated with applying the standard Euler scheme for simulating stochastic processes driven by fractional Brownian motion with $H < \frac{1}{2}$ (which we call the rough models). We then consider an approximate method for simulating such rough models based on the fast algorithm by Ma and Wu, which accounts for a 10x speedup. Finally, we consider applications of these methods to option pricing.
A Multi-Scale Recurrent Framework for Motion Segmentation With Event Camera
Authors: Shaobo Zhang, Lei Sun, Kaiwei Wang
Date: 2023-07-28 00:00:00
Proceeding: IEEE Access 2023 7
Tasks: Motion Segmentation, Segmentation

Motion segmentation is a formidable computer vision task, aiming to segment moving targets from a dynamic scene. In this paper, we choose to introduce an additional modality to bolster the robustness. The event camera is a bio-inspired sensor that accurately detects and captures intensity changes with exceptional temporal resolution and dynamic range, which is an optimal choice for motion segmentation. Therefore, we present a novel framework for event-based motion segmentation and propose Multi-Scale Recurrent Neural Network (MSRNN) to fuse temporal information efficiently. To our best knowledge, it is the first time that a multi-scale recurrent architecture is implemented in event-based motion segmentation. The proposed framework is evaluated through experiments conducted on the EV-IMO dataset. Our method achieves a mean Intersection-over-Union (mIoU) of 82.0%, which sets a new state-of-the-art in motion segmentation. To further validate our approach in arduous real-world scenarios, we introduce the Event Challenging Motion dataset, consisting of 350 images and corresponding events, in which our method outperforms the other methods by 1.5% in Intersection-over-Union (IoU).
Real-time Risk Metrics for Programmatic Stablecoin Crypto Asset-Liability Management (CALM)
Authors: Marcel Bluhm, Adrian Cachinero Vasiljević, Sébastien Derivaux, Søren Terp Hørlück Jessen
Date: 2024-01-24 00:00:00
Tasks: Management

Stablecoins have turned out to be the "killer" use case of the growing digital asset space. However, risk management frameworks, including regulatory ones, have been largely absent. In this paper, we address the critical question of measuring and managing risk in stablecoin protocols, which operate on public blockchain infrastructure. The on-chain environment makes it possible to monitor risk and automate its management via transparent smart-contracts in real-time. We propose two risk metrics covering capitalization and liquidity of stablecoin protocols. We then explore in a case-study type analysis how our risk management framework can be applied to DAI, the biggest decentralized stablecoin by market capitalisation to-date, governed by MakerDAO. Based on our findings, we recommend that the protocol explores implementing automatic capital buffer adjustments and dynamic maturity gap matching. Our analysis demonstrates the practical benefits for scalable (prudential) risk management stemming from real-time availability of high-quality, granular, tamper-resistant on-chain data in the digital asset space. We name this approach Crypto Asset-Liability Management (CALM).
Variational Quantum Linear Solver-based Combination Rules in Dempster–Shafer Theory
Authors: Hao Luo, Qianli Zhou, Zhen Li, Yong Deng
Date: 2023-10-10 00:00:00
Proceeding: journal 2023 10

Dempster–Shafer Theory (DST), as a method of handling uncertain information, is widely used in decisionmaking and information fusion. But the issue of exponential computational complexity limits its real-time application. Since quantum computers with its natural parallel computing capability can theoretically achieve speedup, an HHL-based algorithm for implementing DST operations on quantum circuits has been proposed. However, the algorithm is impossible to be implemented with high accuracy due to the limited performance of quantum computers in the current noisy intermediate-scale quantum (NISQ) era. In response to that, we utilize an up-to-date scheme Variational Quantum Linear Solver (VQLS) to conduct DST operations by solving the linear system. Besides, VQLS further reveals the structural consistency of DST with quantum computation, which enables faster circuit configuration and computation. In this paper, we first realize the transformation in belief functions by VQLS and then extends it to a methods of implementing combination rules. Simulated under a realistic classification task, the feasibility and accuracy of the proposed VQLS-based method are verified. The new VQLS-based method proposed is able to achieve combination rules at a lower error level with fewer quantum resources, which is more suitable for the NISQ era.
Image-based human re-identification: Which covariates are actually (the most) important?
Authors: Kailash Hambarde, Hugo Proença
Date: 2024-01-20 00:00:00
Proceeding: Image and Vision Computing 2024 1
Tasks: Action Recognition, Image-To-Video Person Re-Identification, Person Identification, Person Recognition, Person Re-Identification, Person Retrieval, Video-Based Person Re-Identification

Human re-identification (re-ID) is nowadays among the most popular topics in computer vision, due to the increasing importance given to safety/security in modern societies. Being expected to sun in totally uncontrolled data acquisition settings (e.g., visual surveillance) automated re-ID not only depends on various factors that may occur in non-controlled data acquisition settings, but - most importantly - performance varies with respect to different subject features (e.g., gender, height, ethnicity, clothing, and action being performed), which may result in highly biased and undesirable automata. While many efforts have been putted in increase the robustness of identification to uncontrolled settings, a systematic assessment of the actual variations in performance with respect to each subject feature remains to be done. Accordingly, the contributions of this paper are threefold: 1) we report the correlation between the performance of three state-of-the-art re-ID models and different subject features; 2) we discuss the most concerning features and report valuable insights about the roles of the various features in re-ID performance, which can be used to develop more effective and unbiased re-ID systems; and 3) we leverage the concept of biometric menagerie, in order to identify the groups of individuals that typically fall into the most common menagerie families (e.g., goats, lambs, and wolves). Our findings not only contribute to a better understanding of the factors affecting re-ID performance, but also may offer practical guidance for researchers and practitioners concerned on human re-identification development.
Evaluating and Enhancing Large Language Models Performance in Domain-specific Medicine: Osteoarthritis Management with DocOA
Authors: Xi Chen, MingKe You, Li Wang, Weizhi Liu, Yu Fu, Jie Xu, Shaoting Zhang, Gang Chen, Kang Li, Jian Li
Date: 2024-01-20 00:00:00
Tasks: Management, RAG, Retrieval, Retrieval-augmented Generation

The efficacy of large language models (LLMs) in domain-specific medicine, particularly for managing complex diseases such as osteoarthritis (OA), remains largely unexplored. This study focused on evaluating and enhancing the clinical capabilities of LLMs in specific domains, using osteoarthritis (OA) management as a case study. A domain specific benchmark framework was developed, which evaluate LLMs across a spectrum from domain-specific knowledge to clinical applications in real-world clinical scenarios. DocOA, a specialized LLM tailored for OA management that integrates retrieval-augmented generation (RAG) and instruction prompts, was developed. The study compared the performance of GPT-3.5, GPT-4, and a specialized assistant, DocOA, using objective and human evaluations. Results showed that general LLMs like GPT-3.5 and GPT-4 were less effective in the specialized domain of OA management, particularly in providing personalized treatment recommendations. However, DocOA showed significant improvements. This study introduces a novel benchmark framework which assesses the domain-specific abilities of LLMs in multiple aspects, highlights the limitations of generalized LLMs in clinical contexts, and demonstrates the potential of tailored approaches for developing domain-specific medical LLMs.