pwc-archive/papers-with-abstracts huggingface dataset:
TMS-EEG Reliability: Bridging the Gap to Clinical Use
Authors: Giacomo Bertazzoli, Carlo Miniussi, Petro Julkunen, Marta Bortoletto
Date: 2024-02-07 00:00:00
Tasks: Diagnostic, EEG, Prognosis

Concurrent transcranial magnetic stimulation (TMS) and electroencephalography (EEG), or TMS-EEG, holds the potential to broaden the clinical applications of TMS beyond its traditional role in evaluating the cortico-spinal tract and motor cortices. TMS-evoked potentials (TEPs) have emerged as valuable tools in clinical research, enabling the assessment of cortical excitability and effective connectivity between cortical regions in various psychiatric and neurological disorders and are increasingly recognized as a promising candidate biomarker for aiding in diagnosis and prognosis. Despite the well-established diagnostic utility of TMS, the clinical implementation of TMS-EEG has yet to meet the necessary standards. One critical aspect that often remains unaddressed is the reliability of TEP measurements. In this context, we outline the crucial reliability assessments required to determine the clinical applicability of TEPs. Firstly, we conduct a comprehensive review of the existing literature on reliability, encompassing both theoretical and statistical considerations. Subsequently, we present the current state of knowledge on TEP reliability. We emphasize the specific elements of reliability that must be incorporated to facilitate a unified, evidence-derived assessment of TMS-EEG as a clinical tool.
Understanding Social Immunity in Ants: A Markovian Approach to Collective Cleaning Strategies
Authors: Isabella Bueno, Gabriel R. Palma, Idemauro A. R. Lara, Rafael A. Moral, Italo Delalibera Jr, Wesley A. C. Godoy
Date: 2024-02-08 00:00:00
Tasks: Decision Making

Understanding social immunity mechanisms in ant colonies remains crucial to comprehending the evolution of defense strategies in eusocial organisms. This study assumes the absence of the role of memory in the ants' defense strategy, considering that they can make a new exploration of collective cleaning behaviors. We investigate how worker interactions and previous behaviors influence the evolution of social immunity strategies in response to the presence of pathogens and vulnerable colony members. In this context, with the application of Markov transition models it was possible to describe changes in cleaning behavior over time influenced by the presence of vulnerable members and treatment with an entomopathogenic fungus. We found a significant effect on ant behavior when exposed to \it{Metarhizium anisopliae} when a fungus garden fragment and one larva were present. Our findings confirm a link between prophylactic cleaning and the presence of vulnerable members. Remarkably, distinct behaviors and transition times vary among treatments, revealing workers' adaptability to threats. Allogrooming shows adaptive changes when exposed to pathogens, potentially affecting pathogen transmission. In addition, our study elucidates the intricate interaction between internal and external factors shaping worker behavior. The influence of environmental context on decision-making principles emerges, emphasizing the importance of both intrinsic colony organization and external threats. By using a Markov model to understand ant hygiene behavior, we offer insights into social immunity mechanisms in eusocial organisms. Deciphering collective cleaning strategies can aid in understanding disease dynamics and decision-making processes in complex societies.
Association between Sitting Time and Urinary Incontinence in the US Population: data from the National Health and Nutrition Examination Survey (NHANES) 2007 to 2018
Authors: Guanbo Wang, Xingpeng Di
Date: 2024-02-08 00:00:00
Tasks: Nutrition, Survey

Background Urinary incontinence (UI) is a common health problem that affects the life and health quality of millions of people in the US. We aimed to investigate the association between sitting time and UI. Methods Across-sectional survey of adult participants of National Health and Nutrition Examination Survey 2007-2018 was performed. Weighted multivariable logistic and regression models were conducted to assess the association between sitting time and UI. Results A total of 22916 participants were enrolled. Prolonged sitting time was associated with urgent UI (UUI, Odds ratio [OR] = 1.184, 95% Confidence interval [CI] = 1.076 to 1.302, P = 0.001). Compared with patients with sitting time shorter than 7 hours, moderate activity increased the risk of prolonged sitting time over 7 hours in the fully-adjusted model (OR = 2.537, 95% CI = 1.419 to 4.536, P = 0.002). Sitting time over 7 hours was related to male mixed UI (MUI, OR = 1.581, 95% CI = 1.129 to 2.213, P = 0.010), and female stress UI (SUI, OR = 0.884, 95% CI = 0.795 to 0.983, P = 0.026) in the fully-adjusted model. Conclusions Prolonged sedentary sitting time (> 7 hours) indicated a high risk of UUI in all populations, female SUI and male MUI. Compared with sitting time shorter than 7 hours, the moderate activity could not reverse the risk of prolonged sitting, which warranted further studies for confirmation.
Downside Risk Reduction Using Regime-Switching Signals: A Statistical Jump Model Approach
Authors: Yizhan Shu, Chenyu Yu, John M. Mulvey
Date: 2024-02-07 00:00:00
Tasks: Time Series

This article investigates a regime-switching investment strategy aimed at mitigating downside risk by reducing market exposure during anticipated unfavorable market regimes. We highlight the statistical jump model (JM) for market regime identification, a recently developed robust model that distinguishes itself from traditional Markov-switching models by enhancing regime persistence through a jump penalty applied at each state transition. Our JM utilizes a feature set comprising risk and return measures derived solely from the return series, with the optimal jump penalty selected through a time-series cross-validation method that directly optimizes strategy performance. Our empirical analysis evaluates the realistic out-of-sample performance of various strategies on major equity indices from the US, Germany, and Japan from 1990 to 2023, in the presence of transaction costs and trading delays. The results demonstrate the consistent outperformance of the JM-guided strategy in reducing risk metrics such as volatility and maximum drawdown, and enhancing risk-adjusted returns like the Sharpe ratio, when compared to both hidden Markov model-guided strategy and the buy-and-hold strategy. These findings underline the enhanced persistence, practicality, and versatility of strategies utilizing JMs for regime-switching signals.
Portfolio Time Consistency and Utility Weighted Discount Rates
Authors: Oumar Mbodji, Traian A. Pirvu
Date: 2023-11-27 00:00:00
Tasks: Management

Merton portfolio management problem is studied in this paper within a stochastic volatility, non constant time discount rate, and power utility framework. This problem is time inconsistent and the way out of this predicament is to consider the subgame perfect strategies. The later are characterized through an extended Hamilton Jacobi Bellman (HJB) equation. A fixed point iteration is employed to solve the extended HJB equation. This is done in a two stage approach: in a first step the utility weighted discount rate is introduced and characterized as the fixed point of a certain operator; in the second step the value function is determined through a linear parabolic partial differential equation. Numerical experiments explore the effect of the time discount rate on the subgame perfect and precommitment strategies.
A Closer Look at the Limitations of Instruction Tuning
Authors: Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Dinesh Manocha
Date: 2024-02-03 00:00:00
Tasks: Hallucination

Instruction Tuning (IT), the process of training large language models (LLMs) using instruction-response pairs, has emerged as the predominant method for transforming base pre-trained LLMs into open-domain conversational agents. While IT has achieved notable success and widespread adoption, its limitations and shortcomings remain underexplored. In this paper, through rigorous experiments and an in-depth analysis of the changes LLMs undergo through IT, we reveal various limitations of IT. In particular, we show that (1) IT fails to enhance knowledge or skills in LLMs. LoRA fine-tuning is limited to learning response initiation and style tokens, and full-parameter fine-tuning leads to knowledge degradation. (2) Copying response patterns from IT datasets derived from knowledgeable sources leads to a decline in response quality. (3) Full-parameter fine-tuning increases hallucination by inaccurately borrowing tokens from conceptually similar instances in the IT dataset for generating responses. (4) Popular methods to improve IT do not lead to performance improvements over a simple LoRA fine-tuned model. Our findings reveal that responses generated solely from pre-trained knowledge consistently outperform responses by models that learn any form of new knowledge from IT on open-source datasets. We hope the insights and challenges revealed in this paper inspire future work in related directions.
More Agents Is All You Need
Authors: Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, Deheng Ye
Date: 2024-02-03 00:00:00
Tasks: All

We find that, simply via a sampling-and-voting method, the performance of large language models (LLMs) scales with the number of agents instantiated. Also, this method, termed as Agent Forest, is orthogonal to existing complicated methods to further enhance LLMs, while the degree of enhancement is correlated to the task difficulty. We conduct comprehensive experiments on a wide range of LLM benchmarks to verify the presence of our finding, and to study the properties that can facilitate its occurrence. Our code is publicly available at: https://github.com/MoreAgentsIsAllYouNeed/AgentForest
A Survey on Data Selection for LLM Instruction Tuning
Authors: Jiahao Wang, Bolin Zhang, Qianlong Du, Jiajun Zhang, Dianhui Chu
Date: 2024-02-04 00:00:00
Tasks: Instruction Following, Survey

Instruction tuning is a vital step of training large language models (LLM), so how to enhance the effect of instruction tuning has received increased attention. Existing works indicate that the quality of the dataset is more crucial than the quantity during instruction tuning of LLM. Therefore, recently a lot of studies focus on exploring the methods of selecting high-quality subset from instruction datasets, aiming to reduce training costs and enhance the instruction-following capabilities of LLMs. This paper presents a comprehensive survey on data selection for LLM instruction tuning. Firstly, we introduce the wildly used instruction datasets. Then, we propose a new taxonomy of the data selection methods and provide a detailed introduction of recent advances,and the evaluation strategies and results of data selection methods are also elaborated in detail. Finally, we emphasize the open challenges and present new frontiers of this task.
Zero-Shot Clinical Trial Patient Matching with LLMs
Authors: Michael Wornow, Alejandro Lozano, Dev Dash, Jenelle Jindal, Kenneth W. Mahaffey, Nigam H. Shah
Date: 2024-02-05 00:00:00

Matching patients to clinical trials is a key unsolved challenge in bringing new drugs to market. Today, identifying patients who meet a trial's eligibility criteria is highly manual, taking up to 1 hour per patient. Automated screening is challenging, however, as it requires understanding unstructured clinical text. Large language models (LLMs) offer a promising solution. In this work, we explore their application to trial matching. First, we design an LLM-based system which, given a patient's medical history as unstructured clinical text, evaluates whether that patient meets a set of inclusion criteria (also specified as free text). Our zero-shot system achieves state-of-the-art scores on the n2c2 2018 cohort selection benchmark. Second, we improve the data and cost efficiency of our method by identifying a prompting strategy which matches patients an order of magnitude faster and more cheaply than the status quo, and develop a two-stage retrieval pipeline that reduces the number of tokens processed by up to a third while retaining high performance. Third, we evaluate the interpretability of our system by having clinicians evaluate the natural language justifications generated by the LLM for each eligibility decision, and show that it can output coherent explanations for 97% of its correct decisions and 75% of its incorrect ones. Our results establish the feasibility of using LLMs to accelerate clinical trial operations.
Graph Neural Network and NER-Based Text Summarization
Authors: Imaad Zaffar Khan, Amaan Aijaz Sheikh, Utkarsh Sinha
Date: 2024-02-05 00:00:00
Tasks: Articles, Graph Neural Network, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Text Summarization

With the abundance of data and information in todays time, it is nearly impossible for man, or, even machine, to go through all of the data line by line. What one usually does is to try to skim through the lines and retain the absolutely important information, that in a more formal term is called summarization. Text summarization is an important task that aims to compress lengthy documents or articles into shorter, coherent representations while preserving the core information and meaning. This project introduces an innovative approach to text summarization, leveraging the capabilities of Graph Neural Networks (GNNs) and Named Entity Recognition (NER) systems. GNNs, with their exceptional ability to capture and process the relational data inherent in textual information, are adept at understanding the complex structures within large documents. Meanwhile, NER systems contribute by identifying and emphasizing key entities, ensuring that the summarization process maintains a focus on the most critical aspects of the text. By integrating these two technologies, our method aims to enhances the efficiency of summarization and also tries to ensures a high degree relevance in the condensed content. This project, therefore, offers a promising direction for handling the ever increasing volume of textual data in an information-saturated world.
Illuminate: A novel approach for depression detection with explainable analysis and proactive therapy using prompt engineering
Authors: Aryan Agrawal
Date: 2024-02-05 00:00:00
Tasks: Depression Detection, Dialogue Management, Management, Prompt Engineering

This paper introduces a novel paradigm for depression detection and treatment using advanced Large Language Models (LLMs): Generative Pre-trained Transformer 4 (GPT-4), Llama 2 chat, and Gemini. These LLMs are fine-tuned with specialized prompts to diagnose, explain, and suggest therapeutic interventions for depression. A unique few-shot prompting method enhances the models' ability to analyze and explain depressive symptoms based on the DSM-5 criteria. In the interaction phase, the models engage in empathetic dialogue management, drawing from resources like PsychDB and a Cognitive Behavioral Therapy (CBT) Guide, fostering supportive interactions with individuals experiencing major depressive disorders. Additionally, the research introduces the Illuminate Database, enriched with various CBT modules, aiding in personalized therapy recommendations. The study evaluates LLM performance using metrics such as F1 scores, Precision, Recall, Cosine similarity, and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) across different test sets, demonstrating their effectiveness. This comprehensive approach blends cutting-edge AI with established psychological methods, offering new possibilities in mental health care and showcasing the potential of LLMs in revolutionizing depression diagnosis and treatment strategies.
Best Practices for Text Annotation with Large Language Models
Authors: Petter Törnberg
Date: 2024-02-05 00:00:00
Tasks: Model Selection, Prompt Engineering, text annotation

Large Language Models (LLMs) have ushered in a new era of text annotation, as their ease-of-use, high accuracy, and relatively low costs have meant that their use has exploded in recent months. However, the rapid growth of the field has meant that LLM-based annotation has become something of an academic Wild West: the lack of established practices and standards has led to concerns about the quality and validity of research. Researchers have warned that the ostensible simplicity of LLMs can be misleading, as they are prone to bias, misunderstandings, and unreliable results. Recognizing the transformative potential of LLMs, this paper proposes a comprehensive set of standards and best practices for their reliable, reproducible, and ethical use. These guidelines span critical areas such as model selection, prompt engineering, structured prompting, prompt stability analysis, rigorous model validation, and the consideration of ethical and legal implications. The paper emphasizes the need for a structured, directed, and formalized approach to using LLMs, aiming to ensure the integrity and robustness of text annotation practices, and advocates for a nuanced and critical engagement with LLMs in social scientific research.
Financial Report Chunking for Effective Retrieval Augmented Generation
Authors: Antonio Jimeno Yepes, Yao You, Jan Milczek, Sebastian Laverde, Renyu Li
Date: 2024-02-05 00:00:00
Tasks: Chunking, document understanding, Information Retrieval, RAG, Retrieval, Retrieval-augmented Generation

Chunking information is a key step in Retrieval Augmented Generation (RAG). Current research primarily centers on paragraph-level chunking. This approach treats all texts as equal and neglects the information contained in the structure of documents. We propose an expanded approach to chunk documents by moving beyond mere paragraph-level chunking to chunk primary by structural element components of documents. Dissecting documents into these constituent elements creates a new way to chunk documents that yields the best chunk size without tuning. We introduce a novel framework that evaluates how chunking based on element types annotated by document understanding models contributes to the overall context and accuracy of the information retrieved. We also demonstrate how this approach impacts RAG assisted Question & Answer task performance. Our research includes a comprehensive analysis of various element types, their role in effective information retrieval, and the impact they have on the quality of RAG outputs. Findings support that element type based chunking largely improve RAG results on financial reporting. Through this research, we are also able to answer how to uncover highly accurate RAG.
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K
Authors: Tao Yuan, Xuefei Ning, Dong Zhou, Zhijie Yang, Shiyao Li, Minghui Zhuang, Zheyue Tan, Zhuyu Yao, Dahua Lin, Boxun Li, Guohao Dai, Shengen Yan, Yu Wang
Date: 2024-02-06 00:00:00
Tasks: 16k, Benchmarking

State-of-the-art large language models (LLMs) are now claiming remarkable supported context lengths of 256k or even more. In contrast, the average context lengths of mainstream benchmarks are insufficient (5k-21k), and they suffer from potential knowledge leakage and inaccurate metrics, resulting in biased evaluation. This paper introduces LV-Eval, a challenging long-context benchmark with five length levels (16k, 32k, 64k, 128k, and 256k) reaching up to 256k words. LV-Eval features two main tasks, single-hop QA and multi-hop QA, comprising 11 bilingual datasets. The design of LV-Eval has incorporated three key techniques, namely confusing facts insertion, keyword and phrase replacement, and keyword-recall-based metric design. The advantages of LV-Eval include controllable evaluation across different context lengths, challenging test instances with confusing facts, mitigated knowledge leakage, and more objective evaluations. We evaluate 15 LLMs on LV-Eval and conduct ablation studies on the benchmarking techniques. The results reveal that: (i) Moonshot-v1 and recent large-scale open-source models, such as Qwen-2.5-72B and Llama-3.1-70B, achieve the highest performance on LV-Eval, particularly at lengths below 64k. (ii) Models exhibit distinct score trends. For example, GLM-4-9B-128k, Yi-6B-200k, and Llama3-8B-1M exhibit a relatively gentle degradation of performance, but their absolute performances may not necessarily be higher than those of LLMs with shorter context lengths. (iii) LLMs' performances can significantly degrade in the presence of confusing information, especially in the pressure test of "needle in a haystack". (iv) Issues related to knowledge leakage and inaccurate metrics introduce bias in evaluation, and these concerns are alleviated in LV-Eval. All datasets and evaluation codes are released at: https://github.com/infinigence/LVEval.
Large Language Model for Table Processing: A Survey
Authors: Weizheng Lu, Jing Zhang, Ju Fan, Zihao Fu, Yueguo Chen, Xiaoyong Du
Date: 2024-02-04 00:00:00
Tasks: Fact Verification, Language Modeling, Language Modelling, Large Language Model, Prompt Engineering, Question Answering, Survey

Tables, typically two-dimensional and structured to store large amounts of data, are essential in daily activities like database queries, spreadsheet manipulations, web table question answering, and image table information extraction. Automating these table-centric tasks with Large Language Models (LLMs) or Visual Language Models (VLMs) offers significant public benefits, garnering interest from academia and industry. This survey provides a comprehensive overview of table-related tasks, examining both user scenarios and technical aspects. It covers traditional tasks like table question answering as well as emerging fields such as spreadsheet manipulation and table data analysis. We summarize the training techniques for LLMs and VLMs tailored for table processing. Additionally, we discuss prompt engineering, particularly the use of LLM-powered agents, for various table-related tasks. Finally, we highlight several challenges, including diverse user input when serving and slow thinking using chain-of-thought.
Unsupervised Motion Retargeting for Human-Robot Imitation
Authors: Louis Annabi, Ziqi Ma, Sao Mai Nguyen
Date: 2024-01-18 00:00:00
Tasks: Decoder, Deep Learning, motion retargeting, Translation

This early-stage research work aims to improve online human-robot imitation by translating sequences of joint positions from the domain of human motions to a domain of motions achievable by a given robot, thus constrained by its embodiment. Leveraging the generalization capabilities of deep learning methods, we address this problem by proposing an encoder-decoder neural network model performing domain-to-domain translation. In order to train such a model, one could use pairs of associated robot and human motions. Though, such paired data is extremely rare in practice, and tedious to collect. Therefore, we turn towards deep learning methods for unpaired domain-to-domain translation, that we adapt in order to perform human-robot imitation.