pwc-archive/papers-with-abstracts huggingface dataset:
Large Language Models in the Clinic: A Comprehensive Benchmark
Authors: Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David A. Clifton
Date: 2024-04-25 00:00:00
Tasks: Decision Making, Document Summarization, Question Answering, Text Generation

The adoption of large language models (LLMs) to assist clinicians has attracted remarkable attention. Existing works mainly adopt the close-ended question-answering (QA) task with answer options for evaluation. However, many clinical decisions involve answering open-ended questions without pre-set options. To better understand LLMs in the clinic, we construct a benchmark ClinicBench. We first collect eleven existing datasets covering diverse clinical language generation, understanding, and reasoning tasks. Furthermore, we construct six novel datasets and clinical tasks that are complex but common in real-world practice, e.g., open-ended decision-making, long document processing, and emerging drug analysis. We conduct an extensive evaluation of twenty-two LLMs under both zero-shot and few-shot settings. Finally, we invite medical experts to evaluate the clinical usefulness of LLMs. The benchmark data is available at https://github.com/AI-in-Health/ClinicBench.
PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Tasks
Authors: Ian Char, Jeff Schneider
Date: 2023-07-12 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Deep Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning (RL) has shown immense potential for learning to control systems through data alone. However, one challenge deep RL faces is that the full state of the system is often not observable. When this is the case, the policy needs to leverage the history of observations to infer the current state. At the same time, differences between the training and testing environments makes it critical for the policy not to overfit to the sequence of observations it sees at training time. As such, there is an important balancing act between having the history encoder be flexible enough to extract relevant information, yet be robust to changes in the environment. To strike this balance, we look to the PID controller for inspiration. We assert the PID controller's success shows that only summing and differencing are needed to accumulate information over time for many control tasks. Following this principle, we propose two architectures for encoding history: one that directly uses PID features and another that extends these core ideas and can be used in arbitrary control tasks. When compared with prior approaches, our encoders produce policies that are often more robust and achieve better performance on a variety of tracking tasks. Going beyond tracking tasks, our policies achieve 1.7x better performance on average over previous state-of-the-art methods on a suite of locomotion control tasks.
Incorporating ESO into Deep Koopman Operator Modelling for Control of Autonomous Vehicles
Authors: Hao Chen, Chen Lv
Date: 2024-05-16 00:00:00
Tasks: Autonomous Vehicles, Model Predictive Control

Koopman operator theory is a kind of data-driven modelling approach that accurately captures the nonlinearities of mechatronic systems such as vehicles against physics-based methods. However, the infinite-dimensional Koopman operator is impossible to implement in real-world applications. To approximate the infinite-dimensional Koopman operator through collection dataset rather than manual trial and error, we adopt deep neural networks (DNNs) to extract basis functions by offline training and map the nonlinearities of vehicle planar dynamics into a linear form in the lifted space. Besides, the effects of the dimensions of basis functions on the model accuracy are explored. Further, the extended state observer (ESO) is introduced to online estimate the total disturbance in the lifted space and compensate for the modelling errors and residuals of the learned deep Koopman operator (DK) while also improving its generalization. Then, the proposed model is applied to predict vehicle states within prediction horizons and later formulates the constrained finite-time optimization problem of model predictive control (MPC), i.e., ESO-DKMPC. In terms of the trajectory tracking of autonomous vehicles, the ESO-DKMPC generates the wheel steering angle to govern lateral motions based on the decoupling control structure. The various conditions under the double-lane change scenarios are built on the CarSim/Simulink co-simulation platform, and extensive comparisons are conducted with the linear MPC (LMPC) and nonlinear MPC (NMPC) informed by the physics-based model. The results indicate that the proposed ESO-DKMPC has better tracking performance and moderate efficacy both within linear and nonlinear regions.
Introducing Learning Rate Adaptation CMA-ES into Rigid 2D/3D Registration for Robotic Navigation in Spine Surgery
Authors: Zhirun Zhang, Minheng Chen
Date: 2024-05-16 00:00:00

The covariance matrix adaptive evolution strategy (CMA-ES) has been widely used in the field of 2D/3D registration in recent years. This optimization method exhibits exceptional robustness and usability for complex surgical scenarios. However, due to the inherent ill-posed nature of the 2D/3D registration task and the presence of numerous local minima in the landscape of similarity measures. Evolution strategies often require a larger population size in each generation in each generation to ensure the stability of registration and the globality and effectiveness of search, which makes the entire process computationally expensive. In this paper, we build a 2D/3D registration framework based on a learning rate adaptation CMA-ES manner. The framework employs a fixed and small population size, leading to minimized runtime and optimal utilization of computing resources. We conduct experimental comparisons between the proposed framework and other intensity-based baselines using a substantial volume of synthetic data. The results suggests that our method demonstrates superiority in both registration accuracy and running time. Code is available at github.com/m1nhengChen/CMAES-reg.
Spatial Cognition: a Wave Hypothesis
Authors: Robert Worden
Date: 2024-05-16 00:00:00
Tasks: Object Tracking

Animals build Bayesian 3D models of their surroundings, to control their movements. There is strong selection pressure to make these models as precise as possible, given their sense data. A previous paper has described how a precise 3D model of space can be built by object tracking. This only works if 3D locations are stored with high spatial precision. Neural models of 3D spatial memory have large random errors; too large to support the tracking model. An alternative is described, in which neurons couple to a wave excitation in the brain, representing 3-D space. This can give high spatial precision, fast response, and other benefits. Three lines of evidence support the wave hypothesis: (1) it has better precision and speed than neural spatial memory, and is good enough to support object tracking; (2) the central body of the insect brain, whose form is highly conserved across all insect species, is well suited to hold a wave; and (3) the thalamus, whose round shape is conserved across all mammal species, is well suited to hold a wave. These lines of evidence strongly support the wave hypothesis.
Spurious reconstruction from brain activity
Authors: Ken Shirakawa, Yoshihiro Nagano, Misato Tanaka, Shuntaro C. Aoki, Kei Majima, Yusuke Muraki, Yukiyasu Kamitani
Date: 2024-05-16 00:00:00
Tasks: Brain Decoding, Hallucination, Image Reconstruction

Advances in brain decoding, particularly visual image reconstruction, have sparked discussions about the societal implications and ethical considerations of neurotechnology. As these methods aim to recover visual experiences from brain activity and achieve prediction beyond training samples (zero-shot prediction), it is crucial to assess their capabilities and limitations to inform public expectations and regulations. Our case study of recent text-guided reconstruction methods, which leverage a large-scale dataset (Natural Scene Dataset, NSD) and text-to-image diffusion models, reveals limitations in their generalizability. We found poor performance when applying these methods to a different dataset designed to prevent category overlaps between training and test sets. UMAP visualization of the text features with NSD images showed a limited diversity of semantic and visual clusters, with overlap between training and test sets. Formal analysis and simulations demonstrated that clustered training samples can lead to "output dimension collapse," restricting predictable output feature dimensions. Simulations further showed that diversifying the training set improved generalizability. However, text features alone are insufficient for mapping to the visual space. We argue that recent realistic reconstructions may primarily be a blend of classification into trained categories and generation of inauthentic images through text-to-image diffusion (hallucination). Diverse datasets and compositional representations spanning the image space are essential for genuine zero-shot prediction. Interdisciplinary discussions grounded in understanding the current capabilities and limitations, as well as ethical considerations, of the technology are crucial for its responsible development.
Semantic Communication via Rate Distortion Perception Bottleneck
Authors: Zihe Zhao, Chunyue Wang
Date: 2024-05-16 00:00:00
Tasks: Image Reconstruction, Semantic Communication

With the advancement of Artificial Intelligence (AI) technology, next-generation wireless communication network is facing unprecedented challenge. Semantic communication has become a novel solution to address such challenges, with enhancing the efficiency of bandwidth utilization by transmitting meaningful information and filtering out superfluous data. Unfortunately, recent studies have shown that classical Shannon information theory primarily focuses on the bit-level distortion, which cannot adequately address the perceptual quality issues of data reconstruction at the receiver end. In this work, we consider the impact of semantic-level distortion on semantic communication. We develop an image inference network based on the Information Bottleneck (IB) framework and concurrently establish an image reconstruction network. This network is designed to achieve joint optimization of perception and bit-level distortion, as well as image inference, associated with compressing information. To maintain consistency with the principles of IB for handling high-dimensional data, we employ variational approximation methods to simplify the optimization problem. Finally, we confirm the existence of the rate distortion perception tradeoff within IB framework through experimental analysis conducted on the MNIST dataset.
Robust Singing Voice Transcription Serves Synthesis
Authors: RuiQi Li, Yu Zhang, Yongqi Wang, Zhiqing Hong, Rongjie Huang, Zhou Zhao
Date: 2024-05-16 00:00:00
Tasks: Decoder, Singing Voice Synthesis

Note-level Automatic Singing Voice Transcription (AST) converts singing recordings into note sequences, facilitating the automatic annotation of singing datasets for Singing Voice Synthesis (SVS) applications. Current AST methods, however, struggle with accuracy and robustness when used for practical annotation. This paper presents ROSVOT, the first robust AST model that serves SVS, incorporating a multi-scale framework that effectively captures coarse-grained note information and ensures fine-grained frame-level segmentation, coupled with an attention-based pitch decoder for reliable pitch prediction. We also established a comprehensive annotation-and-training pipeline for SVS to test the model in real-world settings. Experimental findings reveal that ROSVOT achieves state-of-the-art transcription accuracy with either clean or noisy inputs. Moreover, when trained on enlarged, automatically annotated datasets, the SVS model outperforms its baseline, affirming the capability for practical application. Audio samples are available at https://rosvot.github.io.
HeMiTo-dynamics: a characterisation of mammalian prion toxicity using non-dimensionalisation, linear stability and perturbation analyses
Authors: Johannes G. Borgqvist, Christoffer Gretarsson Alexandersen
Date: 2024-05-16 00:00:00

Prion-like proteins play crucial parts in biological processes in organisms ranging from yeast to humans. For instance, many neurodegenerative diseases are believed to be caused by the production of prion-like proteins in neural tissue. As such, understanding the dynamics of prion-like protein production is a vital step toward treating neurodegenerative disease. Mathematical models of prion-like protein dynamics show great promise as a tool for predicting disease trajectories and devising better treatment strategies for prion-related diseases. Herein, we investigate a generic model for prion-like dynamics consisting of a class of non-linear ordinary differential equations (ODEs), establishing constraints through a linear stability analysis that enforce the expected properties of mammalian prion-like toxicity. Furthermore, we identify that prion toxicity evolves through three distinct phases for which we provide analytical descriptions using perturbation analyses. Specifically, prion-toxicity is initially characterised by the healthy phase, where the dynamics are dominated by the healthy form of prions, thereafter the system enters the mixed phase, where both healthy and toxic prions interact, and lastly, the system enters the toxic phase, where toxic prions dominate, and we refer to these phases as HeMiTo-dynamics. These findings hold the potential to aid researchers in developing precise mathematical models for prion-like dynamics, enabling them to better understand underlying mechanisms and devise effective treatments for prion-related diseases.
Meta results on data-driven control of nonlinear systems
Authors: Nima Monshizadeh, Claudio De Persis, Pietro Tesi
Date: 2024-05-16 00:00:00

This note aims to provide a systematic understanding of direct data-driven control, enriching the existing literature not by adding another isolated result, but rather by offering a comprehensive, versatile, and unifying framework that sets the stage for future explorations and applications in this domain. To this end, we formulate the nonlinear design problem from a high-level perspective as a set of desired controlled systems and propose systematic procedures to synthesize data-driven control algorithms that meet the design requirements specified in the desired set. Various examples are presented to demonstrate the comprehensiveness and adaptability of the proposed approach.
Enhancing Energy Efficiency in O-RAN Through Intelligent xApps Deployment
Authors: Xuanyu Liang, Ahmed Al-Tahmeesschi, Qiao Wang, Swarna Chetty, Chenrui Sun, Hamed Ahmadi
Date: 2024-05-16 00:00:00

The proliferation of 5G technology presents an unprecedented challenge in managing the energy consumption of densely deployed network infrastructures, particularly Base Stations (BSs), which account for the majority of power usage in mobile networks. The O-RAN architecture, with its emphasis on open and intelligent design, offers a promising framework to address the Energy Efficiency (EE) demands of modern telecommunication systems. This paper introduces two xApps designed for the O-RAN architecture to optimize power savings without compromising the Quality of Service (QoS). Utilizing a commercial RAN Intelligent Controller (RIC) simulator, we demonstrate the effectiveness of our proposed xApps through extensive simulations that reflect real-world operational conditions. Our results show a significant reduction in power consumption, achieving up to 50% power savings with a minimal number of User Equipments (UEs), by intelligently managing the operational state of Radio Cards (RCs), particularly through switching between active and sleep modes based on network resource block usage conditions.
Comprehensive Causal Machine Learning
Authors: Michael Lechner, Jana Mareckova
Date: 2024-05-16 00:00:00

Uncovering causal effects in multiple treatment setting at various levels of granularity provides substantial value to decision makers. Comprehensive machine learning approaches to causal effect estimation allow to use a single causal machine learning approach for estimation and inference of causal mean effects for all levels of granularity. Focusing on selection-on-observables, this paper compares three such approaches, the modified causal forest (mcf), the generalized random forest (grf), and double machine learning (dml). It also compares the theoretical properties of the approaches and provides proven theoretical guarantees for the mcf. The findings indicate that dml-based methods excel for average treatment effects at the population level (ATE) and group level (GATE) with few groups, when selection into treatment is not too strong. However, for finer causal heterogeneity, explicitly outcome-centred forest-based approaches are superior. The mcf has three additional benefits: (i) It is the most robust estimator in cases when dml-based approaches underperform because of substantial selection into treatment; (ii) it is the best estimator for GATEs when the number of groups gets larger; and (iii), it is the only estimator that is internally consistent, in the sense that low-dimensional causal ATEs and GATEs are obtained as aggregates of finer-grained causal parameters.
Harmonic and Interharmonic Detection in Power Systems Based on Fractal-Optimized Variational Mode Decomposition
Authors: Pei Yuhang, Yu Min, Yu Yan
Date: 2024-05-16 00:00:00

The proposed method introduces a parameter determination approach based on the minimum Fractal box dimension (FBD) of Variational Mode Decomposition (VMD) components, aiming to address the issue of manual determination of VMD decomposition layers in advance. Initially, VMD is applied to the original power signal, and the layer number for VMD decomposition is determined by selecting the K value associated with the smallest fractal box dimension among its components. Subsequently, several Intrinsic Mode Functions (IMFs) are obtained as fundamental, harmonic, and interharmonic signals representing different aspects of the power system. Furthermore, Hilbert transform(HT) is employed to extract instantaneous amplitude and frequency information from these harmonic signals. Experimental evaluation using simulation data and real-world power system data demonstrates that compared to Empirical Mode Decomposition (EMD) and Ensemble Empirical Mode Decomposition (EEMD), our proposed method achieves more accurate identification and effective extraction of harmonic signals.
Continuous Transfer Learning for UAV Communication-aware Trajectory Design
Authors: Chenrui Sun, Gianluca Fontanesi, Swarna Bindu Chetty, Xuanyu Liang, Berk Canberk, Hamed Ahmadi
Date: 2024-05-16 00:00:00
Tasks: Deep Reinforcement Learning, Trajectory Planning, Transfer Learning

Deep Reinforcement Learning (DRL) emerges as a prime solution for Unmanned Aerial Vehicle (UAV) trajectory planning, offering proficiency in navigating high-dimensional spaces, adaptability to dynamic environments, and making sequential decisions based on real-time feedback. Despite these advantages, the use of DRL for UAV trajectory planning requires significant retraining when the UAV is confronted with a new environment, resulting in wasted resources and time. Therefore, it is essential to develop techniques that can reduce the overhead of retraining DRL models, enabling them to adapt to constantly changing environments. This paper presents a novel method to reduce the need for extensive retraining using a double deep Q network (DDQN) model as a pretrained base, which is subsequently adapted to different urban environments through Continuous Transfer Learning (CTL). Our method involves transferring the learned model weights and adapting the learning parameters, including the learning and exploration rates, to suit each new environment specific characteristics. The effectiveness of our approach is validated in three scenarios, each with different levels of similarity. CTL significantly improves learning speed and success rates compared to DDQN models initiated from scratch. For similar environments, Transfer Learning (TL) improved stability, accelerated convergence by 65%, and facilitated 35% faster adaptation in dissimilar settings.
End-to-End Optimization of Directly Modulated Laser Links using Chirp-Aware Modeling
Authors: Sergio Hernandez F., Christophe Peucheret, Francesco Da Ros, Darko Zibar
Date: 2024-05-16 00:00:00

The rate and reach of directly-modulated laser links is often limited by the interplay between chirp and fiber chromatic dispersion. We address this by optimizing the transmitter, receiver, bias and peak-to-peak current to the laser jointly. Our approach outperforms Volterra post-equalization at various symbol rates.
Layer-Condensed KV Cache for Efficient Inference of Large Language Models
Authors: HaoYi Wu, Kewei Tu
Date: 2024-05-17 00:00:00
Tasks: Language Modeling, Language Modelling

Huge memory consumption has been a major bottleneck for deploying high-throughput large language models in real-world applications. In addition to the large number of parameters, the key-value (KV) cache for the attention mechanism in the transformer architecture consumes a significant amount of memory, especially when the number of layers is large for deep language models. In this paper, we propose a novel method that only computes and caches the KVs of a small number of layers, thus significantly saving memory consumption and improving inference throughput. Our experiments on large language models show that our method achieves up to 26$\times$ higher throughput than standard transformers and competitive performance in language modeling and downstream tasks. In addition, our method is orthogonal to existing transformer memory-saving techniques, so it is straightforward to integrate them with our model, achieving further improvement in inference efficiency. Our code is available at https://github.com/whyNLP/LCKV.
DINO as a von Mises-Fisher mixture model
Authors: Hariprasath Govindarajan, Per Sidén, Jacob Roll, Fredrik Lindsten
Date: 2024-05-17 00:00:00
Proceeding: ICLR 2023 2
Tasks: model

Self-distillation methods using Siamese networks are popular for self-supervised pre-training. DINO is one such method based on a cross-entropy loss between $K$-dimensional probability vectors, obtained by applying a softmax function to the dot product between representations and learnt prototypes. Given the fact that the learned representations are $L^2$-normalized, we show that DINO and its derivatives, such as iBOT, can be interpreted as a mixture model of von Mises-Fisher components. With this interpretation, DINO assumes equal precision for all components when the prototypes are also $L^2$-normalized. Using this insight we propose DINO-vMF, that adds appropriate normalization constants when computing the cluster assignment probabilities. Unlike DINO, DINO-vMF is stable also for the larger ViT-Base model with unnormalized prototypes. We show that the added flexibility of the mixture model is beneficial in terms of better image representations. The DINO-vMF pre-trained model consistently performs better than DINO on a range of downstream tasks. We obtain similar improvements for iBOT-vMF vs iBOT and thereby show the relevance of our proposed modification also for other methods derived from DINO.