pwc-archive/papers-with-abstracts huggingface dataset:
Application and practice of AI technology in quantitative investment
Authors: Shuochen Bi, Wenqing Bao, Jue Xiao, Jiangshan Wang, Tingting Deng
Date: 2024-04-28 00:00:00
Tasks: Decision Making

With the continuous development of artificial intelligence technology, using machine learning technology to predict market trends may no longer be out of reach. In recent years, artificial intelligence has become a research hotspot in the academic circle,and it has been widely used in image recognition, natural language processing and other fields, and also has a huge impact on the field of quantitative investment. As an investment method to obtain stable returns through data analysis, model construction and program trading, quantitative investment is deeply loved by financial institutions and investors. At the same time, as an important application field of quantitative investment, the quantitative investment strategy based on artificial intelligence technology arises at the historic moment.How to apply artificial intelligence to quantitative investment, so as to better achieve profit and risk control, has also become the focus and difficulty of the research. From a global perspective, inflation in the US and the Federal Reserve are the concerns of investors, which to some extent affects the direction of global assets, including the Chinese stock market. This paper studies the application of AI technology, quantitative investment, and AI technology in quantitative investment, aiming to provide investors with auxiliary decision-making, reduce the difficulty of investment analysis, and help them to obtain higher returns.
Value-at-Risk- and Expectile-based Systemic Risk Measures and Second-order Asymptotics: With Applications to Diversification
Authors: Bingzhen Geng, Yang Liu, Yimiao Zhao
Date: 2024-04-27 00:00:00

The systemic risk measure plays a crucial role in analyzing individual losses conditioned on extreme system-wide disasters. In this paper, we provide a unified asymptotic treatment for systemic risk measures. First, we classify them into two families of Value-at-Risk- (VaR-) and expectile-based systemic risk measures. While VaR has been extensively studied, in the latter family, we propose two new systemic risk measures named the Individual Conditional Expectile (ICE) and the Systemic Individual Conditional Expectile (SICE), as alternatives to Marginal Expected Shortfall (MES) and Systemic Expected Shortfall (SES). Second, to characterize general mutually dependent and heavy-tailed risks, we adopt a modeling framework where the system, represented by a vector of random loss variables, follows a multivariate Sarmanov distribution with a common marginal exhibiting second-order regular variation. Third, we provide second-order asymptotic results for both families of systemic risk measures. This analytical framework offers a more accurate estimate compared to traditional first-order asymptotics. Through numerical and analytical examples, we demonstrate the superiority of second-order asymptotics in accurately assessing systemic risk. Further, we conduct a comprehensive comparison between VaR-based and expectile-based systemic risk measures. Expectile-based measures output higher risk evaluation than VaR-based ones, emphasizing the former's potential advantages in reporting extreme events and tail risk. As a financial application, we use the asymptotic treatment to discuss the diversification benefits associated with systemic risk measures. The expectile-based diversification benefits consistently deduce an underestimation and suggest a conservative approximation, while the VaR-based diversification benefits consistently deduce an overestimation and suggest behaving optimistically.
Learning Low-Rank Feature for Thorax Disease Classification
Authors: Rajeev Goel, Utkarsh Nath, Yancheng Wang, Alvin C. Silva, Teresa Wu, Yingzhen Yang
Date: 2024-02-14 00:00:00
Tasks: Classification, Self-Supervised Learning

Deep neural networks, including Convolutional Neural Networks (CNNs) and Visual Transformers (ViT), have achieved stunning success in medical image domain. We study thorax disease classification in this paper. Effective extraction of features for the disease areas is crucial for disease classification on radiographic images. While various neural architectures and training techniques, such as self-supervised learning with contrastive/restorative learning, have been employed for disease classification on radiographic images, there are no principled methods which can effectively reduce the adverse effect of noise and background, or non-disease areas, on the radiographic images for disease classification. To address this challenge, we propose a novel Low-Rank Feature Learning (LRFL) method in this paper, which is universally applicable to the training of all neural networks. The LRFL method is both empirically motivated by the low frequency property observed on all the medical datasets in this paper, and theoretically motivated by our sharp generalization bound for neural networks with low-rank features. In the empirical study, using a neural network such as a ViT or a CNN pre-trained on unlabeled chest X-rays by Masked Autoencoders (MAE), our novel LRFL method is applied on the pre-trained neural network and demonstrate better classification results in terms of both multiclass area under the receiver operating curve (mAUC) and classification accuracy.
The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video
Authors: Michelle R. Greene, Benjamin J. Balas, Mark D. Lescroart, Paul R. MacNeilage, Jennifer A. Hart, Kamran Binaee, Peter A. Hausamann, Ronald Mezile, Bharath Shankar, Christian B. Sinnott, Kaylie Capurro, Savannah Halow, Hunter Howe, Mariam Josyula, Annie Li, Abraham Mieses, Amina Mohamed, Ilya Nudnou, Ezra Parkhill, Peter Riley, Brett Schmidt, Matthew W. Shinkle, Wentao Si, Brian Szekely, Joaquin M. Torres, Eliana Weissmann
Date: 2024-02-15 00:00:00
Tasks: Activity Recognition

We introduce the Visual Experience Dataset (VEDB), a compilation of over 240 hours of egocentric video combined with gaze- and head-tracking data that offers an unprecedented view of the visual world as experienced by human observers. The dataset consists of 717 sessions, recorded by 58 observers ranging from 6-49 years old. This paper outlines the data collection, processing, and labeling protocols undertaken to ensure a representative sample and discusses the potential sources of error or bias within the dataset. The VEDB's potential applications are vast, including improving gaze tracking methodologies, assessing spatiotemporal image statistics, and refining deep neural networks for scene and activity recognition. The VEDB is accessible through established open science platforms and is intended to be a living dataset with plans for expansion and community contributions. It is released with an emphasis on ethical considerations, such as participant privacy and the mitigation of potential biases. By providing a dataset grounded in real-world experiences and accompanied by extensive metadata and supporting code, the authors invite the research community to utilize and contribute to the VEDB, facilitating a richer understanding of visual perception and behavior in naturalistic settings.
What's in the Flow? Exploiting Temporal Motion Cues for Unsupervised Generic Event Boundary Detection
Authors: Sourabh Vasant Gothe, Vibhav Agarwal, Sourav Ghosh, Jayesh Rajkumar Vachhani, Pranay Kashyap, Barath Raj Kandur Raja
Date: 2024-02-15 00:00:00
Proceeding: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2024 1
Tasks: Boundary Detection, Generic Event Boundary Detection, Optical Flow Estimation

Generic Event Boundary Detection (GEBD) task aims to recognize generic, taxonomy-free boundaries that segment a video into meaningful events. Current methods typically involve a neural model trained on a large volume of data, demanding substantial computational power and storage space. We explore two pivotal questions pertaining to GEBD: Can non-parametric algorithms outperform unsupervised neural methods? Does motion information alone suffice for high performance? This inquiry drives us to algorithmically harness motion cues for identifying generic event boundaries in videos. In this work, we propose FlowGEBD, a non-parametric, unsupervised technique for GEBD. Our approach entails two algorithms utilizing optical flow: (i) Pixel Tracking and (ii) Flow Normalization. By conducting thorough experimentation on the challenging Kinetics-GEBD and TAPOS datasets, our results establish FlowGEBD as the new state-of-the-art (SOTA) among unsupervised methods. FlowGEBD exceeds the neural models on the Kinetics-GEBD dataset by obtaining an F1@0.05 score of 0.713 with an absolute gain of 31.7% compared to the unsupervised baseline and achieves an average F1 score of 0.623 on the TAPOS validation dataset.
Dynamic Model Switching for Improved Accuracy in Machine Learning
Authors: Syed Tahir Abbas Hasani
Date: 2024-01-31 00:00:00

In the dynamic landscape of machine learning, where datasets vary widely in size and complexity, selecting the most effective model poses a significant challenge. Rather than fixating on a single model, our research propels the field forward with a novel emphasis on dynamic model switching. This paradigm shift allows us to harness the inherent strengths of different models based on the evolving size of the dataset. Consider the scenario where CatBoost demonstrates exceptional efficacy in handling smaller datasets, providing nuanced insights and accurate predictions. However, as datasets grow in size and intricacy, XGBoost, with its scalability and robustness, becomes the preferred choice. Our approach introduces an adaptive ensemble that intuitively transitions between CatBoost and XGBoost. This seamless switching is not arbitrary; instead, it's guided by a user-defined accuracy threshold, ensuring a meticulous balance between model sophistication and data requirements. The user sets a benchmark, say 80% accuracy, prompting the system to dynamically shift to the new model only if it guarantees improved performance. This dynamic model-switching mechanism aligns with the evolving nature of data in real-world scenarios. It offers practitioners a flexible and efficient solution, catering to diverse dataset sizes and optimising predictive accuracy at every juncture. Our research, therefore, stands at the forefront of innovation, redefining how machine learning models adapt and excel in the face of varying dataset dynamics.
Enhancing Predictive Accuracy in Pharmaceutical Sales Through An Ensemble Kernel Gaussian Process Regression Approach
Authors: Shahin Mirshekari, Mohammadreza Moradi, Hossein Jafari, Mehdi Jafari, Mohammad Ensaf
Date: 2024-04-15 00:00:00
Tasks: Bayesian Optimization, GPR

This research employs Gaussian Process Regression (GPR) with an ensemble kernel, integrating Exponential Squared, Revised Mat\'ern, and Rational Quadratic kernels to analyze pharmaceutical sales data. Bayesian optimization was used to identify optimal kernel weights: 0.76 for Exponential Squared, 0.21 for Revised Mat\'ern, and 0.13 for Rational Quadratic. The ensemble kernel demonstrated superior performance in predictive accuracy, achieving an \( R^2 \) score near 1.0, and significantly lower values in Mean Squared Error (MSE), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE). These findings highlight the efficacy of ensemble kernels in GPR for predictive analytics in complex pharmaceutical sales datasets.
Multidimensional Compressed Sensing for Spectral Light Field Imaging
Authors: Wen Cao, Ehsan Miandji, Jonas Unger
Date: 2024-02-27 00:00:00
Tasks: compressed sensing

This paper considers a compressive multi-spectral light field camera model that utilizes a one-hot spectralcoded mask and a microlens array to capture spatial, angular, and spectral information using a single monochrome sensor. We propose a model that employs compressed sensing techniques to reconstruct the complete multi-spectral light field from undersampled measurements. Unlike previous work where a light field is vectorized to a 1D signal, our method employs a 5D basis and a novel 5D measurement model, hence, matching the intrinsic dimensionality of multispectral light fields. We mathematically and empirically show the equivalence of 5D and 1D sensing models, and most importantly that the 5D framework achieves orders of magnitude faster reconstruction while requiring a small fraction of the memory. Moreover, our new multidimensional sensing model opens new research directions for designing efficient visual data acquisition algorithms and hardware.
Expanding the Horizon: Enabling Hybrid Quantum Transfer Learning for Long-Tailed Chest X-Ray Classification
Authors: Skylar Chan, Pranav Kulkarni, Paul H. Yi, Vishwa S. Parekh
Date: 2024-04-30 00:00:00
Tasks: Binary Classification, Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Quantum Machine Learning, Transfer Learning, X-ray Classification

Quantum machine learning (QML) has the potential for improving the multi-label classification of rare, albeit critical, diseases in large-scale chest x-ray (CXR) datasets due to theoretical quantum advantages over classical machine learning (CML) in sample efficiency and generalizability. While prior literature has explored QML with CXRs, it has focused on binary classification tasks with small datasets due to limited access to quantum hardware and computationally expensive simulations. To that end, we implemented a Jax-based framework that enables the simulation of medium-sized qubit architectures with significant improvements in wall-clock time over current software offerings. We evaluated the performance of our Jax-based framework in terms of efficiency and performance for hybrid quantum transfer learning for long-tailed classification across 8, 14, and 19 disease labels using large-scale CXR datasets. The Jax-based framework resulted in up to a 58% and 95% speed-up compared to PyTorch and TensorFlow implementations, respectively. However, compared to CML, QML demonstrated slower convergence and an average AUROC of 0.70, 0.73, and 0.74 for the classification of 8, 14, and 19 CXR disease labels. In comparison, the CML models had an average AUROC of 0.77, 0.78, and 0.80 respectively. In conclusion, our work presents an accessible implementation of hybrid quantum transfer learning for long-tailed CXR classification with a computationally efficient Jax-based framework.
Gradient-based Automatic Mixed Precision Quantization for Neural Networks On-Chip
Authors: Chang Sun, Thea K. Årrestad, Vladimir Loncar, Jennifer Ngadiuba, Maria Spiropulu
Date: 2024-05-01 00:00:00
Tasks: Jet Tagging, Quantization

Model size and inference speed at deployment time, are major challenges in many deep learning applications. A promising strategy to overcome these challenges is quantization. However, a straightforward uniform quantization to very low precision can result in significant accuracy loss. Mixed-precision quantization, based on the idea that certain parts of the network can accommodate lower precision without compromising performance compared to other parts, offers a potential solution. In this work, we present High Granularity Quantization (HGQ), an innovative quantization-aware training method that could fine-tune the per-weight and per-activation precision by making them optimizable through gradient descent. This approach enables ultra-low latency and low power neural networks on hardware capable of performing arithmetic operations with an arbitrary number of bits, such as FPGAs and ASICs. We demonstrate that HGQ can outperform existing methods by a substantial margin, achieving resource reduction by up to a factor of 20 and latency improvement by a factor of 5 while preserving accuracy.
Self-Play Preference Optimization for Language Model Alignment
Authors: Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, Quanquan Gu
Date: 2024-05-01 00:00:00
Tasks: Language Modeling, Language Modelling, model

Standard reinforcement learning from human feedback (RLHF) approaches relying on parametric models like the Bradley-Terry model fall short in capturing the intransitivity and irrationality in human preferences. Recent advancements suggest that directly working with preference probabilities can yield a more accurate reflection of human preferences, enabling more flexible and accurate language model alignment. In this paper, we propose a self-play-based method for language model alignment, which treats the problem as a constant-sum two-player game aimed at identifying the Nash equilibrium policy. Our approach, dubbed Self-Play Preference Optimization (SPPO), utilizes iterative policy updates to provably approximate the Nash equilibrium. Additionally, we propose a new SPPO objective which is both strongly motivated by theory and is simple and effective in practice. In our experiments, using only 60k prompts (without responses) from the UltraFeedback dataset and without any prompt augmentation, by leveraging a pre-trained preference model PairRM with only 0.4B parameters, SPPO can obtain a model from fine-tuning Mistral-7B-Instruct-v0.2 that achieves the state-of-the-art length-controlled win-rate of 28.53% against GPT-4-Turbo on AlpacaEval 2.0. It also outperforms the (iterative) DPO and IPO on MT-Bench, Arena-Hard, and the Open LLM Leaderboard. Starting from a stronger base model Llama-3-8B-Instruct, we are able to achieve a length-controlled win rate of 38.77%. Notably, the strong performance of SPPO is achieved without additional external supervision (e.g., responses, preferences, etc.) from GPT-4 or other stronger language models. Codes are available at https://github.com/uclaml/SPPO.
Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3
Authors: Junsang Yoon, Akshat Gupta, Gopala Anumanchipalli
Date: 2024-05-01 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Model Editing

This study presents a targeted model editing analysis focused on the latest large language model, Llama-3. We explore the efficacy of popular model editing techniques - ROME, MEMIT, and EMMET, which are designed for precise layer interventions. We identify the most effective layers for targeted edits through an evaluation that encompasses up to 4096 edits across three distinct strategies: sequential editing, batch editing, and a hybrid approach we call as sequential-batch editing. Our findings indicate that increasing edit batch-sizes may degrade model performance more significantly than using smaller edit batches sequentially for equal number of edits. With this, we argue that sequential model editing is an important component for scaling model editing methods and future research should focus on methods that combine both batched and sequential editing. This observation suggests a potential limitation in current model editing methods which push towards bigger edit batch sizes, and we hope it paves way for future investigations into optimizing batch sizes and model editing performance.
RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization
Authors: Dongqi Liu, Vera Demberg
Date: 2024-05-01 00:00:00
Tasks: Abstractive Text Summarization, Document Summarization, parameter-efficient fine-tuning

For long document summarization, discourse structure is important to discern the key content of the text and the differences in importance level between sentences. Unfortunately, the integration of rhetorical structure theory (RST) into parameter-efficient fine-tuning strategies for long document summarization remains unexplored. Therefore, this paper introduces RST-LoRA and proposes four RST-aware variants to explicitly incorporate RST into the LoRA model. Our empirical evaluation demonstrates that incorporating the type and uncertainty of rhetorical relations can complementarily enhance the performance of LoRA in summarization tasks. Furthermore, the best-performing variant we introduced outperforms the vanilla LoRA and full-parameter fine-tuning models, as confirmed by multiple automatic and human evaluations, and even surpasses previous state-of-the-art methods.
ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints
Authors: Robert J. Moss, Arec Jamgochian, Johannes Fischer, Anthony Corso, Mykel J. Kochenderfer
Date: 2024-05-01 00:00:00
Tasks: Collision Avoidance

To plan safely in uncertain environments, agents must balance utility with safety constraints. Safe planning problems can be modeled as a chance-constrained partially observable Markov decision process (CC-POMDP) and solutions often use expensive rollouts or heuristics to estimate the optimal value and action-selection policy. This work introduces the ConstrainedZero policy iteration algorithm that solves CC-POMDPs in belief space by learning neural network approximations of the optimal value and policy with an additional network head that estimates the failure probability given a belief. This failure probability guides safe action selection during online Monte Carlo tree search (MCTS). To avoid overemphasizing search based on the failure estimates, we introduce $\Delta$-MCTS, which uses adaptive conformal inference to update the failure threshold during planning. The approach is tested on a safety-critical POMDP benchmark, an aircraft collision avoidance system, and the sustainability problem of safe CO$_2$ storage. Results show that by separating safety constraints from the objective we can achieve a target level of safety without optimizing the balance between rewards and costs.
When Quantization Affects Confidence of Large Language Models?
Authors: Irina Proskurina, Luc Brun, Guillaume Metzler, Julien Velcin
Date: 2024-05-01 00:00:00
Tasks: Language Modeling, Language Modelling, Quantization

Recent studies introduced effective compression techniques for Large Language Models (LLMs) via post-training quantization or low-bit weight representation. Although quantized weights offer storage efficiency and allow for faster inference, existing works have indicated that quantization might compromise performance and exacerbate biases in LLMs. This study investigates the confidence and calibration of quantized models, considering factors such as language model type and scale as contributors to quantization loss. Firstly, we reveal that quantization with GPTQ to 4-bit results in a decrease in confidence regarding true labels, with varying impacts observed among different language models. Secondly, we observe fluctuations in the impact on confidence across different scales. Finally, we propose an explanation for quantization loss based on confidence levels, indicating that quantization disproportionately affects samples where the full model exhibited low confidence levels in the first place.
HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach
Authors: Malte Lehna, Clara Holzhüter, Sven Tomforde, Christoph Scholz
Date: 2024-05-01 00:00:00
Tasks: Deep Reinforcement Learning

With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly shown significant potential in Learning to Run a Power Network (L2RPN) challenges. However, only individual actions at the substation level have been subjected to topology optimization by most existing DRL algorithms. In contrast, we propose a more holistic approach by proposing specific Target Topologies (TTs) as actions. These topologies are selected based on their robustness. As part of this paper, we present a search algorithm to find the TTs and upgrade our previously developed DRL agent CurriculumAgent (CAgent) to a novel topology agent. We compare the upgrade to the previous CAgent and can increase their L2RPN score significantly by 10%. Further, we achieve a 25% better median survival time with our TTs included. Later analysis shows that almost all TTs are close to the base topology, explaining their robustness