pwc-archive/papers-with-abstracts huggingface dataset:
The role of debt valuation factors in systemic risk assessment
Authors: Kamil Fortuna, Janusz Szwabiński
Date: 2024-11-15 00:00:00

The fragility of financial systems was starkly demonstrated in early 2023 through a cascade of major bank failures in the United States, including the second, third, and fourth largest collapses in the US history. The highly interdependent financial networks and the associated high systemic risk have been deemed the cause of the crashes. The goal of this paper is to enhance existing systemic risk analysis frameworks by incorporating essential debt valuation factors. Our results demonstrate that these additional elements substantially influence the outcomes of risk assessment. Notably, by modeling the dynamic relationship between interest rates and banks' credibility, our framework can detect potential cascading failures that standard approaches might miss. The proposed risk assessment methodology can help regulatory bodies prevent future failures, while also allowing companies to more accurately predict turmoil periods and strengthen their survivability during such events.
Residual Multi-Task Learner for Applied Ranking
Authors: Cong Fu, Kun Wang, Jiahua Wu, Yizhou Chen, Guangda Huzhang, Yabo Ni, AnXiang Zeng, Zhiming Zhou
Date: 2024-10-30 00:00:00
Tasks: Multi-Task Learning

Modern e-commerce platforms rely heavily on modeling diverse user feedback to provide personalized services. Consequently, multi-task learning has become an integral part of their ranking systems. However, existing multi-task learning methods encounter two main challenges: some lack explicit modeling of task relationships, resulting in inferior performance, while others have limited applicability due to being computationally intensive, having scalability issues, or relying on strong assumptions. To address these limitations and better fit our real-world scenario, pre-rank in Shopee Search, we introduce in this paper ResFlow, a lightweight multi-task learning framework that enables efficient cross-task information sharing via residual connections between corresponding layers of task networks. Extensive experiments on datasets from various scenarios and modalities demonstrate its superior performance and adaptability over state-of-the-art methods. The online A/B tests in Shopee Search showcase its practical value in large-scale industrial applications, evidenced by a 1.29% increase in OPU (order-per-user) without additional system latency. ResFlow is now fully deployed in the pre-rank module of Shopee Search. To facilitate efficient online deployment, we propose a novel offline metric Weighted Recall@K, which aligns well with our online metric OPU, addressing the longstanding online-offline metric misalignment issue. Besides, we propose to fuse scores from the multiple tasks additively when ranking items, which outperforms traditional multiplicative fusion. The code is released at https://github.com/BrunoTruthAlliance/ResFlow
Decoding Fatigue Levels of Pilots Using EEG Signals with Hybrid Deep Neural Networks
Authors: Dae-Hyeok Lee, Sung-Jin Kim, Si-Hyun Kim
Date: 2024-10-30 00:00:00
Tasks: EEG

The detection of pilots' mental states is critical, as abnormal mental states have the potential to cause catastrophic accidents. This study demonstrates the feasibility of using deep learning techniques to classify different fatigue levels, specifically a normal state, low fatigue, and high fatigue. To the best of our knowledge, this is the first study to classify fatigue levels in pilots. Our approach employs the hybrid deep neural network comprising five convolutional blocks and one long short-term memory block to extract the significant features from electroencephalography signals. Ten pilots participated in the experiment, which was conducted in a simulated flight environment. Compared to four conventional models, our proposed model achieved a superior grand-average accuracy of 0.8801, outperforming other models by at least 0.0599 in classifying fatigue levels. In addition to successfully classifying fatigue levels, our model provided valuable feedback to subjects. Therefore, we anticipate that our study will make the significant contributions to the advancement of autonomous flight and driving technologies, leveraging artificial intelligence in the future.
Feature Selection via Dynamic Graph-based Attention Block in MI-based EEG Signals
Authors: Hyeon-Taek Han, Dae-Hyeok Lee, Heon-Gyu Kwak
Date: 2024-10-31 00:00:00
Tasks: Brain Computer Interface, EEG, Electroencephalogram (EEG), feature selection, Motor Imagery

Brain-computer interface (BCI) technology enables direct interaction between humans and computers by analyzing brain signals. Electroencephalogram (EEG) is one of the non-invasive tools used in BCI systems, providing high temporal resolution for real-time applications. However, EEG signals are often affected by a low signal-to-noise ratio, physiological artifacts, and individual variability, representing challenges in extracting distinct features. Also, motor imagery (MI)-based EEG signals could contain features with low correlation to MI characteristics, which might cause the weights of the deep model to become biased towards those features. To address these problems, we proposed the end-to-end deep preprocessing method that effectively enhances MI characteristics while attenuating features with low correlation to MI characteristics. The proposed method consisted of the temporal, spatial, graph, and similarity blocks to preprocess MI-based EEG signals, aiming to extract more discriminative features and improve the robustness. We evaluated the proposed method using the public dataset 2a of BCI Competition IV to compare the performances when integrating the proposed method into the conventional models, including the DeepConvNet, the M-ShallowConvNet, and the EEGNet. The experimental results showed that the proposed method could achieve the improved performances and lead to more clustered feature distributions of MI tasks. Hence, we demonstrated that our proposed method could enhance discriminative features related to MI characteristics.
AI-Driven Feedback Loops in Digital Technologies: Psychological Impacts on User Behaviour and Well-Being
Authors: Anthonette Adanyin
Date: 2024-10-30 00:00:00
Tasks: Descriptive

The rapid spread of digital technologies has produced data-driven feedback loops, wearable devices, social media networks, and mobile applications that shape user behavior, motivation, and mental well-being. While these systems encourage self-improvement and the development of healthier habits through real-time feedback, they also create psychological risks such as technostress, addiction, and loss of autonomy. The present study also aims to investigate the positive and negative psychological consequences of feedback mechanisms on users' behaviour and well-being. Employing a descriptive survey method, the study collected data from 200 purposely selected users to assess changes in behaviour, motivation, and mental well-being related to health, social, and lifestyle applications. Results indicate that while feedback mechanisms facilitate goal attainment and social interconnection through streaks and badges, among other components, they also enhance anxiety, mental weariness, and loss of productivity due to actions that are considered feedback-seeking. Furthermore, test subjects reported that their actions are unconsciously shaped by app feedback, often at the expense of personal autonomy, while real-time feedback minimally influences professional or social interactions. The study shows that data-driven feedback loops deliver not only motivational benefits but also psychological challenges. To mitigate these risks, users should establish boundaries regarding their use of technology to prevent burnout and addiction, while developers need to refine feedback mechanisms to reduce cognitive load and foster more inclusive participation. Future research should focus on designing feedback mechanisms that promote well-being without compromising individual freedom or increasing social comparison.
AI-Driven Human-Autonomy Teaming in Tactical Operations: Proposed Framework, Challenges, and Future Directions
Authors: Desta Haileselassie Hagos, Hassan El alami, Danda B. Rawat
Date: 2024-10-28 00:00:00
Tasks: Decision Making

Artificial Intelligence (AI) techniques, particularly machine learning techniques, are rapidly transforming tactical operations by augmenting human decision-making capabilities. This paper explores AI-driven Human-Autonomy Teaming (HAT) as a transformative approach, focusing on how it empowers human decision-making in complex environments. While trust and explainability continue to pose significant challenges, our exploration focuses on the potential of AI-driven HAT to transform tactical operations. By improving situational awareness and supporting more informed decision-making, AI-driven HAT can enhance the effectiveness and safety of such operations. To this end, we propose a comprehensive framework that addresses the key components of AI-driven HAT, including trust and transparency, optimal function allocation between humans and AI, situational awareness, and ethical considerations. The proposed framework can serve as a foundation for future research and development in the field. By identifying and discussing critical research challenges and knowledge gaps in this framework, our work aims to guide the advancement of AI-driven HAT for optimizing tactical operations. We emphasize the importance of developing scalable and ethical AI-driven HAT systems that ensure seamless human-machine collaboration, prioritize ethical considerations, enhance model transparency through Explainable AI (XAI) techniques, and effectively manage the cognitive load of human operators.
GWQ: Gradient-Aware Weight Quantization for Large Language Models
Authors: Yihua Shao, Siyu Liang, Zijian Ling, Minxi Yan, Haiyang Liu, Siyu Chen, Ziyang Yan, Chenyu Zhang, Haotong Qin, Michele Magno, Yang Yang, Zhen Lei, Yan Wang, Jingcai Guo, Ling Shao, Hao Tang
Date: 2024-10-30 00:00:00
Tasks: Outlier Detection, Quantization

Large language models (LLMs) show impressive performance in solving complex language tasks. However, its large number of parameters present significant challenges for the deployment and application of the model on edge devices. Compressing large language models to low bits can enable them to run on resource-constrained devices, often leading to performance degradation. To address this problem, we propose gradient-aware weight quantization (GWQ), the first quantization approach for low-bit weight quantization that leverages gradients to localize outliers, requiring only a minimal amount of calibration data for outlier detection. GWQ retains the weights corresponding to the top 1% outliers preferentially at FP16 precision, while the remaining non-outlier weights are stored in a low-bit format. GWQ found experimentally that utilizing the sensitive weights in the gradient localization model is more scientific compared to utilizing the sensitive weights in the Hessian matrix localization model. Compared to current quantization methods, GWQ can be applied to multiple language models and achieves lower PPL on the WikiText2 and C4 dataset. In the zero-shot task, GWQ quantized models have higher accuracy compared to other quantization methods. GWQ is also suitable for multimodal model quantization, and the quantized Qwen-VL family model is more accurate than other methods. Zero-shot target detection task dataset RefCOCO outperforms the current stat-of-the-arts method SPQR. GWQ achieves 1.2 times inference speedup in comparison to the original model, and effectively reduces the inference memory.
A Comprehensive Study on Quantization Techniques for Large Language Models
Authors: Jiedong Lang, Zhehao Guo, Shuyu Huang
Date: 2024-10-30 00:00:00
Tasks: Quantization

Large Language Models (LLMs) have been extensively researched and used in both academia and industry since the rise in popularity of the Transformer model, which demonstrates excellent performance in AI. However, the computational demands of LLMs are immense, and the energy resources required to run them are often limited. For instance, popular models like GPT-3, with 175 billion parameters and a storage requirement of 350 GB, present significant challenges for deployment on resource-constrained IoT devices and embedded systems. These systems often lack the computational capacity to handle such large models. Quantization, a technique that reduces the precision of model values to a smaller set of discrete values, offers a promising solution by reducing the size of LLMs and accelerating inference. In this research, we provide a comprehensive analysis of quantization techniques within the machine learning field, with a particular focus on their application to LLMs. We begin by exploring the mathematical theory of quantization, followed by a review of common quantization methods and how they are implemented. Furthermore, we examine several prominent quantization methods applied to LLMs, detailing their algorithms and performance outcomes.
Accelerated AI Inference via Dynamic Execution Methods
Authors: Haim Barad, Jascha Achterberg, Tien Pei Chou, Jean Yu
Date: 2024-10-30 00:00:00
Tasks: Quantization

In this paper, we focus on Dynamic Execution techniques that optimize the computation flow based on input. This aims to identify simpler problems that can be solved using fewer resources, similar to human cognition. The techniques discussed include early exit from deep networks, speculative sampling for language models, and adaptive steps for diffusion models. Experimental results demonstrate that these dynamic approaches can significantly improve latency and throughput without compromising quality. When combined with model-based optimizations, such as quantization, dynamic execution provides a powerful multi-pronged strategy to optimize AI inference. Generative AI requires a large amount of compute resources. This is expected to grow, and demand for resources in data centers through to the edge is expected to continue to increase at high rates. We take advantage of existing research and provide additional innovations for some generative optimizations. In the case of LLMs, we provide more efficient sampling methods that depend on the complexity of the data. In the case of diffusion model generation, we provide a new method that also leverages the difficulty of the input prompt to predict an optimal early stopping point. Therefore, dynamic execution methods are relevant because they add another dimension of performance optimizations. Performance is critical from a competitive point of view, but increasing capacity can result in significant power savings and cost savings. We have provided several integrations of these techniques into several Intel performance libraries and Huggingface Optimum. These integrations will make them easier to use and increase the adoption of these techniques.
LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators
Authors: Krishna Teja Chitty-Venkata, Siddhisanket Raskar, Bharat Kale, Farah Ferdaus, Aditya Tanikanti, Ken Raffenetti, Valerie Taylor, Murali Emani, Venkatram Vishwanath
Date: 2024-10-31 00:00:00
Tasks: Benchmarking, Text Generation

Large Language Models (LLMs) have propelled groundbreaking advancements across several domains and are commonly used for text generation applications. However, the computational demands of these complex models pose significant challenges, requiring efficient hardware acceleration. Benchmarking the performance of LLMs across diverse hardware platforms is crucial to understanding their scalability and throughput characteristics. We introduce LLM-Inference-Bench, a comprehensive benchmarking suite to evaluate the hardware inference performance of LLMs. We thoroughly analyze diverse hardware platforms, including GPUs from Nvidia and AMD and specialized AI accelerators, Intel Habana and SambaNova. Our evaluation includes several LLM inference frameworks and models from LLaMA, Mistral, and Qwen families with 7B and 70B parameters. Our benchmarking results reveal the strengths and limitations of various models, hardware platforms, and inference frameworks. We provide an interactive dashboard to help identify configurations for optimal performance for a given hardware platform.
Kernelized Cumulants: Beyond Kernel Mean Embeddings
Authors: Patric Bonnier, Harald Oberhauser, Zoltán Szabó
Date: 2023-01-29 00:00:00
Proceeding: NeurIPS 2023 11

In $\mathbb R^d$, it is well-known that cumulants provide an alternative to moments that can achieve the same goals with numerous benefits such as lower variance estimators. In this paper we extend cumulants to reproducing kernel Hilbert spaces (RKHS) using tools from tensor algebras and show that they are computationally tractable by a kernel trick. These kernelized cumulants provide a new set of all-purpose statistics; the classical maximum mean discrepancy and Hilbert-Schmidt independence criterion arise as the degree one objects in our general construction. We argue both theoretically and empirically (on synthetic, environmental, and traffic data analysis) that going beyond degree one has several advantages and can be achieved with the same computational complexity and minimal overhead in our experiments.
Improving Fraud Detection with 1D-Convolutional Spiking Neural Networks Through Bayesian Optimization
Authors: Dylan Perdigão, Francisco Antunes, Catarina Silva, Bernardete Ribeiro
Date: 2024-11-16 00:00:00
Proceeding: EPIA Conference on Artificial Intelligence 2024 11
Tasks: Bayesian Optimization, Fairness, Fraud Detection

The digitalization of the banking sector has enabled an increasing number of fraudulent activities in the past years. The development of new practical solutions for fraud detection is particularly challenging since the industry needs to respect a 5% threshold of false detection of fraud, avoiding monetary losses. The usage of traditional machine learning algorithms faces other challenges, such as classification discrimination and high energy consumption. Spiking Neural Networks, designed to mimic the brain’s natural processes, present a compelling alternative. Particularly on neuromorphic hardware, they exhibit remarkable energy efficiency, paving the way for the advancement of sustainable Artificial Intelligence. Our approach consists of applying 1D-Convolutional Spiking Neural Networks to improve fraud detection on the Bank Account Fraud dataset with the Bayesian optimization of hyperparameters. As a result, we show that these architectures can solve the problem of biased data while maintaining algorithm performance when compared against more traditional approaches such as Gradient Boosting models. Moreover, we lay out the Bayesian optimization process for the hyperparameters, thereby enhancing the performance of these algorithms.
Exploring Neural Joint Activity in Spiking Neural Networks for Fraud Detection
Authors: Dylan Perdigão, Francisco Antunes, Catarina Silva, Bernardete Ribeiro
Date: 2024-11-17 00:00:00
Proceeding: Iberoamerican Congress on Pattern Recognition 2024 11
Tasks: Bayesian Optimization, Fairness, Fraud Detection

Spiking Neural Networks (SNNs), inspired by the real brain’s behavior, offer an energy-efficient alternative to traditional artificial neural networks coupled with their neural joint activity, also referred to as population coding. This population coding is replicated in SNNs by attributing more than one neuron to each class in the output layer. This study leverages SNNs for fraud detection through real-world datasets, namely the Bank Account Fraud dataset suite, addressing the fairness and bias issues inherent in conventional machine learning algorithms. Different configurations of time steps and population sizes were compared within a 1D-Convolutional Spiking Neural Network, whose hyperparameters were optimized through a Bayesian optimization process. Our proposed SNN approach with neural joint activity enables the classification of fraudulent opening of bank accounts more accurately and fairly than standard SNNs. The results highlight the potential of SNNs to surpass non-population coding baselines by achieving an average of 47.08% of recall at a business constraint of 5% of false positive rate, offering a robust solution for fraud detection. Moreover, the proposed approach attains comparable results to gradient-boosting machine models while maintaining predictive equality towards sensitive attributes above 90%.
Question Answering over Linked Data with GPT-3
Authors: Bruno Faria, Dylan Perdigão, Hugo Gonçalo Oliveira
Date: 2023-08-15 00:00:00
Proceeding: Symposium on Languages, Applications and Technologies 2023 8
Tasks: Knowledge Base Question Answering, Question Answering

This paper explores GPT-3 for answering natural language questions over Linked Data. Different engines of the model and different approaches are adopted for answering questions in the QALD-9 dataset, namely: zero and few-shot SPARQL generation, as well as fine-tuning in the training portion of the dataset. Answers retrieved by the generated queries and answers generated directly by the model are also compared. Overall results are generally poor, but several insights are provided on using GPT-3 for the proposed task.
Vision Eagle Attention: a new lens for advancing image classification
Authors: Mahmudul Hasan
Date: 2024-11-15 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection, Visual Tracking

In computer vision tasks, the ability to focus on relevant regions within an image is crucial for improving model performance, particularly when key features are small, subtle, or spatially dispersed. Convolutional neural networks (CNNs) typically treat all regions of an image equally, which can lead to inefficient feature extraction. To address this challenge, I have introduced Vision Eagle Attention, a novel attention mechanism that enhances visual feature extraction using convolutional spatial attention. The model applies convolution to capture local spatial features and generates an attention map that selectively emphasizes the most informative regions of the image. This attention mechanism enables the model to focus on discriminative features while suppressing irrelevant background information. I have integrated Vision Eagle Attention into a lightweight ResNet-18 architecture, demonstrating that this combination results in an efficient and powerful model. I have evaluated the performance of the proposed model on three widely used benchmark datasets: FashionMNIST, Intel Image Classification, and OracleMNIST, with a primary focus on image classification. Experimental results show that the proposed approach improves classification accuracy. Additionally, this method has the potential to be extended to other vision tasks, such as object detection, segmentation, and visual tracking, offering a computationally efficient solution for a wide range of vision-based applications. Code is available at: https://github.com/MahmudulHasan11085/Vision-Eagle-Attention.git