pwc-archive/papers-with-abstracts huggingface dataset:
Mack's estimator motivated by large exposure asymptotics in a compound Poisson setting
Authors: Nils Engler, Filip Lindskog
Date: 2023-10-18 00:00:00
Tasks: Prediction

The distribution-free chain ladder of Mack justified the use of the chain ladder predictor and enabled Mack to derive an estimator of conditional mean squared error of prediction for the chain ladder predictor. Classical insurance loss models, i.e. of compound Poisson type, are not consistent with Mack's distribution-free chain ladder. However, for a sequence of compound Poisson loss models indexed by exposure (e.g. number of contracts), we show that the chain ladder predictor and Mack's estimator of conditional mean squared error of prediction can be derived by considering large exposure asymptotics. Hence, quantifying chain ladder prediction uncertainty can be done with Mack's estimator without relying on the validity of the model assumptions of the distribution-free chain ladder.
Mean-field Libor market model and valuation of long term guarantees
Authors: Florian Gach, Simon Hochgerner, Eva Kienbacher, Gabriel Schachinger
Date: 2023-10-13 00:00:00
Tasks: Management

Existence and uniqueness of solutions to the multi-dimensional mean-field Libor market model (introduced by [7]) is shown. This is used as the basis for a numerical asset-liability management (ALM) model capable of calculating future discretionary benefits in accordance with Solvency~II regulation. This ALM model is complimented with aggregated life insurance data to perform a realistic numerical study. This yields numerical evidence for heuristic assumptions which allow to derive estimators of lower and upper bounds for future discretionary benefits. These estimators are applied to publicly available life insurance data.
On the impact of insurance on households susceptible to random proportional losses: An analysis of poverty trapping
Authors: Kira Henshaw, Jorge M. Ramirez, José M. Flores-Contró, Enrique A. Thomann, Sooie-Hoe Loke, Corina Constantinescu
Date: 2023-09-22 00:00:00

In this paper, we consider a risk process with deterministic growth and multiplicative jumps to model the capital of a low-income household. Reflecting the high-risk nature of the low-income environment, capital losses are assumed to be proportional to the level of accumulated capital at the jump time. Our aim is to derive the probability that a household falls below the poverty line, i.e. the trapping probability, where ``trapping" occurs when the level of capital of a household holds falls below the poverty line, to an area from which it is difficult to escape without external help. Considering the remaining proportion of capital to be distributed as a special case of the beta distribution, closed-form expressions for the trapping probability are obtained via analysis of the Laplace transform of the infinitesimal generator of the process. To study the impact of insurance on this probability, introduction of an insurance product offering proportional coverage is presented. The infinitesimal generator of the insured process gives rise to non-local differential equations. To overcome this, we propose a recursive method for deriving a closed-form solution of the integro-differential equation associated with the infinitesimal generator of the insured process and provide a numerical estimation method for obtaining the trapping probability. Constraints on the rate parameters of the process that prevent certain trapping are derived in both the uninsured and insured cases using classical results from risk theory.
BaitBuster-Bangla: A Comprehensive Dataset for Clickbait Detection in Bangla with Multi-Feature and Multi-Modal Analysis
Authors: Abdullah Al Imran, Md Sakib Hossain Shovon, M. F. Mridha
Date: 2023-10-13 00:00:00
Tasks: Classification, Clickbait Detection, Multilingual NLP

This study presents a large multi-modal Bangla YouTube clickbait dataset consisting of 253,070 data points collected through an automated process using the YouTube API and Python web automation frameworks. The dataset contains 18 diverse features categorized into metadata, primary content, engagement statistics, and labels for individual videos from 58 Bangla YouTube channels. A rigorous preprocessing step has been applied to denoise, deduplicate, and remove bias from the features, ensuring unbiased and reliable analysis. As the largest and most robust clickbait corpus in Bangla to date, this dataset provides significant value for natural language processing and data science researchers seeking to advance modeling of clickbait phenomena in low-resource languages. Its multi-modal nature allows for comprehensive analyses of clickbait across content, user interactions, and linguistic dimensions to develop more sophisticated detection methods with cross-linguistic applications.
Classic machine learning methods
Authors: Johann Faouzi, Olivier Colliot
Date: 2023-05-24 00:00:00
Tasks: Clustering, Dimensionality Reduction, regression

In this chapter, we present the main classic machine learning methods. A large part of the chapter is devoted to supervised learning techniques for classification and regression, including nearest-neighbor methods, linear and logistic regressions, support vector machines and tree-based algorithms. We also describe the problem of overfitting as well as strategies to overcome it. We finally provide a brief overview of unsupervised learning methods, namely for clustering and dimensionality reduction.
GPT-4 as an interface between researchers and computational software: improving usability and reproducibility
Authors: Juan C. Verduzco, Ethan Holbrook, Alejandro Strachan
Date: 2023-10-04 00:00:00
Tasks: Code Generation

Large language models (LLMs) are playing an increasingly important role in science and engineering. For example, their ability to parse and understand human and computer languages makes them powerful interpreters and their use in applications like code generation are well-documented. We explore the ability of the GPT-4 LLM to ameliorate two major challenges in computational materials science: i) the high barriers for adoption of scientific software associated with the use of custom input languages, and ii) the poor reproducibility of published results due to insufficient details in the description of simulation methods. We focus on a widely used software for molecular dynamics simulations, the Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS), and quantify the usefulness of input files generated by GPT-4 from task descriptions in English and its ability to generate detailed descriptions of computational tasks from input files. We find that GPT-4 can generate correct and ready-to-use input files for relatively simple tasks and useful starting points for more complex, multi-step simulations. In addition, GPT-4's description of computational tasks from input files can be tuned from a detailed set of step-by-step instructions to a summary description appropriate for publications. Our results show that GPT-4 can reduce the number of routine tasks performed by researchers, accelerate the training of new users, and enhance reproducibility.
Elastic deep autoencoder for text embedding clustering by an improved graph regularization
Authors: Fatemeh Daneshfar, Sayvan Soleymanbaigi, Ali Nafisi, Pedram Yamini
Date: 2023-09-23 00:00:00
Proceeding: Expert System with application journal 2023 9
Tasks: Clustering, Dimensionality Reduction, Recommendation Systems, Sentiment Analysis, Text Clustering

Text clustering is a task for grouping extracted information of the text in different clusters, which has many applications in recommender systems, sentiment analysis, and more. Deep learning-based methods have become increasingly popular due to their high accuracy in identifying nonlinear structures. They usually consist of two major parts: dimensionality reduction and clustering. Autoencoders are simple unsupervised neural networks used for better representation of low-dimensional data and have shown good performance in dealing with non-linear features. However, while they utilize the Frobenius norm to deal well with Gaussian noise, they are sensitive to outlier data and Laplacian noise. In this paper, a deep autoencoder with an adapted elastic loss for text embedding clustering (EDA-TEC) is proposed. The elastic loss is a combination of the Frobenius norm and L2,1-norm to consider both types of noises.  Additionally, to maintain the high-dimensional data geometric structure, a modified graph regularization term based on the weighted cosine similarity measure is used. EDA-TEC also improves clustering results by considering the sparsity regularization of the manifold representation data. In this jointly end-to-end deep learning model, better representation and text clustering results are achieved with high accuracy on common datasets compared to existing methods.
Frequency Regularization: Reducing Information Redundancy in Convolutional Neural Networks
Authors: Chenqiu Zhao, Guanfang Dong, Shupei Zhang, Zijie Tan, Anup Basu
Date: 2023-09-25 00:00:00
Proceeding: IEEE Access 2023 9

Convolutional neural networks have demonstrated impressive results in many computer vision tasks. However, the increasing size of these networks raises concerns about the information overload resulting from the large number of network parameters. In this paper, we propose Frequency Regularization to restrict the non-zero elements of the network parameters in the frequency domain. The proposed approach operates at the tensor level, and can be applied to almost all network architectures. Specifically, the tensors of parameters are maintained in the frequency domain, where high-frequency components can be eliminated by zigzag setting tensor elements to zero. Then, the inverse discrete cosine transform (IDCT) is used to reconstruct the spatial tensors for matrix operations during network training. Since high-frequency components of images are known to be less critical, a large proportion of these parameters can be set to zero when networks are trained with the proposed frequency regularization. Comprehensive evaluations on various state-of-the-art network architectures, including LeNet, Alexnet, VGG, Resnet, ViT, UNet, GAN, and VAE, demonstrate the effectiveness of the proposed frequency regularization. For a very small accuracy decrease (less than 2%), a LeNet5 with 0.4M parameters can be represented by only 776 float16 numbers (over 1100× reduction), and a UNet with 34M parameters can be represented by only 759 float16 numbers (over 80000× reduction). In particular, the original size of the UNet model is reduced from 366 Mb to 4.5 Kb.
Putting the Object Back into Video Object Segmentation
Authors: Ho Kei Cheng, Seoung Wug Oh, Brian Price, Joon-Young Lee, Alexander Schwing
Date: 2023-10-19 00:00:00
Proceeding: CVPR 2024 1
Tasks: Object, Segmentation, Semantic Segmentation, Semi-Supervised Video Object Segmentation, Video Object Segmentation, Video Semantic Segmentation, Visual Object Tracking

We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentation result. Recent works on VOS employ bottom-up pixel-level memory reading which struggles due to matching noise, especially in the presence of distractors, resulting in lower performance in more challenging data. In contrast, Cutie performs top-down object-level memory reading by adapting a small set of object queries. Via those, it interacts with the bottom-up pixel features iteratively with a query-based object transformer (qt, hence Cutie). The object queries act as a high-level summary of the target object, while high-resolution feature maps are retained for accurate segmentation. Together with foreground-background masked attention, Cutie cleanly separates the semantics of the foreground object from the background. On the challenging MOSE dataset, Cutie improves by 8.7 J&F over XMem with a similar running time and improves by 4.2 J&F over DeAOT while being three times faster. Code is available at: https://hkchengrex.github.io/Cutie
SemantIC: Semantic Interference Cancellation Towards 6G Wireless Communications
Authors: Wensheng Lin, Yuna Yan, Lixin Li, Zhu Han, Tad Matsumoto
Date: 2023-10-19 00:00:00
Tasks: Decoder

This letter proposes a novel anti-interference technique, semantic interference cancellation (SemantIC), for enhancing information quality towards the sixth-generation (6G) wireless networks. SemantIC only requires the receiver to concatenate the channel decoder with a semantic auto-encoder. This constructs a turbo loop which iteratively and alternately eliminates noise in the signal domain and the semantic domain. From the viewpoint of network information theory, the neural network of the semantic auto-encoder stores side information by training, and provides side information in iterative decoding, as an implementation of the Wyner-Ziv theorem. Simulation results verify the performance improvement by SemantIC without extra channel resource cost.
Scene-Text Oriented Reffering Expression Comprehension
Authors: Yuqi Bu, Liuwu Li, Jiayuan Xie, Qiong Liu, Yi Cai, Qingbao Huang, Qing Li
Date: 2022-11-04 00:00:00
Proceeding: 2023 2022 11
Tasks: Object Localization, Referring Expression, Referring Expression Comprehension

Abstract—Referring expression comprehension (REC) aims to
identify and locate a specific object in visual scenes referred
to by a natural language expression. Existing studies of REC
only focus on basic visual attributes and neglect scene text.
Since scene text has the functions of object identification and
disambiguation, it is naturally and frequently used to refer to
objects. However, existing methods do not explicitly recognize text
in images and fail to align scene text mentioned in expressions
with the text shown in images, resulting in object localization
errors. This study takes the first step toward addressing these
limitations. First, we introduce a new task called scene-text
oriented referring expression comprehension, which aims to align
visual cues and textual semantics of scene text with referring
expressions and visual contents. Second, we propose a scene text
awareness network that can bridge the gap between texts from
two modalities by grounding visual representations of expressioncorrelated scene texts. Specifically, we propose a correlated text
extraction module to solve the problem of lacking semantic understanding, and a correlated region activation module to address
the fixed alignment problem and absent alignment problem.
These modules ensure that the proposed method focuses on local
regions that are most relevant to scene text, thus mitigating the
misalignment of scene text with irrelevant regions. Third, to
conduct quantitative evaluations, we establish a new benchmark
dataset called RefText. Experimental results demonstrate that the
proposed method can effectively comprehend scene-text oriented
referring expressions and achieves excellent performance.
Index Terms—Referring expression comprehension, scene text
representation, multimodal alignment.
Variational Inference for SDEs Driven by Fractional Noise
Authors: Rembert Daems, Manfred Opper, Guillaume Crevecoeur, Tolga Birdal
Date: 2023-10-19 00:00:00
Tasks: Variational Inference, Video Prediction

We present a novel variational framework for performing inference in (neural) stochastic differential equations (SDEs) driven by Markov-approximate fractional Brownian motion (fBM). SDEs offer a versatile tool for modeling real-world continuous-time dynamic systems with inherent noise and randomness. Combining SDEs with the powerful inference capabilities of variational methods, enables the learning of representative function distributions through stochastic gradient descent. However, conventional SDEs typically assume the underlying noise to follow a Brownian motion (BM), which hinders their ability to capture long-term dependencies. In contrast, fractional Brownian motion (fBM) extends BM to encompass non-Markovian dynamics, but existing methods for inferring fBM parameters are either computationally demanding or statistically inefficient. In this paper, building upon the Markov approximation of fBM, we derive the evidence lower bound essential for efficient variational inference of posterior path measures, drawing from the well-established field of stochastic analysis. Additionally, we provide a closed-form expression to determine optimal approximation coefficients. Furthermore, we propose the use of neural networks to learn the drift, diffusion and control terms within our variational posterior, leading to the variational training of neural-SDEs. In this framework, we also optimize the Hurst index, governing the nature of our fractional noise. Beyond validation on synthetic data, we contribute a novel architecture for variational latent video prediction,-an approach that, to the best of our knowledge, enables the first variational neural-SDE application to video perception.
Frozen Transformers in Language Models Are Effective Visual Encoder Layers
Authors: Ziqi Pang, Ziyang Xie, Yunze Man, Yu-Xiong Wang
Date: 2023-10-19 00:00:00
Tasks: Action Recognition, Image-text Retrieval, Motion Forecasting, Point Cloud Classification, Question Answering, Text Retrieval, Visual Question Answering

This paper reveals that large language models (LLMs), despite being trained solely on textual data, are surprisingly strong encoders for purely visual tasks in the absence of language. Even more intriguingly, this can be achieved by a simple yet previously overlooked strategy -- employing a frozen transformer block from pre-trained LLMs as a constituent encoder layer to directly process visual tokens. Our work pushes the boundaries of leveraging LLMs for computer vision tasks, significantly departing from conventional practices that typically necessitate a multi-modal vision-language setup with associated language prompts, inputs, or outputs. We demonstrate that our approach consistently enhances performance across a diverse range of tasks, encompassing pure 2D and 3D visual recognition tasks (e.g., image and point cloud classification), temporal modeling tasks (e.g., action recognition), non-semantic tasks (e.g., motion forecasting), and multi-modal tasks (e.g., 2D/3D visual question answering and image-text retrieval). Such improvements are a general phenomenon, applicable to various types of LLMs (e.g., LLaMA and OPT) and different LLM transformer blocks. We additionally propose the information filtering hypothesis to explain the effectiveness of pre-trained LLMs in visual encoding -- the pre-trained LLM transformer blocks discern informative visual tokens and further amplify their effect. This hypothesis is empirically supported by the observation that the feature activation, after training with LLM transformer blocks, exhibits a stronger focus on relevant regions. We hope that our work inspires new perspectives on utilizing LLMs and deepening our understanding of their underlying mechanisms. Code is available at https://github.com/ziqipang/LM4VisualEncoding.
Does Your Model Think Like an Engineer? Explainable AI for Bearing Fault Detection with Deep Learning
Authors: Thomas Decker, Michael Lebacher, Volker Tresp
Date: 2023-10-19 00:00:00
Tasks: Fault Detection

Deep Learning has already been successfully applied to analyze industrial sensor data in a variety of relevant use cases. However, the opaque nature of many well-performing methods poses a major obstacle for real-world deployment. Explainable AI (XAI) and especially feature attribution techniques promise to enable insights about how such models form their decision. But the plain application of such methods often fails to provide truly informative and problem-specific insights to domain experts. In this work, we focus on the specific task of detecting faults in rolling element bearings from vibration signals. We propose a novel and domain-specific feature attribution framework that allows us to evaluate how well the underlying logic of a model corresponds with expert reasoning. Utilizing the framework we are able to validate the trustworthiness and to successfully anticipate the generalization ability of different well-performing deep learning models. Our methodology demonstrates how signal processing tools can effectively be used to enhance Explainable AI techniques and acts as a template for similar problems.
AutoMix: Automatically Mixing Language Models
Authors: Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, Mausam
Date: 2023-10-19 00:00:00
Tasks: Diversity

Large language models (LLMs) are now available from cloud API providers in various sizes and configurations. While this diversity offers a broad spectrum of choices, effectively leveraging the options to optimize computational cost and performance remains challenging. In this work, we present Automix, an approach that strategically routes queries to larger LMs, based on the approximate correctness of outputs from a smaller LM. Central to Automix are two key technical contributions. First, it has a few-shot self-verification mechanism, which estimates the reliability of its own outputs without requiring extensive training. Second, given that self-verification can be noisy, it employs a POMDP based router that can effectively select an appropriately sized model, based on answer confidence. Experiments across five language models and five challenging datasets show that Automix consistently surpasses strong baselines, reducing computational cost by over 50% for comparable performance.
An Emulator for Fine-Tuning Large Language Models using Small Language Models
Authors: Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, Christopher D. Manning
Date: 2023-10-19 00:00:00
Tasks: Instruction Following

Widely used language models (LMs) are typically built by scaling up a two-stage training pipeline: a pre-training stage that uses a very large, diverse dataset of text and a fine-tuning (sometimes, 'alignment') stage that uses targeted examples or other specifications of desired behaviors. While it has been hypothesized that knowledge and skills come from pre-training, and fine-tuning mostly filters this knowledge and skillset, this intuition has not been extensively tested. To aid in doing so, we introduce a novel technique for decoupling the knowledge and skills gained in these two stages, enabling a direct answer to the question, "What would happen if we combined the knowledge learned by a large model during pre-training with the knowledge learned by a small model during fine-tuning (or vice versa)?" Using an RL-based framework derived from recent developments in learning from human preferences, we introduce emulated fine-tuning (EFT), a principled and practical method for sampling from a distribution that approximates (or 'emulates') the result of pre-training and fine-tuning at different scales. Our experiments with EFT show that scaling up fine-tuning tends to improve helpfulness, while scaling up pre-training tends to improve factuality. Beyond decoupling scale, we show that EFT enables test-time adjustment of competing behavioral traits like helpfulness and harmlessness without additional training. Finally, a special case of emulated fine-tuning, which we call LM up-scaling, avoids resource-intensive fine-tuning of large pre-trained models by ensembling them with small fine-tuned models, essentially emulating the result of fine-tuning the large pre-trained model. Up-scaling consistently improves helpfulness and factuality of instruction-following models in the Llama, Llama-2, and Falcon families, without additional hyperparameters or training.