pwc-archive/papers-with-abstracts huggingface dataset:
Exact Bayesian Gaussian Cox Processes Using Random Integral
Authors: Bingjing Tang, Julia Palacios
Date: 2024-06-28 00:00:00
Tasks: Data Augmentation, Point Processes

A Gaussian Cox process is a popular model for point process data, in which the intensity function is a transformation of a Gaussian process. Posterior inference of this intensity function involves an intractable integral (i.e., the cumulative intensity function) in the likelihood resulting in doubly intractable posterior distribution. Here, we propose a nonparametric Bayesian approach for estimating the intensity function of an inhomogeneous Poisson process without reliance on large data augmentation or approximations of the likelihood function. We propose to jointly model the intensity and the cumulative intensity function as a transformed Gaussian process, allowing us to directly bypass the need of approximating the cumulative intensity function in the likelihood. We propose an exact MCMC sampler for posterior inference and evaluate its performance on simulated data. We demonstrate the utility of our method in three real-world scenarios including temporal and spatial event data, as well as aggregated time count data collected at multiple resolutions. Finally, we discuss extensions of our proposed method to other point processes.
Filtration learning in exact multi-parameter persistent homology and classification of time-series data
Authors: Keunsu Kim, Jae-Hun Jung
Date: 2024-06-28 00:00:00
Tasks: Time Series, Time Series Analysis

To analyze the topological properties of the given discrete data, one needs to consider a continuous transform called filtration. Persistent homology serves as a tool to track changes of homology in the filtration. The outcome of the topological analysis of data varies depending on the choice of filtration, making the selection of filtration crucial. Filtration learning is an attempt to find an optimal filtration that minimizes the loss function. Exact Multi-parameter Persistent Homology (EMPH) has been recently proposed, particularly for topological time-series analysis, that utilizes the exact formula of rank invariant instead of calculating it. In this paper, we propose a framework for filtration learning of EMPH. We formulate an optimization problem and propose an algorithm for solving the problem. We then apply the proposed algorithm to several classification problems. Particularly, we derive the exact formula of the gradient of the loss function with respect to the filtration parameters, which makes it possible to directly update the filtration without using automatic differentiation, significantly enhancing the learning process.
Improving Finite Sample Performance of Causal Discovery by Exploiting Temporal Structure
Authors: Christine W Bang, Janine Witte, Ronja Foraita, Vanessa Didelez
Date: 2024-06-27 00:00:00
Tasks: Causal Discovery

Methods of causal discovery aim to identify causal structures in a data driven way. Existing algorithms are known to be unstable and sensitive to statistical errors, and are therefore rarely used with biomedical or epidemiological data. We present an algorithm that efficiently exploits temporal structure, so-called tiered background knowledge, for estimating causal structures. Tiered background knowledge is readily available from, e.g., cohort or registry data. When used efficiently it renders the algorithm more robust to statistical errors and ultimately increases accuracy in finite samples. We describe the algorithm and illustrate how it proceeds. Moreover, we offer formal proofs as well as examples of desirable properties of the algorithm, which we demonstrate empirically in an extensive simulation study. To illustrate its usefulness in practice, we apply the algorithm to data from a children's cohort study investigating the interplay of diet, physical activity and other lifestyle factors for health outcomes.
Rateless Stochastic Coding for Delay-Constrained Semantic Communication
Authors: Cheng Peng, Rulong Wang, Yong Xiao
Date: 2024-06-28 00:00:00
Tasks: Decoder, Perceptual Distance, Quantization, Semantic Communication

We consider the problem of joint source-channel coding for semantic communication from a rateless perspective, the purpose of which is to settle the balance between reliability (distortion/perception) and effectiveness (rate) of transmission over uncertain channels. In particular, we propose a more general communication objective that minimizes the perceptual distance by incorporating a semantic-level reconstruction objective in addition to the conventional pixel-level reconstruction objective. Based on the proposed objective, we then propose a novel JSCC coding scheme called rateless stochastic coding (RSC) by introducing a generative decoder and dithered quantization. The coding scheme enables reconstruction based on both distortion and perception metrics through rateless transmission. Extensive experiments demonstrate that the proposed RSC can achieve variable rates of transmission maintaining an excellent trade-off between distortion and perception.
Doc2Token: Bridging Vocabulary Gap by Predicting Missing Tokens for E-commerce Search
Authors: Kaihao Li, Juexin Lin, Tony Lee
Date: 2024-06-28 00:00:00
Tasks: Diversity, Information Retrieval, Retrieval

Addressing the "vocabulary mismatch" issue in information retrieval is a central challenge for e-commerce search engines, because product pages often miss important keywords that customers search for. Doc2Query[1] is a popular document-expansion technique that predicts search queries for a document and includes the predicted queries with the document for retrieval. However, this approach can be inefficient for e-commerce search, because the predicted query tokens are often already present in the document. In this paper, we propose Doc2Token, a technique that predicts relevant tokens (instead of queries) that are missing from the document and includes these tokens in the document for retrieval. For the task of predicting missing tokens, we introduce a new metric, "novel ROUGE score". Doc2Token is demonstrated to be superior to Doc2Query in terms of novel ROUGE score and diversity of predictions. Doc2Token also exhibits efficiency gains by reducing both training and inference times. We deployed the feature to production and observed significant revenue gain in an online A/B test, and launched the feature to full traffic on Walmart.com. [1] R. Nogueira, W. Yang, J. Lin, K. Cho, Document expansion by query prediction, arXiv preprint arXiv:1904.08375 (2019)
PID4LaTe: a physics-informed deep learning model for lake multi-depth temperature prediction
Authors: Lingling Chen, Li Wang, Weixiang Ma, Xiaoya Xu & Hao Wang
Date: 2024-06-24 00:00:00
Proceeding: Earth Science Informatics 2024 6
Tasks: Depth Estimation, Depth Prediction

Lake temperature plays a pivotal role in the physical and chemical processes in the water. It has a significant impact on the distribution of lake organisms. In lake temperature modelling, the physics-based models have the shortcomings of parameter calibration and generalization difficulties. The data-driven models are highly data dependent. This makes hybrid models an effective solution at present. In this paper, we explore the spatial and temporal co-evolution process for multi-depth lake temperature and propose a Physics-Informed Deep learning model for Lake multi-depth Temperature prediction, PID4LaTe. It consists of three sub-models, two of which are long short-term memory (LSTM) models for spatial and temporal prediction respectively, and the other is a physical model, the General Lake Model. The physical model offers simulation data based on its rich knowledge for data-driven model learning, while guaranteeing consistency between model results and physical mechanisms. We compared PID4LaTe with the Process-Based model (PB), the Deep Learning model (DL) and the Physics-Guided Recurrent Neural Networks model (PGRNN), and used Root Mean Square Error (RMSE), Mean Square Error (MSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) to assess the effectiveness of the models. Extensive experiments show the superiority of our hybrid model for lake multi-depth prediction over PB, DL and PGRNN with RMSE of 0.798, MSE of 0.644, MAE of 0.567 and MAPE of 4.367% in Mendota Lake and RMSE of 1.099, MSE of 1.261, MAE of 0.783 and MAPE of 7.936% in Sparkling Lake. The two-cases study indicates that the hybrid model combining the physics-based model with the data-driven model is a promising technique for multi-depth lake temperature predicting. This study provides a reference method for accurate prediction of temperature at multiple depths in lakes.
Less-supervised learning with knowledge distillation for sperm morphology analysis
Authors: Ali Nabipour, Mohammad Javad Shams Nejati, Yasaman Boreshban, Seyed Abolghasem Mirroshandel
Date: 2024-05-08 00:00:00
Proceeding: Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 2024 5
Tasks: Anomaly Detection, Knowledge Distillation, Unsupervised Anomaly Detection

Sperm Morphology Analysis (SMA) is pivotal in diagnosing male infertility. However, manual analysis is subjective and time-intensive. Artificial intelligence presents automated alternatives, but hurdles like limited data and image quality constraints hinder its efficacy. These challenges impede Deep Learning (DL) models from grasping crucial sperm features. A solution enabling DL models to learn sample nuances, even with limited data, would be invaluable. This study proposes a Knowledge Distillation (KD) method to distinguish normal from abnormal sperm cells, leveraging the Modified Human Sperm Morphology Analysis dataset. Despite low-resolution, blurry images, our method yields relevant results. We exclusively utilize normal samples to train the model for anomaly detection, crucial in scenarios lacking abnormal data – a common issue in medical tasks. Our aim is to train an Anomaly Detection model using a dataset comprising unclear images and limited samples, without direct exposure to abnormal data. Our method achieves Receiver ROC/AUC scores of 70.4%, 87.6%, and 71.1% for head, vacuole, and acrosome, respectively, our method matches traditional DL model performance with less than 70% of the data. This less-supervised approach shows promise in advancing SMA despite data scarcity. Furthermore, KD enables model adaptability to edge devices in fertility clinics, requiring less processing power.
BPE-knockout: Pruning Pre-existing BPE Tokenisers with Backwards-compatible Morphological Semi-supervision
Authors: Thomas Bauwens, Pieter Delobelle
Date: 2024-06-13 00:00:00
Proceeding: NAACL 2024 6
Tasks: Morphological Analysis

Byte-pair encoding (BPE) has become the default subword tokeniser in language models (LMs), allowing the representation of an infinite space of text with a finite set of units. Yet, BPE training is unsupervised, receiving no explicit information about a language’s morphology. This results in a subword vocabulary wherein many units are a concatenation of partial morphemes, preventing their formation as tokens. This, in turn, causes consistent intra-word patterns to be displayed inconsistently to downstream models, and bloats the vocabulary, hence requiring unnecessary embedding storage. In this paper, we address this issue by identifying blameworthy BPE merges and removing the resulting subwords from the BPE vocabulary, without impeding further use of merges that relied on them. We find that our method, BPE-knockout, is effective at making BPE’s segmentation positions adhere better to derivational and compound boundaries in English, Dutch and German, and improves token-based tasks in Dutch RoBERTa models, indicating that a tokeniser’s adherence to morphology impacts downstream models. We demonstrate the latter not only by training LMs from scratch, but also by continuing the pre-training of existing LMs. This proves promising, showing that suboptimal tokenisers can be remedied whilst salvaging training cost of downstream LMs.
Rescaling Large Datasets Based on Validation Outcomes of a Pre-trained Network
Authors: Thanh Tuan NGUYEN, Thanh Phuong Nguyen
Date: 2024-07-01 00:00:00
Proceeding: Pattern Recognition Letters 2024 7
Tasks: image-classification, Image Classification

In fact, several categories in a large dataset are not difficult for recent advanced deep
neural networks to recognize. Eliminating them for a challenging smaller subset will
assist the early network proposals in taking a quick trial of verification. To this end, we
propose an efficient rescaling method based on the validation outcomes of a pre-trained
model. Firstly, we will take out the sensitive images of the lowest-accuracy classes
of the validation outcomes. Each of such images is then considered to identify which
label it was confused with. Gathering the lowest-accuracy classes along with the most
confused ones can produce a smaller subset with a higher challenge for quick validation
of an early network draft. Finally, a rescaling application is introduced to rescale two
popular large datasets (ImageNet and Places365) for different tiny subsets (i.e., ReINΩ
and RePLΩ respectively). Experiments for image classification have proved that neural
networks obtaining good performance on the original datasets also achieve good results
on their rescaled subsets. For instance, MobileNetV1 and MobileNetV2 with 70.6%
and 72% on ImageNet respectively obtained 46.53% and 47.47% on its small subset
ReIN30, which only contains about 39000 images. It can be observed that the better
performance of MobileNetV2 on ImageNet correspondingly leads to the better rate on
its rescaled subset. Appropriately, utilizing these rescaled sets would help researchers
save time and computational costs in the way of designing deep neural architectures.
Temporal distribution of clusters of investors and their application in prediction with expert advice
Authors: Wojciech Wisniewski, Yuri Kalnishkan, David Lindsay, Siân Lindsay
Date: 2024-06-04 00:00:00

Financial organisations such as brokers face a significant challenge in servicing the investment needs of thousands of their traders worldwide. This task is further compounded since individual traders will have their own risk appetite and investment goals. Traders may look to capture short-term trends in the market which last only seconds to minutes, or they may have longer-term views which last several days to months. To reduce the complexity of this task, client trades can be clustered. By examining such clusters, we would likely observe many traders following common patterns of investment, but how do these patterns vary through time? Knowledge regarding the temporal distributions of such clusters may help financial institutions manage the overall portfolio of risk that accumulates from underlying trader positions. This study contributes to the field by demonstrating that the distribution of clusters derived from the real-world trades of 20k Foreign Exchange (FX) traders (from 2015 to 2017) is described in accordance with Ewens' Sampling Distribution. Further, we show that the Aggregating Algorithm (AA), an on-line prediction with expert advice algorithm, can be applied to the aforementioned real-world data in order to improve the returns of portfolios of trader risk. However we found that the AA 'struggles' when presented with too many trader ``experts'', especially when there are many trades with similar overall patterns. To help overcome this challenge, we have applied and compared the use of Statistically Validated Networks (SVN) with a hierarchical clustering approach on a subset of the data, demonstrating that both approaches can be used to significantly improve results of the AA in terms of profitability and smoothness of returns.
Predicting Customer Goals in Financial Institution Services: A Data-Driven LSTM Approach
Authors: Andrew Estornell, Stylianos Loukas Vasileiou, William Yeoh, Daniel Borrajo, Rui Silva
Date: 2024-05-22 00:00:00

In today's competitive financial landscape, understanding and anticipating customer goals is crucial for institutions to deliver a personalized and optimized user experience. This has given rise to the problem of accurately predicting customer goals and actions. Focusing on that problem, we use historical customer traces generated by a realistic simulator and present two simple models for predicting customer goals and future actions -- an LSTM model and an LSTM model enhanced with state-space graph embeddings. Our results demonstrate the effectiveness of these models when it comes to predicting customer goals and actions.
Composite Adaptive Disturbance Rejection in Robotics via Instrumental Variables based DREM
Authors: Anton Glushchenko, Konstantin Lastochkin
Date: 2024-06-28 00:00:00
Tasks: parameter estimation

In this paper we consider trajectory tracking problem for robotic systems affected by unknown external perturbations. Considering possible solutions, we restrict our attention to composite adaptation, which, particularly, ensures parametric error convergence being desirable to enhance overall stability and robustness of a closed-loop system. At the same time, existing composite approaches cannot simultaneously relax stringent persistence of excitation requirement and guarantee convergence of parametric error to zero for a perturbed scenario. So, a new composite adaptation scheme is proposed, which successfully overcomes mentioned problems of known counterparts and has several salient features. First, it includes a novel adaptive disturbance rejection control law for a general n-DoF dynamical model in the Euler-Lagrange form, which, without achievement of the parameter estimation goal, ensures global stability via application of a high-gain external torque observer augmented with some adaptation law. Secondly, such law is extended with a composite summand derived via the recently proposed Instrumental Variables based Dynamic Regressor Extension and Mixing procedure, which relaxes excitation conditions and ensures asymptotic parameter estimation and reference tracking in the presence of external torque under some non-restrictive assumptions. An illustrative example shows the effectiveness and superiority of the proposed approach in comparison with existing solutions.
Identifying Ordinary Differential Equations for Data-efficient Model-based Reinforcement Learning
Authors: Tobias Nagel, Marco F. Huber
Date: 2024-06-28 00:00:00
Tasks: Model-based Reinforcement Learning, Physics-informed machine learning

The identification of a mathematical dynamics model is a crucial step in the designing process of a controller. However, it is often very difficult to identify the system's governing equations, especially in complex environments that combine physical laws of different disciplines. In this paper, we present a new approach that allows identifying an ordinary differential equation by means of a physics-informed machine learning algorithm. Our method introduces a special neural network that allows exploiting prior human knowledge to a certain degree and extends it autonomously, so that the resulting differential equations describe the system as accurately as possible. We validate the method on a Duffing oscillator with simulation data and, additionally, on a cascaded tank example with real-world data. Subsequently, we use the developed algorithm in a model-based reinforcement learning framework by alternately identifying and controlling a system to a target state. We test the performance by swinging-up an inverted pendulum on a cart.
Condition monitoring of wind turbine blades via learning-based methods
Authors: Giovanni Zaniboni, Alessio Dallabona, Johnny Nielsen, Dimitrios Papageorgiou
Date: 2024-06-28 00:00:00
Tasks: Fault Detection

This paper addresses the topic of condition monitoring of wind turbine blades and presents a learning-based approach to fault detection. The proposed scheme utilises Principal Components Analysis and Autoencoders to derive data-driven models from root-bending moment and other measurements. The models are trained with real data obtained from a fault-free wind turbine, and then validated on data corresponding to unknown health condition. Online test statistics, employing static thresholds and Generalized Likelihood Ratio tests, are used on residual signals generated by discrepancies between the actual and reconstructed measurements to detect deviations from nominal operation. The efficacy and effectiveness of the proposed methods are demonstrated using real-life data collected from wind turbines experiencing blade faults.
Global Regulation of Feedforward Nonlinear Systems: A Logic-Based Switching Gain Approach
Authors: Debao Fan, Xianfu Zhang, Gang Feng, Hanfeng Li
Date: 2024-06-28 00:00:00

In this article, we investigate the global regulation problem for a class of feedforward nonlinear systems. Notably, the systems under consideration allow unknown input-output-dependent nonlinear growth rates, which has not been considered in existing works. A novel logic-based switching (LBS) gain approach is proposed to counteract system uncertainties and nonlinearities. Furthermore, a tanh-type speed-regulation function is embedded into the switching mechanism for the first time to improve the convergence speed and transient performance. Then, a switching adaptive output feedback (SAOF) controller is proposed based on the developed switching mechanism, which is of a concise form and low-complexity characteristic. It is shown that the objective of global regulation is achieved with faster convergence speed and better transient performance under the proposed controller. Moreover, by strengthening the switching mechanism, the improved control approach can deal with feedforward nonlinear systems with external disturbances. Finally, representative examples are presented to demonstrate the effectiveness and advantages of our approach in comparison with the existing approaches.
Practical Power System Inertia Monitoring Based on Pumped Storage Hydropower Operation Signature
Authors: Hongyu Li, Chang Chen, Mark Baldwin, Shutang You, Wenpeng Yu, Lin Zhu, Yilu Liu
Date: 2024-06-28 00:00:00
Tasks: Decision Making, Event Detection

This paper proposes a practical method to monitor power system inertia using Pumped Storage Hydropower (PSH) switching-off events. This approach offers real-time system-level inertia estimation with minimal expenses, no disruption, and the inclusion of behind-the-meter inertia. First, accurate inertia estimation is achieved through improved RoCoF calculation that accounts for pre-event RoCoF, reducing common random frequency fluctuations in practice. Second, PSH field data is analyzed, highlighting the benefits of using switching-off events for grid inertia estimation. Third, an event detection trigger is designed to capture pump switching-off events based on local and system features. Fourth, the method is validated on the U.S. Eastern Interconnection model with over 60,000 buses, demonstrating very high accuracy (3%-5% error rate). Finally, it is applied to the U.S. Western Interconnection, with field validation showing a 9.9% average absolute error rate. Despite challenges in practical power system inertia estimation, this method enhances decision-making for power grid reliability and efficiency, addressing challenges posed by renewable energy integration.