pwc-archive/papers-with-abstracts huggingface dataset:
Improving PTM Site Prediction by Coupling of Multi-Granularity Structure and Multi-Scale Sequence Representation
Authors: Zhengyi Li, Menglu Li, Lida Zhu, Wen Zhang
Date: 2024-01-04 00:00:00
Tasks: Contrastive Learning, Prediction, Representation Learning

Protein post-translational modification (PTM) site prediction is a fundamental task in bioinformatics. Several computational methods have been developed to predict PTM sites. However, existing methods ignore the structure information and merely utilize protein sequences. Furthermore, designing a more fine-grained structure representation learning method is urgently needed as PTM is a biological event that occurs at the atom granularity. In this paper, we propose a PTM site prediction method by Coupling of Multi-Granularity structure and Multi-Scale sequence representation, PTM-CMGMS for brevity. Specifically, multigranularity structure-aware representation learning is designed to learn neighborhood structure representations at the amino acid, atom, and whole protein granularity from AlphaFold predicted structures, followed by utilizing contrastive learning to optimize the structure representations.Additionally, multi-scale sequence representation learning is used to extract context sequence information, and motif generated by aligning all context sequences of PTM sites assists the prediction. Extensive experiments on three datasets show that PTM-CMGMS outperforms the state-of-the-art methods.
Regenerative Medicine for Tendon/Ligament Injuries: De Novo Equine Tendon/Ligament Neotissue Generation and Application
Authors: Takashi Taguchi
Date: 2023-10-24 00:00:00

Tendon and ligament injuries are debilitating conditions across species. Poor regenerative capacities of these tissues limit restoration of original functions. The first study of this dissertation evaluated the effect of cellular administration on tendon/ligament injuries in horses using meta-analysis. The findings led to the second study that engineered implantable de novo tendon neotissue using equine adipose-derived multipotent stromal cells and collagen type I. The neotendon was evaluated for its biocompatibility and therapeutic potential in the third study using immunocompetent and immunocompromised rat bilateral calcaneal tendon elongation model. The fourth study investigated the therapeutic effects of neotendon in surgically-induced non-terminal equine accessory ligament of deep digital flexor tendon injury model.
Consistent asset modelling with random coefficients and switches between regimes
Authors: Felix L. Wolf, Griselda Deelstra, Lech A. Grzelak
Date: 2024-01-18 00:00:00

We explore a stochastic model that enables capturing external influences in two specific ways. The model allows for the expression of uncertainty in the parametrisation of the stochastic dynamics and incorporates patterns to account for different behaviours across various times or regimes. To establish our framework, we initially construct a model with random parameters, where the switching between regimes can be dictated either by random variables or deterministically. Such a model is highly interpretable. We further ensure mathematical consistency by demonstrating that the framework can be elegantly expressed through local volatility models taking the form of standard jump diffusions. Additionally, we consider a Markov-modulated approach for the switching between regimes characterised by random parameters. For all considered models, we derive characteristic functions, providing a versatile tool with wide-ranging applications. In a numerical experiment, we apply the framework to the financial problem of option pricing. The impact of parameter uncertainty is analysed in a two-regime model, where the asset process switches between periods of high and low volatility imbued with high and low uncertainty, respectively.
An Exploration to the Correlation Structure and Clustering of Macroeconomic Variables
Authors: Garvit Arora, Shubhangi Tiwari, Ying Wu, Xuan Mei
Date: 2024-01-18 00:00:00
Tasks: Clustering, Management

As a quantitative characterization of the complicated economy, Macroeconomic Variables (MEVs), including GDP, inflation, unemployment, income, spending, interest rate, etc., are playing a crucial role in banks' portfolio management and stress testing exercise. In recent years, especially during the COVID-19 period and the current high inflation environment, people are frequently talking about the changing "correlation structure" of MEVs. In this paper, we use a principal component based algorithm to perform unsupervised clustering on MEVs so we can quantify and better understand MEVs' correlation structure in any given period. We also demonstrate how this method can be used to visualize historical MEVs pattern changes between 2000 and 2022. Further, we use this method to compare different hypothetical and/or historical macroeconomic scenarios and present our key findings. One of these interesting observations is that, for a list of 132 transformations derived from 44 targeted MEVs that cover 5 different aspects of the U.S. economy (which takes as a subset the 10+ key MEVs published by FRB), compared to benign years where there are typically 20-25 clusters, during the great financial crisis (GFC), i.e., 2007-2010, they exhibited a more synchronized and less diversified pattern of movement, forming roughly 15 clusters. We also see this contrast in the hypothetical CCAR2023 FRB scenarios where the Severely Adverse scenario has 15 clusters and the Baseline scenario has 21 clusters. We provide our interpretation to this observation and hope this research can inspire and benefit researchers from different domains all over the world.
Cross-Domain Behavioral Credit Modeling: transferability from private to central data
Authors: O. Didkovskyi, N. Jean, G. Le Pera, C. Nordio
Date: 2024-01-18 00:00:00

This paper introduces a credit risk rating model for credit risk assessment in quantitative finance, aiming to categorize borrowers based on their behavioral data. The model is trained on data from Experian, a widely recognized credit bureau, to effectively identify instances of loan defaults among bank customers. Employing state-of-the-art statistical and machine learning techniques ensures the model's predictive accuracy. Furthermore, we assess the model's transferability by testing it on behavioral data from the Bank of Italy, demonstrating its potential applicability across diverse datasets during prediction. This study highlights the benefits of incorporating external behavioral data to improve credit risk assessment in financial institutions.
Object Attribute Matters in Visual Question Answering
Authors: Peize Li, Qingyi Si, Peng Fu, Zheng Lin, Yan Wang
Date: 2023-12-20 00:00:00
Tasks: Attribute, Graph Neural Network, Knowledge Distillation, Object, Question Answering, Representation Learning, Scene Understanding, Visual Question Answering, Visual Question Answering (VQA)

Visual question answering is a multimodal task that requires the joint comprehension of visual and textual information. However, integrating visual and textual semantics solely through attention layers is insufficient to comprehensively understand and align information from both modalities. Intuitively, object attributes can naturally serve as a bridge to unify them, which has been overlooked in previous research. In this paper, we propose a novel VQA approach from the perspective of utilizing object attribute, aiming to achieve better object-level visual-language alignment and multimodal scene understanding. Specifically, we design an attribute fusion module and a contrastive knowledge distillation module. The attribute fusion module constructs a multimodal graph neural network to fuse attributes and visual features through message passing. The enhanced object-level visual features contribute to solving fine-grained problem like counting-question. The better object-level visual-language alignment aids in understanding multimodal scenes, thereby improving the model's robustness. Furthermore, to augment scene understanding and the out-of-distribution performance, the contrastive knowledge distillation module introduces a series of implicit knowledge. We distill knowledge into attributes through contrastive loss, which further strengthens the representation learning of attribute features and facilitates visual-linguistic alignment. Intensive experiments on six datasets, COCO-QA, VQAv2, VQA-CPv2, VQA-CPv1, VQAvs and TDIUC, show the superiority of the proposed method.
CRD: Collaborative Representation Distance for Practical Anomaly Detection
Authors: Chao Han, Yudong Yan
Date: 2023-12-20 00:00:00
Tasks: Anomaly Detection, Computational Efficiency, Defect Detection, GPU

Visual defect detection plays an important role in intelligent industry. Patch based methods consider visual images as a collection of image patches according to positions, which have stronger discriminative ability for small defects in products, e.g. scratches on pills. However, the nearest neighbor search for the query image and the stored patches will occupy $O(n)$ complexity in terms of time and space requirements, posing strict challenges for deployment in edge environments. In this paper, we propose an alternative approach to the distance calculation of image patches via collaborative representation models. Starting from the nearest neighbor distance with $L_0$ constraint, we relax the constraint to $L_2$ constraint and solve the distance quickly in close-formed without actually accessing the original stored collection of image patches. Furthermore, we point out that the main computational burden of this close-formed solution can be pre-computed by high-performance server before deployment. Consequently, the distance calculation on edge devices only requires a simple matrix multiplication, which is extremely lightweight and GPU-friendly. Performance on real industrial scenarios demonstrates that compared to the existing state-of-the-art methods, this distance achieves several hundred times improvement in computational efficiency with slight performance drop, while greatly reducing memory overhead.
Explainable Multimodal Sentiment Analysis on Bengali Memes
Authors: Kazi Toufique Elahi, Tasnuva Binte Rahman, Shakil Shahriar, Samir Sarker, Sajib Kumar Saha Joy, Faisal Muhammad Shah
Date: 2023-12-20 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Multimodal Sentiment Analysis, Sentiment Analysis

Memes have become a distinctive and effective form of communication in the digital era, attracting online communities and cutting across cultural barriers. Even though memes are frequently linked with humor, they have an amazing capacity to convey a wide range of emotions, including happiness, sarcasm, frustration, and more. Understanding and interpreting the sentiment underlying memes has become crucial in the age of information. Previous research has explored text-based, image-based, and multimodal approaches, leading to the development of models like CAPSAN and PromptHate for detecting various meme categories. However, the study of low-resource languages like Bengali memes remains scarce, with limited availability of publicly accessible datasets. A recent contribution includes the introduction of the MemoSen dataset. However, the achieved accuracy is notably low, and the dataset suffers from imbalanced distribution. In this study, we employed a multimodal approach using ResNet50 and BanglishBERT and achieved a satisfactory result of 0.71 weighted F1-score, performed comparison with unimodal approaches, and interpreted behaviors of the models using explainable artificial intelligence (XAI) techniques.
Voila-A: Aligning Vision-Language Models with User's Gaze Attention
Authors: Kun Yan, Lei Ji, Zeyu Wang, Yuntao Wang, Nan Duan, Shuai Ma
Date: 2023-12-22 00:00:00

In recent years, the integration of vision and language understanding has led to significant advancements in artificial intelligence, particularly through Vision-Language Models (VLMs). However, existing VLMs face challenges in handling real-world applications with complex scenes and multiple objects, as well as aligning their focus with the diverse attention patterns of human users. In this paper, we introduce gaze information, feasibly collected by AR or VR devices, as a proxy for human attention to guide VLMs and propose a novel approach, Voila-A, for gaze alignment to enhance the interpretability and effectiveness of these models in real-world applications. First, we collect hundreds of minutes of gaze data to demonstrate that we can mimic human gaze modalities using localized narratives. We then design an automatic data annotation pipeline utilizing GPT-4 to generate the VOILA-COCO dataset. Additionally, we innovate the Voila Perceiver modules to integrate gaze information into VLMs while preserving their pretrained knowledge. We evaluate Voila-A using a hold-out validation set and a newly collected VOILA-GAZE Testset, which features real-life scenarios captured with a gaze-tracking device. Our experimental results demonstrate that Voila-A significantly outperforms several baseline models. By aligning model attention with human gaze patterns, Voila-A paves the way for more intuitive, user-centric VLMs and fosters engaging human-AI interaction across a wide range of applications.
Triamese-ViT: A 3D-Aware Method for Robust Brain Age Estimation from MRIs
Authors: Zhaonian Zhang, Richard Jiang
Date: 2024-01-13 00:00:00
Tasks: Age Estimation, Diagnostic

The integration of machine learning in medicine has significantly improved diagnostic precision, particularly in the interpretation of complex structures like the human brain. Diagnosing challenging conditions such as Alzheimer's disease has prompted the development of brain age estimation techniques. These methods often leverage three-dimensional Magnetic Resonance Imaging (MRI) scans, with recent studies emphasizing the efficacy of 3D convolutional neural networks (CNNs) like 3D ResNet. However, the untapped potential of Vision Transformers (ViTs), known for their accuracy and interpretability, persists in this domain due to limitations in their 3D versions. This paper introduces Triamese-ViT, an innovative adaptation of the ViT model for brain age estimation. Our model uniquely combines ViTs from three different orientations to capture 3D information, significantly enhancing accuracy and interpretability. Tested on a dataset of 1351 MRI scans, Triamese-ViT achieves a Mean Absolute Error (MAE) of 3.84, a 0.9 Spearman correlation coefficient with chronological age, and a -0.29 Spearman correlation coefficient between the brain age gap (BAG) and chronological age, significantly better than previous methods for brian age estimation. A key innovation of Triamese-ViT is its capacity to generate a comprehensive 3D-like attention map, synthesized from 2D attention maps of each orientation-specific ViT. This feature is particularly beneficial for in-depth brain age analysis and disease diagnosis, offering deeper insights into brain health and the mechanisms of age-related neural changes.
Rethinking FID: Towards a Better Evaluation Metric for Image Generation
Authors: Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, Sanjiv Kumar
Date: 2023-11-30 00:00:00
Proceeding: CVPR 2024 1
Tasks: Image Generation

As with many machine learning problems, the progress of image generation methods hinges on good evaluation metrics. One of the most popular is the Frechet Inception Distance (FID). FID estimates the distance between a distribution of Inception-v3 features of real images, and those of images generated by the algorithm. We highlight important drawbacks of FID: Inception's poor representation of the rich and varied content generated by modern text-to-image models, incorrect normality assumptions, and poor sample complexity. We call for a reevaluation of FID's use as the primary quality metric for generated images. We empirically demonstrate that FID contradicts human raters, it does not reflect gradual improvement of iterative text-to-image models, it does not capture distortion levels, and that it produces inconsistent results when varying the sample size. We also propose an alternative new metric, CMMD, based on richer CLIP embeddings and the maximum mean discrepancy distance with the Gaussian RBF kernel. It is an unbiased estimator that does not make any assumptions on the probability distribution of the embeddings and is sample efficient. Through extensive experiments and analysis, we demonstrate that FID-based evaluations of text-to-image models may be unreliable, and that CMMD offers a more robust and reliable assessment of image quality.
A locally statistical active contour model for SAR image segmentation can be solved by denoising algorithms
Authors: Guangming Liu, Quanying Sun, Jing Liang, Qi Liu
Date: 2024-01-10 00:00:00
Tasks: Denoising, Image Segmentation, Semantic Segmentation

In this paper, we propose a novel locally statistical variational active contour model based on I-divergence-TV denoising model, which hybrides geodesic active contour (GAC) model with active contours without edges (ACWE) model, and can be used to segment images corrupted by multiplicative gamma noise. By adding a diffusion term into the level set evolution (LSE) equation of the proposed model, we construct a reaction-diffusion (RD) equation, which can gradually regularize the level set function (LSF) to be piecewise constant in each segment domain and gain the stable solution. We further transform the proposed model into classic ROF model by adding a proximity term. Inspired by a fast denoising algorithm proposed by Jia-Zhao recently, we propose two fast fixed point algorithms to solve SAR image segmentation question. Experimental results for real SAR images show that the proposed image segmentation model can efficiently stop the contours at weak or blurred edges, and can automatically detect the exterior and interior boundaries of images with multiplicative gamma noise. The proposed FPRD1/FPRD2 models are about 1/2 (or less than) of the time required for the SBRD model based on the Split Bregman technique.
Multispectral Stereo-Image Fusion for 3D Hyperspectral Scene Reconstruction
Authors: Eric L. Wisotzky, Jost Triller, Anna Hilsmann, Peter Eisert
Date: 2023-12-15 00:00:00
Tasks: 3D Reconstruction

Spectral imaging enables the analysis of optical material properties that are invisible to the human eye. Different spectral capturing setups, e.g., based on filter-wheel, push-broom, line-scanning, or mosaic cameras, have been introduced in the last years to support a wide range of applications in agriculture, medicine, and industrial surveillance. However, these systems often suffer from different disadvantages, such as lack of real-time capability, limited spectral coverage or low spatial resolution. To address these drawbacks, we present a novel approach combining two calibrated multispectral real-time capable snapshot cameras, covering different spectral ranges, into a stereo-system. Therefore, a hyperspectral data-cube can be continuously captured. The combined use of different multispectral snapshot cameras enables both 3D reconstruction and spectral analysis. Both captured images are demosaicked avoiding spatial resolution loss. We fuse the spectral data from one camera into the other to receive a spatially and spectrally high resolution video stream. Experiments demonstrate the feasibility of this approach and the system is investigated with regard to its applicability for surgical assistance monitoring.
RoleCraft-GLM: Advancing Personalized Role-Playing in Large Language Models
Authors: Meiling Tao, Xuechen Liang, Tianyu Shi, Lei Yu, Yiting Xie
Date: 2023-12-17 00:00:00
Tasks: Language Modeling, Language Modelling

This study presents RoleCraft-GLM, an innovative framework aimed at enhancing personalized role-playing with Large Language Models (LLMs). RoleCraft-GLM addresses the key issue of lacking personalized interactions in conversational AI, and offers a solution with detailed and emotionally nuanced character portrayals. We contribute a unique conversational dataset that shifts from conventional celebrity-centric characters to diverse, non-celebrity personas, thus enhancing the realism and complexity of language modeling interactions. Additionally, our approach includes meticulous character development, ensuring dialogues are both realistic and emotionally resonant. The effectiveness of RoleCraft-GLM is validated through various case studies, highlighting its versatility and skill in different scenarios. Our framework excels in generating dialogues that accurately reflect characters' personality traits and emotions, thereby boosting user engagement. In conclusion, RoleCraft-GLM marks a significant leap in personalized AI interactions, and paves the way for more authentic and immersive AI-assisted role-playing experiences by enabling more nuanced and emotionally rich dialogues
Online Handbook of Argumentation for AI: Volume 4
Authors: Lars Bengel, Lydia Blümel, Elfia Bezou-Vrakatseli, Federico Castagna, Giulia D'Agostino, Isabelle Kuhlmann, Jack Mumford, Daphne Odekerken, Fabrizio Russo, Stefan Sarkadi, Madeleine Waller, Andreas Xydis
Date: 2023-12-20 00:00:00

This volume contains revised versions of the papers selected for the fourth volume of the Online Handbook of Argumentation for AI (OHAAI). Previously, formal theories of argument and argument interaction have been proposed and studied, and this has led to the more recent study of computational models of argument. Argumentation, as a field within artificial intelligence (AI), is highly relevant for researchers interested in symbolic representations of knowledge and defeasible reasoning. The purpose of this handbook is to provide an open access and curated anthology for the argumentation research community. OHAAI is designed to serve as a research hub to keep track of the latest and upcoming PhD-driven research on the theory and application of argumentation in all areas related to AI.
Tumbug: A pictorial, universal knowledge representation method
Authors: Mark A. Atkins
Date: 2023-12-22 00:00:00
Tasks: Attribute

Since the key to artificial general intelligence (AGI) is commonly believed to be commonsense reasoning (CSR) or, roughly equivalently, discovery of a knowledge representation method (KRM) that is particularly suitable for CSR, the author developed a custom KRM for CSR. This novel KRM called Tumbug was designed to be pictorial in nature because there exists increasing evidence that the human brain uses some pictorial type of KRM, and no well-known prior research in AGI has researched this KRM possibility. Tumbug is somewhat similar to Roger Schank's Conceptual Dependency (CD) theory, but Tumbug is pictorial and uses about 30 components based on fundamental concepts from the sciences and human life, in contrast to CD theory, which is textual and uses about 17 components (= 6 Primitive Conceptual Categories + 11 Primitive Acts) based mainly on human-oriented activities. All the Building Blocks of Tumbug were found to generalize to only five Basic Building Blocks that exactly correspond to the three components {O, A, V} of traditional Object-Attribute-Value representation plus two new components {C, S}, which are Change and System. Collectively this set of five components, called "SCOVA," seems to be a universal foundation for all knowledge representation.
Reasoning with random sets: An agenda for the future
Authors: Fabio Cuzzolin
Date: 2023-12-19 00:00:00
Tasks: Learning Theory

In this paper, we discuss a potential agenda for future work in the theory of random sets and belief functions, touching upon a number of focal issues: the development of a fully-fledged theory of statistical reasoning with random sets, including the generalisation of logistic regression and of the classical laws of probability; the further development of the geometric approach to uncertainty, to include general random sets, a wider range of uncertainty measures and alternative geometric representations; the application of this new theory to high-impact areas such as climate change, machine learning and statistical learning theory.