pwc-archive/papers-with-abstracts huggingface dataset:
CTNet: A Convolutional Transformer Network for EEG-Based Motor Imagery Classification
Authors: Wei Zhao, Xiaolu Jiang, Baocan Zhang, Shixiao Xiao & Sujun Weng
Date: 2024-08-30 00:00:00
Proceeding: Scientific Reports 2024 8
Tasks: Brain Computer Interface, EEG, Eeg Decoding, Motor Imagery

Brain-computer interface (BCI) technology bridges the direct communication between the brain and machines, unlocking new possibilities for human interaction and rehabilitation. EEG-based motor imagery (MI) plays a pivotal role in BCI, enabling the translation of thought into actionable commands for interactive and assistive technologies. However, the constrained decoding performance of brain signals poses a limitation to the broader application and development of BCI systems. In this study, we introduce a convolutional Transformer network (CTNet) designed for EEG-based MI classification. Firstly, CTNet employs a convolutional module analogous to EEGNet, dedicated to extracting local and spatial features from EEG time series. Subsequently, it incorporates a Transformer encoder module, leveraging a multi-head attention mechanism to discern the global dependencies of EEG's high-level features. Finally, a straightforward classifier module comprising fully connected layers is followed to categorize EEG signals. In subject-specific evaluations, CTNet achieved remarkable decoding accuracies of 82.52% and 88.49% on the BCI IV-2a and IV-2b datasets, respectively. Furthermore, in the challenging cross-subject assessments, CTNet achieved recognition accuracies of 58.64% on the BCI IV-2a dataset and 76.27% on the BCI IV-2b dataset. In both subject-specific and cross-subject evaluations, CTNet holds a leading position when compared to some of the state-of-the-art methods. This underscores the exceptional efficacy of our approach and its potential to set a new benchmark in EEG decoding.
Simulasi Dinamika Fluida Dam Break 2D Menggunakan Metoda Beda Hingga Skema Forward Time Backward Scheme (FTBS)
Authors: Ilham Adi Panuntun, Rida SN Mahmudah, Restu Widiatmono, Denny Darmawan
Date: 2022-07-05 00:00:00
Proceeding: Wahana Fisika 2022 7

Dam break merupakan salah satu simulasi dasar dalam komputasi dinamika fluida yang sering digunakan sebagai benchmark. Penelitian ini bertujuan membuat simulasi dam break 2D menggunakan metoda beda hingga skema Forward Time Backward Scheme (FTBS) yang tervalidasi oleh solusi analitis dan hasil dari penelitian sebelumnya. Pada penelitian ini, dibuat solusi numerik untuk persamaan Shallow Water (SWE) atau Saint-Venant yang didasarkan pada kekekalan massa dan momentum yang diturunkan dari persamaan Navier-Stokes. Kode simulasi numerik dibuat menggunakan metoda beda hingga yang merupakan metoda paling umum dan paling sederhana untuk pemodelan dam break. Skema numerik yang digunakan adalah Forward Time Backward Scheme (FTBS). Pada tahap awal, dibuat simulasi dam break 1D yang hasilnya divalidasi dengan solusi analitik. Dari hasil analisis data, diperoleh nilai RSME yang relatif kecil, yaitu 0,0076. Selanjutnya, code tersebut dikembangkan untuk membuat simulasi dam break 2D yang hasilnya dibandingkan dengan penelitian sebelumnya. Hasil perbandingan secara kualitatif dan kuantitatif menunjukkan bahwa code simulasi 2D yang dibangun dengan FTBS dapat merepresentasikan fenomena dam break dengan cukup memuaskan. The code available on https://github.com/ilham-ap/Dam-break
Improving diabetic retinopathy screening using Artificial Intelligence: design, evaluation and before-and-after study of a custom development
Authors: Imanol Pinto, Álvaro Olazarán, David Jurío, Borja De La Osa, Miguel Sainz, Aritz Oscoz, Jerónimo Ballaz, Javier Gorricho, Mikel Galar, José Andonegui
Date: 2024-12-18 00:00:00

Background: The worst outcomes of diabetic retinopathy (DR) can be prevented by implementing DR screening programs assisted by AI. At the University Hospital of Navarre (HUN), Spain, general practitioners (GPs) grade fundus images in an ongoing DR screening program, referring to a second screening level (ophthalmologist) target patients. Methods: After collecting their requirements, HUN decided to develop a custom AI tool, called NaIA-RD, to assist their GPs in DR screening. This paper introduces NaIA-RD, details its implementation, and highlights its unique combination of DR and retinal image quality grading in a single system. Its impact is measured in an unprecedented before-and-after study that compares 19,828 patients screened before NaIA-RD's implementation and 22,962 patients screened after. Results: NaIA-RD influenced the screening criteria of 3/4 GPs, increasing their sensitivity. Agreement between NaIA-RD and the GPs was high for non-referral proposals (94.6% or more), but lower and variable (from 23.4\% to 86.6%) for referral proposals. An ophthalmologist discarded a NaIA-RD error in most of contradicted referral proposals by labeling the 93% of a sample of them as referable. In an autonomous setup, NaIA-RD would have reduced the study visualization workload by 4.27 times without missing a single case of sight-threatening DR referred by a GP. Conclusion: DR screening was more effective when supported by NaIA-RD, which could be safely used to autonomously perform the first level of screening. This shows how AI devices, when seamlessly integrated into clinical workflows, can help improve clinical pathways in the long term.
ICA-based Resting-State Networks Obtained on Large Autism fMRI Dataset ABIDE
Authors: Sjir J. C. Schielen, Jesper Pilmeyer, Albert P. Aldenkamp, Danny Ruijters, Svitlana Zinger
Date: 2024-12-18 00:00:00
Tasks: Dimensionality Reduction

Functional magnetic resonance imaging (fMRI) has become instrumental in researching brain function. One application of fMRI is investigating potential neural features that distinguish people with autism spectrum disorder (ASD) from healthy controls. The Autism Brain Imaging Data Exchange (ABIDE) facilitates this research through its extensive data-sharing initiative. While ABIDE offers data preprocessed with various atlases, independent component analysis (ICA) for dimensionality reduction remains underutilized. We address this gap by presenting ICA-based resting-state networks (RSNs) from preprocessed scans from ABIDE, now publicly available: https://github.com/SjirSchielen/groupICAonABIDE. These RSNs unveil neural activation clusters without atlas constraints, offering a perspective on ASD analyses that complements the predominantly atlas-based literature. This contribution provides a valuable resource for further research into ASD, potentially aiding in developing new analytical approaches.
Advancing Vehicle Plate Recognition: Multitasking Visual Language Models with VehiclePaliGemma
Authors: Nouar AlDahoul, Myles Joshua Toledo Tan, Raghava Reddy Tera, Hezerul Abdul Karim, Chee How Lim, Manish Kumar Mishra, Yasir Zaki
Date: 2024-12-14 00:00:00
Tasks: GPU, License Plate Recognition, Optical Character Recognition, Optical Character Recognition (OCR)

License plate recognition (LPR) involves automated systems that utilize cameras and computer vision to read vehicle license plates. Such plates collected through LPR can then be compared against databases to identify stolen vehicles, uninsured drivers, crime suspects, and more. The LPR system plays a significant role in saving time for institutions such as the police force. In the past, LPR relied heavily on Optical Character Recognition (OCR), which has been widely explored to recognize characters in images. Usually, collected plate images suffer from various limitations, including noise, blurring, weather conditions, and close characters, making the recognition complex. Existing LPR methods still require significant improvement, especially for distorted images. To fill this gap, we propose utilizing visual language models (VLMs) such as OpenAI GPT4o, Google Gemini 1.5, Google PaliGemma (Pathways Language and Image model + Gemma model), Meta Llama 3.2, Anthropic Claude 3.5 Sonnet, LLaVA, NVIDIA VILA, and moondream2 to recognize such unclear plates with close characters. This paper evaluates the VLM's capability to address the aforementioned problems. Additionally, we introduce ``VehiclePaliGemma'', a fine-tuned Open-sourced PaliGemma VLM designed to recognize plates under challenging conditions. We compared our proposed VehiclePaliGemma with state-of-the-art methods and other VLMs using a dataset of Malaysian license plates collected under complex conditions. The results indicate that VehiclePaliGemma achieved superior performance with an accuracy of 87.6\%. Moreover, it is able to predict the car's plate at a speed of 7 frames per second using A100-80GB GPU. Finally, we explored the multitasking capability of VehiclePaliGemma model to accurately identify plates containing multiple cars of various models and colors, with plates positioned and oriented in different directions.
Benchmarking Harmonized Tariff Schedule Classification Models
Authors: Bryce Judy
Date: 2024-12-04 00:00:00
Tasks: Benchmarking, Classification, Language Model Evaluation, Language Modeling, Language Modelling

The Harmonized Tariff System (HTS) classification industry, essential to e-commerce and international trade, currently lacks standardized benchmarks for evaluating the effectiveness of classification solutions. This study establishes and tests a benchmark framework for imports to the United States, inspired by the benchmarking approaches used in language model evaluation, to systematically compare prominent HTS classification tools. The framework assesses key metrics--such as speed, accuracy, rationality, and HTS code alignment--to provide a comprehensive performance comparison. The study evaluates several industry-leading solutions, including those provided by Zonos, Tarifflo, Avalara, and WCO BACUDA, identifying each tool's strengths and limitations. Results highlight areas for industry-wide improvement and innovation, paving the way for more effective and standardized HTS classification solutions across the international trade and e-commerce sectors.
Improved Semantic Inpainting Architecture Augmented with a Facial Landmark Detector
Authors: Mirza Sami, Israt Naiyer, Ehsanul Khan, Jia Uddin
Date: 2022-03-03 00:00:00
Proceeding: The International Arab Journal of Information Technology 2022 3
Tasks: Facial Inpainting, Facial Landmark Detection, Image Manipulation, SSIM

This paper presents an augmented method for image completion, particularly for images of human faces by
leveraging on deep learning based inpainting techniques. Face completion generally tend to be a daunting task because of the
relatively low uniformity of a face attributed to structures like eyes, nose, etc. Here, understanding the top level context is
paramount for proper semantic completion. The method presented improves upon existing inpainting techniques that reduce
context difference by locating the closest encoding of the damaged image in the latent space of a pre-trained deep generator.
However, these existing methods fail to consider key facial structures (eyes, nose, jawline, etc.,) and their respective location
to each other. This paper mitigates this by introducing a face landmark detector and a corresponding landmark loss. This
landmark loss is added to the construction loss between the damaged and generated image and the adversarial loss of the
generative model. The model was trained with the celeb A dataset, tools like pyamg, pillow and the OpenCV library was used
for image manipulation and facial landmark detection. There are three main weighted parameters that balance the effect of the
three loss functions in this paper, namely context loss, landmark loss and prior loss. Experimental results demonstrate that the
added landmark loss attributes to better understanding of top-level context and hence the model can generate more visually
appealing in painted images than the existing model.The model obtained average Structural Similarity Index (SSIM) and Peak
Signal-to-Noise Ratio (PNSR) scores of 0.851 and 33.448 for different orientations of the face and 0.896 and 31.473,
respectively, for various types masks.
Variational Weighting for Kernel Density Ratios
Authors: Sangwoong Yoon, Frank C. Park, Gunsu S Yun, Iljung Kim, Yung-Kyun Noh
Date: 2023-11-06 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Density Estimation

Kernel density estimation (KDE) is integral to a range of generative and discriminative tasks in machine learning. Drawing upon tools from the multidimensional calculus of variations, we derive an optimal weight function that reduces bias in standard kernel density estimates for density ratios, leading to improved estimates of prediction posteriors and information-theoretic measures. In the process, we shed light on some fundamental aspects of density estimation, particularly from the perspective of algorithms that employ KDEs as their main building blocks.
PyOD 2: A Python Library for Outlier Detection with LLM-powered Model Selection
Authors: Sihan Chen, Zhuangzhuang Qian, Wingchun Siu, Xingcan Hu, Jiaqi Li, Shawn Li, Yuehan Qin, Tiankai Yang, Zhuo Xiao, Wanghao Ye, Yichi Zhang, Yushun Dong, Yue Zhao
Date: 2024-12-11 00:00:00
Tasks: Anomaly Detection, Fraud Detection, Intrusion Detection, Large Language Model, Model Selection, Network Intrusion Detection, Outlier Detection, Recommendation Systems

Outlier detection (OD), also known as anomaly detection, is a critical machine learning (ML) task with applications in fraud detection, network intrusion detection, clickstream analysis, recommendation systems, and social network moderation. Among open-source libraries for outlier detection, the Python Outlier Detection (PyOD) library is the most widely adopted, with over 8,500 GitHub stars, 25 million downloads, and diverse industry usage. However, PyOD currently faces three limitations: (1) insufficient coverage of modern deep learning algorithms, (2) fragmented implementations across PyTorch and TensorFlow, and (3) no automated model selection, making it hard for non-experts. To address these issues, we present PyOD Version 2 (PyOD 2), which integrates 12 state-of-the-art deep learning models into a unified PyTorch framework and introduces a large language model (LLM)-based pipeline for automated OD model selection. These improvements simplify OD workflows, provide access to 45 algorithms, and deliver robust performance on various datasets. In this paper, we demonstrate how PyOD 2 streamlines the deployment and automation of OD models and sets a new standard in both research and industry. PyOD 2 is accessible at [https://github.com/yzhao062/pyod](https://github.com/yzhao062/pyod). This study aligns with the Web Mining and Content Analysis track, addressing topics such as the robustness of Web mining methods and the quality of algorithmically-generated Web data.
From Galaxy Zoo DECaLS to BASS/MzLS: detailed galaxy morphology classification with unsupervised domain adaption
Authors: Renhao Ye, Shiyin Shen, Rafael S. de Souza, Quanfeng Xu, Mi Chen, Zhu Chen, Emille E. O. Ishida, Alberto Krone-Martins, Rupesh Durgesh
Date: 2024-12-20 00:00:00
Tasks: Domain Adaptation, Morphology classification, Survey, Unsupervised Domain Adaptation

The DESI Legacy Imaging Surveys (DESI-LIS) comprise three distinct surveys: the Dark Energy Camera Legacy Survey (DECaLS), the Beijing-Arizona Sky Survey (BASS), and the Mayall z-band Legacy Survey (MzLS).The citizen science project Galaxy Zoo DECaLS 5 (GZD-5) has provided extensive and detailed morphology labels for a sample of 253,287 galaxies within the DECaLS survey. This dataset has been foundational for numerous deep learning-based galaxy morphology classification studies. However, due to differences in signal-to-noise ratios and resolutions between the DECaLS images and those from BASS and MzLS (collectively referred to as BMz), a neural network trained on DECaLS images cannot be directly applied to BMz images due to distributional mismatch.In this study, we explore an unsupervised domain adaptation (UDA) method that fine-tunes a source domain model trained on DECaLS images with GZD-5 labels to BMz images, aiming to reduce bias in galaxy morphology classification within the BMz survey. Our source domain model, used as a starting point for UDA, achieves performance on the DECaLS galaxies' validation set comparable to the results of related works. For BMz galaxies, the fine-tuned target domain model significantly improves performance compared to the direct application of the source domain model, reaching a level comparable to that of the source domain. We also release a catalogue of detailed morphology classifications for 248,088 galaxies within the BMz survey, accompanied by usage recommendations.
On the Structural Memory of LLM Agents
Authors: Ruihong Zeng, Jinyuan Fang, Siwei Liu, Zaiqiao Meng
Date: 2024-12-17 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Question Answering, Reranking, Retrieval

Memory plays a pivotal role in enabling large language model~(LLM)-based agents to engage in complex and long-term interactions, such as question answering (QA) and dialogue systems. While various memory modules have been proposed for these tasks, the impact of different memory structures across tasks remains insufficiently explored. This paper investigates how memory structures and memory retrieval methods affect the performance of LLM-based agents. Specifically, we evaluate four types of memory structures, including chunks, knowledge triples, atomic facts, and summaries, along with mixed memory that combines these components. In addition, we evaluate three widely used memory retrieval methods: single-step retrieval, reranking, and iterative retrieval. Extensive experiments conducted across four tasks and six datasets yield the following key insights: (1) Different memory structures offer distinct advantages, enabling them to be tailored to specific tasks; (2) Mixed memory structures demonstrate remarkable resilience in noisy environments; (3) Iterative retrieval consistently outperforms other methods across various scenarios. Our investigation aims to inspire further research into the design of memory systems for LLM-based agents.
Offline Safe Reinforcement Learning Using Trajectory Classification
Authors: Ze Gong, Akshat Kumar, Pradeep Varakantham
Date: 2024-12-19 00:00:00
Tasks: Classification, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Reinforcement Learning

Offline safe reinforcement learning (RL) has emerged as a promising approach for learning safe behaviors without engaging in risky online interactions with the environment. Most existing methods in offline safe RL rely on cost constraints at each time step (derived from global cost constraints) and this can result in either overly conservative policies or violation of safety constraints. In this paper, we propose to learn a policy that generates desirable trajectories and avoids undesirable trajectories. To be specific, we first partition the pre-collected dataset of state-action trajectories into desirable and undesirable subsets. Intuitively, the desirable set contains high reward and safe trajectories, and undesirable set contains unsafe trajectories and low-reward safe trajectories. Second, we learn a policy that generates desirable trajectories and avoids undesirable trajectories, where (un)desirability scores are provided by a classifier learnt from the dataset of desirable and undesirable trajectories. This approach bypasses the computational complexity and stability issues of a min-max objective that is employed in existing methods. Theoretically, we also show our approach's strong connections to existing learning paradigms involving human feedback. Finally, we extensively evaluate our method using the DSRL benchmark for offline safe RL. Empirically, our method outperforms competitive baselines, achieving higher rewards and better constraint satisfaction across a wide variety of benchmark tasks.
Tokenphormer: Structure-aware Multi-token Graph Transformer for Node Classification
Authors: Zijie Zhou, Zhaoqi Lu, Xuekai Wei, Rongqin Chen, Shenghui Zhang, Pak Lon Ip, Leong Hou U
Date: 2024-12-19 00:00:00
Tasks: Node Classification, Representation Learning

Graph Neural Networks (GNNs) are widely used in graph data mining tasks. Traditional GNNs follow a message passing scheme that can effectively utilize local and structural information. However, the phenomena of over-smoothing and over-squashing limit the receptive field in message passing processes. Graph Transformers were introduced to address these issues, achieving a global receptive field but suffering from the noise of irrelevant nodes and loss of structural information. Therefore, drawing inspiration from fine-grained token-based representation learning in Natural Language Processing (NLP), we propose the Structure-aware Multi-token Graph Transformer (Tokenphormer), which generates multiple tokens to effectively capture local and structural information and explore global information at different levels of granularity. Specifically, we first introduce the walk-token generated by mixed walks consisting of four walk types to explore the graph and capture structure and contextual information flexibly. To ensure local and global information coverage, we also introduce the SGPM-token (obtained through the Self-supervised Graph Pre-train Model, SGPM) and the hop-token, extending the length and density limit of the walk-token, respectively. Finally, these expressive tokens are fed into the Transformer model to learn node representations collaboratively. Experimental results demonstrate that the capability of the proposed Tokenphormer can achieve state-of-the-art performance on node classification tasks.
Pre-training Graph Neural Networks on Molecules by Using Subgraph-Conditioned Graph Information Bottleneck
Authors: Van Thuy Hoang; O-Joun Lee
Date: 2025-02-20 00:00:00
Tasks: Graph Classification, Graph Neural Network, Graph Regression, Molecular Property Prediction, Self-Supervised Learning

This study aims to build a pre-trained Graph Neural Network (GNN) model on molecules without human annotations or prior knowledge. Although various attempts have been proposed to overcome limitations in acquiring labeled molecules, the previous pre-training methods still rely on semantic subgraphs, i.e., functional groups. Only focusing on the functional groups could overlook the graph-level distinctions. The key challenge to build a pre-trained GNN on molecules is how to (1) generate well-distinguished graph-level representations and (2) automatically discover the functional groups without prior knowledge. To solve it, we propose a novel Subgraph-conditioned Graph Information Bottleneck, named S-CGIB, for pre-training GNNs to recognize core subgraphs (graph cores) and significant subgraphs. The main idea is that the graph cores contain compressed and sufficient information that could generate well-distinguished graph-level representations and reconstruct the input graph conditioned on significant subgraphs across molecules under the S-CGIB principle. To discover significant subgraphs without prior knowledge about functional groups, we propose generating a set of functional group candidates, i.e., ego networks, and using an attention-based interaction between the graph core and the candidates. Despite being identified from self-supervised learning, our learned subgraphs match the real-world functional groups. Extensive experiments on molecule datasets across various domains demonstrate the superiority of S-CGIB.