pwc-archive/papers-with-abstracts huggingface dataset:
Statistical Mechanics of Support Vector Regression
Authors: Abdulkadir Canatar, SueYeon Chung
Date: 2024-12-06 00:00:00
Tasks: regression

A key problem in deep learning and computational neuroscience is relating the geometrical properties of neural representations to task performance. Here, we consider this problem for continuous decoding tasks where neural variability may affect task precision. Using methods from statistical mechanics, we study the average-case learning curves for $\varepsilon$-insensitive Support Vector Regression ($\varepsilon$-SVR) and discuss its capacity as a measure of linear decodability. Our analysis reveals a phase transition in the training error at a critical load, capturing the interplay between the tolerance parameter $\varepsilon$ and neural variability. We uncover a double-descent phenomenon in the generalization error, showing that $\varepsilon$ acts as a regularizer, both suppressing and shifting these peaks. Theoretical predictions are validated both on toy models and deep neural networks, extending the theory of Support Vector Machines to continuous tasks with inherent neural variability.
Towards Brain Passage Retrieval -- An Investigation of EEG Query Representations
Authors: Niall McGuire, Yashar Moshfeghi
Date: 2024-12-09 00:00:00
Tasks: cross-modal alignment, EEG, Information Retrieval, Passage Retrieval, Retrieval, Translation

Information Retrieval (IR) systems primarily rely on users' ability to translate their internal information needs into (text) queries. However, this translation process is often uncertain and cognitively demanding, leading to queries that incompletely or inaccurately represent users' true needs. This challenge is particularly acute for users with ill-defined information needs or physical impairments that limit traditional text input, where the gap between cognitive intent and query expression becomes even more pronounced. Recent neuroscientific studies have explored Brain-Machine Interfaces (BMIs) as a potential solution, aiming to bridge the gap between users' cognitive semantics and their search intentions. However, current approaches attempting to decode explicit text queries from brain signals have shown limited effectiveness in learning robust brain-to-text representations, often failing to capture the nuanced semantic information present in brain patterns. To address these limitations, we propose BPR (Brain Passage Retrieval), a novel framework that eliminates the need for intermediate query translation by enabling direct retrieval of relevant passages from users' brain signals. Our approach leverages dense retrieval architectures to map EEG signals and text passages into a shared semantic space. Through comprehensive experiments on the ZuCo dataset, we demonstrate that BPR achieves up to 8.81% improvement in precision@5 over existing EEG-to-text baselines, while maintaining effectiveness across 30 participants. Our ablation studies reveal the critical role of hard negative sampling and specialised brain encoders in achieving robust cross-modal alignment. These results establish the viability of direct brain-to-passage retrieval and provide a foundation for developing more natural interfaces between users' cognitive states and IR systems.
Learning Cluster Representatives for Approximate Nearest Neighbor Search
Authors: Thomas Vecchiato
Date: 2024-12-08 00:00:00
Tasks: Clustering, Information Retrieval, Learning-To-Rank

Developing increasingly efficient and accurate algorithms for approximate nearest neighbor search is a paramount goal in modern information retrieval. A primary approach to addressing this question is clustering, which involves partitioning the dataset into distinct groups, with each group characterized by a representative data point. By this method, retrieving the top-k data points for a query requires identifying the most relevant clusters based on their representatives -- a routing step -- and then conducting a nearest neighbor search within these clusters only, drastically reducing the search space. The objective of this thesis is not only to provide a comprehensive explanation of clustering-based approximate nearest neighbor search but also to introduce and delve into every aspect of our novel state-of-the-art method, which originated from a natural observation: The routing function solves a ranking problem, making the function amenable to learning-to-rank. The development of this intuition and applying it to maximum inner product search has led us to demonstrate that learning cluster representatives using a simple linear function significantly boosts the accuracy of clustering-based approximate nearest neighbor search.
ULMRec: User-centric Large Language Model for Sequential Recommendation
Authors: Minglai Shao, Hua Huang, Qiyao Peng, Hongtao Liu
Date: 2024-12-07 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Quantization, Sequential Recommendation

Recent advances in Large Language Models (LLMs) have demonstrated promising performance in sequential recommendation tasks, leveraging their superior language understanding capabilities. However, existing LLM-based recommendation approaches predominantly focus on modeling item-level co-occurrence patterns while failing to adequately capture user-level personalized preferences. This is problematic since even users who display similar behavioral patterns (e.g., clicking or purchasing similar items) may have fundamentally different underlying interests. To alleviate this problem, in this paper, we propose ULMRec, a framework that effectively integrates user personalized preferences into LLMs for sequential recommendation. Considering there has the semantic gap between item IDs and LLMs, we replace item IDs with their corresponding titles in user historical behaviors, enabling the model to capture the item semantics. For integrating the user personalized preference, we design two key components: (1) user indexing: a personalized user indexing mechanism that leverages vector quantization on user reviews and user IDs to generate meaningful and unique user representations, and (2) alignment tuning: an alignment-based tuning stage that employs comprehensive preference alignment tasks to enhance the model's capability in capturing personalized information. Through this design, ULMRec achieves deep integration of language semantics with user personalized preferences, facilitating effective adaptation to recommendation. Extensive experiments on two public datasets demonstrate that ULMRec significantly outperforms existing methods, validating the effectiveness of our approach.
Visualization of Knowledge Graphs with Embeddings: an Essay on Recent Trends and Methods
Authors: Davide Riva, Cristina Rossetti
Date: 2024-11-21 00:00:00
Tasks: Graph Embedding, Knowledge Graph Embedding, Knowledge Graph Embeddings, Knowledge Graphs, Relation

In this essay we discuss the recent trends in visual analysis and exploration of Knowledge Graphs, particularly in conjunction with Knowledge Graph Embedding techniques. We present an overview of the current state of visualization techniques and frameworks for KGs, in relation to four identified challenges. The challenges in visualizing Knowledge Graphs include the need for intuitive and modular interfaces, performance in handling big data, and difficulties for users in understanding and using query languages. We find frameworks that generally satisfy the intuitive UI, performance, and query support requirements, but few satisfying the modularity requirement. In the context of Knowledge Graph Embeddings, we divide the approaches that use embeddings to facilitate exploration of Knowledge Graphs from those that aim at the explanation of the embeddings themselves. We find significant differences between the two perspectives. Finally, we highlight some possible directions for future work, including diffusion of the unmet requirements, implementation of new visual features, and experimentation with relation visualization as a peculiar element of Knowledge Graphs.
FogETex: Fog Computing Framework for Electronic Textile Applications
Authors: Kadir Ozlem, Asli Tuncay Atalay, Ozgur Atalay, Gökhan Ince
Date: 2024-11-04 00:00:00
Proceeding: IEEE Internet of Things Journal 2024 11

Textile products are present in almost every aspect of human life. With the introduction of electronic textiles, textile products have become capable of converting various physiological and environmental stimuli into electrical signals, many of which are of vital importance to humans. Therefore, these products require real-time (low-latency) and robust computing systems. However, due to comfort considerations, they cannot accommodate powerful computing resources. In this study, a novel fog computing-based framework (FogETex) is proposed to meet the needs of e-textile applications. FogETex is a Platform-as-a-Service model that is cross-platform supported, scalable, and operates in real-time. This framework encompasses end-to-end integration of the system including textile-based Internet of Things (T-IoT) device, fog devices, and the cloud. Fog devices consist of a broker that manages the fog node and a worker that handles incoming computation requests. Sensor data is transmitted to the fog node through a mobile application, and system architecture can be monitored through developed user interfaces. Resource usage from broker devices is monitored in real time to prevent worker devices from experiencing overload. For the system case study, a deep learning-based gait phase analysis application using textile-based capacitive sensors is employed. FogETex was evaluated in terms of time characteristics, resource usage, and network bandwidth usage using a mock client to determine the ideal system performance and an actual client to conduct real-world tests. The fog devices outperformed the cloud system in these metrics. Besides being developed primarily for electronic textile applications, FogETex framework can accommodate other IoT devices as well.
TALEC: Teach Your LLM to Evaluate in Specific Domain with In-house Criteria by Criteria Division and Zero-shot Plus Few-shot
Authors: Kaiqi Zhang, Shuai Yuan, Honghan Zhao
Date: 2024-06-25 00:00:00
Tasks: In-Context Learning, Text Generation

With the rapid development of large language models (LLM), the evaluation of LLM becomes increasingly important. Measuring text generation tasks such as summarization and article creation is very difficult. Especially in specific application domains (e.g., to-business or to-customer service), in-house evaluation criteria have to meet not only general standards (correctness, helpfulness and creativity, etc.) but also specific needs of customers and business security requirements at the same time, making the evaluation more difficult. So far, the evaluation of LLM in business scenarios has mainly relied on manual, which is expensive and time-consuming. In this paper, we propose a model-based evaluation method: TALEC, which allows users to flexibly set their own evaluation criteria, and uses in-context learning (ICL) to teach judge model these in-house criteria. In addition, we try combining zero-shot and few-shot to make the judge model focus on more information. We also propose a prompt paradigm and an engineering approach to adjust and iterate the shots ,helping judge model to better understand the complex criteria. We then compare fine-tuning with ICL, finding that fine-tuning can be replaced by ICL. TALEC demonstrates a strong capability to accurately reflect human preferences and achieves a correlation of over 80% with human judgments, outperforming even the inter-human correlation in some tasks. The code is released in https://github.com/zlkqz/auto_eval
StackEval: Benchmarking LLMs in Coding Assistance
Authors: Nidhish Shah, Zulkuf Genc, Dogu Araci
Date: 2024-11-21 00:00:00
Tasks: Benchmarking

We present two comprehensive benchmarks to evaluate the performance of language models in coding assistance tasks, covering code writing, debugging, code review, and conceptual understanding. Our main contribution includes two curated datasets: StackEval, a large-scale benchmark derived from Stack Overflow questions, and StackUnseen, a dynamic benchmark featuring the most recent Stack Overflow content. These benchmarks offer novel insights into the capabilities and limitations of LLMs, particularly in handling new and emerging content. Additionally, we assess LLMs' proficiency as judges for coding tasks using a curated, human-annotated dataset, exploring their evaluation capabilities and potential biases, including whether they favor their own generated solutions. Our findings underscore the potential of these benchmarks to advance LLM development and application in coding assistance. To ensure reproducibility, we publicly share our datasets and evaluation code at https://github.com/ProsusAI/stack-eval .
Benchmarking framework for machine learning classification from fNIRS data
Authors: Johann Benerradi, Jeremie Clos, Aleksandra Landowska, Michel F. Valstar, Max L. Wilson
Date: 2023-03-03 00:00:00
Proceeding: Frontiers in Neuroergonomics 2023 3
Tasks: Benchmarking, Brain Computer Interface, Classification

Background: While efforts to establish best practices with functional near infrared spectroscopy (fNIRS) signal processing have been published, there are still no community standards for applying machine learning to fNIRS data. Moreover, the lack of open source benchmarks and standard expectations for reporting means that published works often claim high generalisation capabilities, but with poor practices or missing details in the paper. These issues make it hard to evaluate the performance of models when it comes to choosing them for brain-computer interfaces.

Methods: We present an open-source benchmarking framework, BenchNIRS, to establish a best practice machine learning methodology to evaluate models applied to fNIRS data, using five open access datasets for brain-computer interface (BCI) applications. The BenchNIRS framework, using a robust methodology with nested cross-validation, enables researchers to optimise models and evaluate them without bias. The framework also enables us to produce useful metrics and figures to detail the performance of new models for comparison. To demonstrate the utility of the framework, we present a benchmarking of six baseline models [linear discriminant analysis (LDA), support-vector machine (SVM), k-nearest neighbours (kNN), artificial neural network (ANN), convolutional neural network (CNN), and long short-term memory (LSTM)] on the five datasets and investigate the influence of different factors on the classification performance, including: number of training examples and size of the time window of each fNIRS sample used for classification. We also present results with a sliding window as opposed to simple classification of epochs, and with a personalised approach (within subject data classification) as opposed to a generalised approach (unseen subject data classification).

Results and discussion: Results show that the performance is typically lower than the scores often reported in literature, and without great differences between models, highlighting that predicting unseen data remains a difficult task. Our benchmarking framework provides future authors, who are achieving significant high classification scores, with a tool to demonstrate the advances in a comparable way. To complement our framework, we contribute a set of recommendations for methodology decisions and writing papers, when applying machine learning to fNIRS data.
Exploring machine learning approaches for classifying mental workload using fNIRS data from HCI tasks
Authors: Johann Benerradi, Horia A. Maior, Adrian Marinescu, Jeremie Clos, Max L. Wilson
Date: 2019-11-19 00:00:00
Proceeding: Halfway to the Future 2019 11
Tasks: EEG

Functional Near-Infrared Spectroscopy (fNIRS) has shown promise for being potentially more suitable (than e.g. EEG) for brain-based Human Computer Interaction (HCI). While some machine learning approaches have been used in prior HCI work, this paper explores different approaches and configurations for classifying Mental Workload (MWL) from a continuous HCI task, to identify and understand potential limitations and data processing decisions. In particular, we investigate three overall approaches: a logistic regression method, a supervised shallow method (SVM), and a supervised deep learning method (CNN). We examine personalised and generalised models, as well as consider different features and ways of labelling the data. Our initial explorations show that generalised models can perform as well as personalised ones and that deep learning can be a suitable approach for medium size datasets. To provide additional practical advice for future brain-computer interaction systems, we conclude by discussing the limitations and data-preparation needs of different machine learning approaches. We also make recommendations for avenues of future work that are most promising for the machine learning of fNIRS data.
NeuroXAI: Adaptive, robust, explainable surrogate framework for determination of channel importance in EEG application
Authors: Choel-Hui Lee, Daesun Ahn, Hakseung Kim, Eun Jin Ha, Jung-Bin Kim, Dong-Joo Kim
Date: 2025-09-12 00:00:00
Proceeding: Expert Systems with Applications 2025 9
Tasks: channel selection, EEG, Electroencephalogram (EEG), Explainable Artificial Intelligence (XAI), Motor Imagery

Electroencephalogram (EEG)-based applications often require numerous channels to achieve high performance, which limits their widespread use. Various channel selection methods have been proposed to identify minimum EEG channels without compromising performance. However, most methods are limited to specific data paradigms or prediction models. We propose NeuroXAI, a novel method that identifies channel importance regardless of the type of EEG application. It integrates the surrogate analysis algorithm to optimize EEG signals and the data sampling algorithm, which effectively selects from highly voluminous EEG data. The efficacy of channel selection via the proposed method was evaluated through three datasets acquired under different paradigms (motor imagery, steady-state visually evoked potentials, and event-related potentials). On datasets based on these paradigms, NeuroXAI-based channel selection reduced the number of channels while maintaining or enhancing performance. The advantages of the proposed method include enhanced performance, robustness over varying data paradigms and the type of prediction model. The XAI technique enables intuitive interpretation of the constructed model operation, making it applicable in various fields such as model debugging and model interpretation. NeuroXAI has the potential to be used as a practical tool to develop better EEG applications.
Deep transfer learning method based on automatic domain alignment and moment matching
Authors: Jingui Zhang, Chuangji Meng, Cunlu Xu, Jingyong Ma, Wei Su
Date: 2022-07-21 00:00:00
Proceeding: Mathematics 2022 7
Tasks: Domain Adaptation, Image Classification, Transfer Learning

Domain discrepancy is a key research problem in the field of deep domain adaptation. Two main strategies are used to reduce the discrepancy: the parametric method and the nonparametric method. Both methods have achieved good results in practical applications. However, research on whether the combination of the two can further reduce domain discrepancy has not been conducted. Therefore, in this paper, a deep transfer learning method based on automatic domain alignment and moment matching (DA-MM) is proposed. First, an automatic domain alignment layer is embedded in the front of each domain-specific layer of a neural network structure to preliminarily align the source and target domains. Then, a moment matching measure (such as MMD distance) is added between every domain-specific layer to map the source and target domain features output by the alignment layer to a common reproduced Hilbert space. The results of an extensive experimental analysis over several public benchmarks show that DA-MM can reduce the distribution discrepancy between the two domains and improve the domain adaptation performance.
EMPATH: MediaPipe-Aided Ensemble Learning with Attention-Based Transformers for Accurate Recognition of Bangla Word-Level Sign Language
Authors: Kazi Reyazul Hasan, Muhammad Abdullah Adnan
Date: 2024-12-04 00:00:00
Proceeding: International Conference on Pattern Recognition 2024 12
Tasks: Ensemble Learning, Sign Language Recognition

In this paper, we introduce EMPATH, an advanced computational framework developed to substantially enhance the recognition of Bangla Sign Language (BdSL). By integrating Ensemble Learning, MediaPipe Holistic for gesture tracking, and an Attention-based Transformer model, EMPATH addresses the challenges of sign language interpretation, setting new accuracy benchmarks and significantly surpassing previous records: achieving 79.81% on SignBD-Word-90 (previously best at 66.05%), 70.58% on SignBD-Word (previously best at 57%), and 99.25% on BdSL40 (previously best at 89%).

A pioneering feature of EMPATH is its innovative interpolation model, built to overcome the limitations posed by missing Hand keypoints of MediaPipe. Validated in both EMPATH and a basic MLP framework, this model showcases remarkable versatility across architectures. EMPATH strategically selects its preprocessing and postprocessing techniques, optimizing each for maximum impact on accuracy and performance.

Extensively trained across various word-level datasets beyond Bangla, including INCLUDE-50, INCLUDE, WLASL-100, and the Malaysian Sign Language Medical Dataset, EMPATH demonstrates its broad applicability and global potential. This diverse training scheme establishes superior accuracy benchmarks: achieving an impressive 100% on INCLUDE-50, 94.67% on INCLUDE, and 93.46% on the MSL Medical Dataset.

Through EMPATH, we aspire to bridge communication barriers for the deaf and hard-of-hearing communities, showcasing the profound impact of integrating advanced technological solutions to tackle the complexities of sign language recognition.

Source code is available at https://github.com/kreyazulh/EMPATH.
Deep convolution stack for waveform in underwater acoustic target recognition
Authors: Shengzhao Tian1, Duanbing Chen1, 2, 3, Hang Wang1 & Jingfa Liu4, 5*
Date: 2021-05-05 00:00:00
Proceeding: journal 2021 5

In underwater acoustic target recognition, deep learning methods have been proved to be effective
on recognizing original signal waveform. Previous methods often utilize large convolutional kernels
to extract features at the beginning of neural networks. It leads to a lack of depth and structural
imbalance of networks. The power of nonlinear transformation brought by deep network has not been
fully utilized. Deep convolution stack is a kind of network frame with flexible and balanced structure
and it has not been explored well in underwater acoustic target recognition, even though such frame
has been proven to be effective in other deep learning fields. In this paper, a multiscale residual unit
(MSRU) is proposed to construct deep convolution stack network. Based on MSRU, a multiscale
residual deep neural network (MSRDN) is presented to classify underwater acoustic target. Dataset
acquired in a real-world scenario is used to verify the proposed unit and model. By adding MSRU into
Generative Adversarial Networks, the validity of MSRU is proved. Finally, MSRDN achieves the best
recognition accuracy of 83.15%, improved by 6.99% from the structure related networks which take
the original signal waveform as input and 4.48% from the networks which take the time-frequency
representation as input.