pwc-archive/papers-with-abstracts huggingface dataset:
Polyphonic Piano Music Transcription System Exploiting Mutual Correlations of Different Musical Note States
Authors: Taehyeon Kim, Donghyeon Lee, Man-Je Kim, Chang Wook Ahn
Date: 2024-07-08 00:00:00
Proceeding: IEEE Access 2024 7
Tasks: Information Retrieval, Music Information Retrieval, Music Transcription

Generally, polyphonic piano music transcription systems are designed to estimate and determine pitch activities along with various note states for each audio frame. While the music transcription system has multiple uses in the Music Information Retrieval (MIR) field, due to the complicated structures of the note events, precisely predicting various note states is still regarded as a challenging task. Accordingly, approaches to designing neural network architectures have evolved to facilitate the joint prediction of each note state. However, recent models have not been able to efficiently exploit mutual correlations among different note states. The key contribution of our work is that we verified mutual correlations between the different note states and reflected them in the model architecture. It enables the transcription system to recognize clearer note events and produce high-quality real-world results. We propose a kernel-sharing feature extractor module for exploiting those mutual correlations in the feature extraction step. Moreover, to make a system recognize the shape of the pitch envelope, we added some connections between the note state-specific detector modules in the note state detection step. The efficacy of our architecture was thoroughly validated in a series of experiments using the publicly available MAESTRO datasets proposed by Google Magenta. Furthermore, ablation studies are performed to demonstrate notions of those mutual correlations and show the impact and significance of the suggested approach.
ASQuery: A Query-based Model for Action Segmentation
Authors: Gan Ziliang, Jin Lei, Nie Lei, Zheng Wang, Li Zhou, Li Liang, Wang Zhecan, Li Jianshu, Xing Junliang, Zhao Jian
Date: 2024-09-30 00:00:00
Proceeding: 2024 IEEE International Conference on Multimedia and Expo (ICME) 2024 9
Tasks: Action Segmentation, Decoder, model, Segmentation, Temporal Action Segmentation

For the task of temporal action segmentation, existing works commonly treat it as a frame-wise classification problem. In this paper, we propose a straight but effective model namely ASQuery by learning central representation of each action category, which transforms the classification problem to the similarity calculation between category-specific queries and frame features. These central representations are dynamically generated through our Transformer decoder module, endowing them more flexible and comprehensive perception of the whole video. Moreover, we first introduce the boundary query for refining segmentation results, aiding to alleviating the troublesome over-segmentation problem. ASQuery demonstrates superior performance compared to state-of-the-art models, achieving improvements of 0.9% and 4.1% in the mean metrics on two public action segmentation datasets, i.e., Breakfast and Assembly101, respectively. The source codes are available at https://github.com/zlngan/ASQuery.
Work in Progress: Linear Transformers for TinyML
Authors: Moritz Scherer, Cristian Cioflan, Michele Magno, Luca Benini
Date: 2024-03-25 00:00:00
Proceeding: Design, Automation & Test in Europe Conference & Exhibition (DATE) 2024 3
Tasks: Keyword Spotting, Keyword Spotting on Google Speech Commands, Quantization

We present the WaveFormer, a neural network architecture based on a linear attention transformer to enable long sequence inference for TinyML devices. Waveformer achieves a new state-of-the-art accuracy of 98.8 % and 99.1 % on the Google Speech V2 keyword spotting (KWS) dataset for the 12 and 35 class problems with only 130 kB of weight storage, compatible with MCU class devices. Top-1 accuracy is improved by 0.1 and 0.9 percentage points while reducing the model size and number of operations by 2.5× and 4.7× compared to the state of the art. We also propose a hardware-friendly 8-bit integer quantization algorithm for the linear attention operator, enabling efficient deployment on low-cost, ultra-low-power microcontrollers without loss of accuracy.
Decomposed Meta-Learning for Few-Shot Sequence Labeling
Authors: Tingting Ma, Qianhui Wu, Huiqiang Jiang, Jieru Lin, Börje F. Karlsson, Tiejun Zhao, Chin-Yew Lin
Date: 2024-03-04 00:00:00
Proceeding: IEEE/ACM Transactions on Audio, Speech, and Language Processing (Volume: 32) 2024 3
Tasks: Entity Typing, Event Detection, Few-shot NER, Meta-Learning, Miscellaneous, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Understanding, Part-Of-Speech Tagging, POS Tagging, Slot Filling

Few-shot sequence labeling is a general problem formulation for many natural language understanding tasks in data-scarcity scenarios, which require models to generalize to new types via only a few labeled examples. Recent advances mostly adopt metric-based meta--earning and thus face the challenges of modeling the miscellaneous Other prototype and the inability to generalize to classes with large domain gaps. To overcome these challenges, we propose a decomposed meta-learning framework for few-shot sequence labeling that breaks down the task into few-shot mention detection and few-shot type classification, and sequentially tackles them via meta-learning. Specifically, we employ model-agnostic meta-learning (MAML) to prompt the mention detection model to learn boundary knowledge shared across types. With the detected mention spans, we further leverage the MAML-enhanced span-level prototypical network for few-shot type classification. In this way, the decomposition framework bypasses the requirement of modeling the miscellaneous Other prototype. Meanwhile, the adoption of the MAML algorithm enables us to explore the knowledge contained in
support examples more efficiently, so that our model can quickly adapt to new types using only a few labeled examples. Under our framework, we explore a basic implementation that uses two separate models for the two subtasks. We further propose a joint model to reduce model size and inference time, making our framework more applicable for scenarios with limited resources. Extensive experiments on nine benchmark datasets, including named entity recognition, slot tagging, event detection, and part-of-speech tagging, show that the proposed approach achieves start-of-the-art performance across various few-shot sequence labeling tasks.
A method for detecting dead fish on large water surfaces based on improved YOLOv10
Authors: Qingbin Tian, Yukang Huo, Mingyuan Yao, Haihua Wang
Date: 2024-08-31 00:00:00

Dead fish frequently appear on the water surface due to various factors. If not promptly detected and removed, these dead fish can cause significant issues such as water quality deterioration, ecosystem damage, and disease transmission. Consequently, it is imperative to develop rapid and effective detection methods to mitigate these challenges. Conventional methods for detecting dead fish are often constrained by manpower and time limitations, struggling to effectively manage the intricacies of aquatic environments. This paper proposes an end-to-end detection model built upon an enhanced YOLOv10 framework, designed specifically to swiftly and precisely detect deceased fish across extensive water surfaces.Key enhancements include: (1) Replacing YOLOv10's backbone network with FasterNet to reduce model complexity while maintaining high detection accuracy; (2) Improving feature fusion in the Neck section through enhanced connectivity methods and replacing the original C2f module with CSPStage modules; (3) Adding a compact target detection head to enhance the detection performance of smaller objects. Experimental results demonstrate significant improvements in P(precision), R(recall), and AP(average precision) compared to the baseline model YOLOv10n. Furthermore, our model outperforms other models in the YOLO series by significantly reducing model size and parameter count, while sustaining high inference speed and achieving optimal AP performance. The model facilitates rapid and accurate detection of dead fish in large-scale aquaculture systems. Finally, through ablation experiments, we systematically analyze and assess the contribution of each model component to the overall system performance.
Learning what to Monitor: using Machine Learning to Improve Past STL Monitoring
Authors: Andrea Brunello, Luca Geatti, Angelo Montanari, Nicola Saccomanno
Date: 2024-08-03 00:00:00
Proceeding: IJCAI International Joint Conference on Artificial Intelligence 2024 8
Tasks: Fault Detection

Monitoring is a runtime verification technique that can be used to check whether an execution of a system (trace) satisfies or not a given set of properties. Compared to other formal verification techniques, e.g., model checking, one needs to specify the properties to be monitored, but a complete model of the system is no longer necessary. First, we introduce the pure past fragment of Signal Temporal Logic (ppSTL), and we use it to define the monitorable safety (G(ppSTL)) and cosafety (F(ppSTL)) fragments of STL, which properly extend the commonly-used bounded-future fragment. Then, we devise a multi-objective genetic programming algorithm to automatically extend the set of properties to monitor on the basis of the history of failure traces collected over time. The framework resulting from the integration of the monitor and the learning algorithm is then experimentally validated on various public datasets. The outcomes of the experimentation confirm the effectiveness of the proposed solution.
PixelBytes: Catching Unified Representation for Multimodal Generation
Authors: Fabien Furfaro
Date: 2024-09-16 00:00:00
Tasks: Mamba, multimodal generation, Representation Learning, State Space Models

This report presents PixelBytes, an approach for unified multimodal representation learning. Drawing inspiration from sequence models like Image Transformers, PixelCNN, and Mamba-Bytes, we explore integrating text, audio, action-state, and pixelated images (sprites) into a cohesive representation. We conducted experiments on a PixelBytes Pokemon dataset and an Optimal-Control dataset. Our investigation covered various model architectures, including Recurrent Neural Networks (RNNs), State Space Models (SSMs), and Attention-based models, with a focus on bidirectional processing and our PxBy embedding technique. We evaluated models based on data reduction strategies and autoregressive learning, specifically examining Long Short-Term Memory (LSTM) networks in predictive and autoregressive modes. Our results indicate that autoregressive models perform better than predictive models in this context. Additionally, we found that diffusion models can be applied to control problems and parallelized generation. PixelBytes aims to contribute to the development of foundation models for multimodal data processing and generation. The project's code, models, and datasets are available online.
Adaptive Open-Set Active Learning with Distance-Based Out-of-Distribution Detection for Robust Task-Oriented Dialog System
Authors: Sai Keerthana Goruganthu, Roland R. Oruche, Prasad Calyam
Date: 2024-09-01 00:00:00
Proceeding: Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue 2024 9
Tasks: Active Learning, Informativeness, intent-classification, Intent Classification, Out-of-Distribution Detection

The advancements in time-efficient data collection techniques such as active learning (AL) has become salient for user intent classification performance in task-oriented dialog systems (TODS). In realistic settings, however, traditional AL techniques often fail to efficiently select targeted in-distribution (IND) data when encountering newly acquired out-of-distribution (OOD) user intents in the unlabeled pool. In this paper, we introduce a novel AL framework viz., AOSAL for TODS that combines a distance-based OOD detector using adaptive false positive rate threshold with an informativeness measure (e.g., entropy) to strategically select informative IND data points in the unlabeled pool. Specifically, we utilize the adaptive OOD detector to classify and filter out OOD samples from the unlabeled pool, then prioritize the acquisition of classified IND instances based on their informativeness scores. To validate our approach, we conduct experiments that display our framework’s flexibility and performance over multiple distance-based approaches and informativeness measures against deep AL baselines on benchmark text datasets. The results suggest that our AOSAL approach consistently outperforms the baselines on IND classification and OOD detection, advancing knowledge on improving robustness of task-oriented dialog systems.
Similarity Learning with neural networks
Authors: Gabriel Sanfins, Fabio Ramos, Danilo Naiff
Date: 2024-09-26 00:00:00

In this work, we introduce a neural network algorithm designed to automatically identify similarity relations from data. By uncovering these similarity relations, our network approximates the underlying physical laws that relate dimensionless quantities to their dimensionless variables and coefficients. Additionally, we develop a linear algebra framework, accompanied by code, to derive the symmetry groups associated with these similarity relations. While our approach is general, we illustrate its application through examples in fluid mechanics, including laminar Newtonian and non-Newtonian flows in smooth pipes, as well as turbulent flows in both smooth and rough pipes. Such examples are chosen to highlight the framework's capability to handle both simple and intricate cases, and further validates its effectiveness in discovering underlying physical laws from data.
Toward generalizable learning of all (linear) first-order methods via memory augmented Transformers
Authors: Sanchayan Dutta, Suvrit Sra
Date: 2024-10-08 00:00:00
Tasks: All, Mixture-of-Experts, Test-time Adaptation

We show that memory-augmented Transformers can implement the entire class of linear first-order methods (LFOMs), a class that contains gradient descent (GD) and more advanced methods such as conjugate gradient descent (CGD), momentum methods and all other variants that linearly combine past gradients. Building on prior work that studies how Transformers simulate GD, we provide theoretical and empirical evidence that memory-augmented Transformers can learn more advanced algorithms. We then take a first step toward turning the learned algorithms into actually usable methods by developing a mixture-of-experts (MoE) approach for test-time adaptation to out-of-distribution (OOD) samples. Lastly, we show that LFOMs can themselves be treated as learnable algorithms, whose parameters can be learned from data to attain strong performance.
Boosting the Performance of Decentralized Federated Learning via Catalyst Acceleration
Authors: Qinglun Li, Miao Zhang, Yingqi Liu, Quanjun Yin, Li Shen, Xiaochun Cao
Date: 2024-10-09 00:00:00
Tasks: Federated Learning

Decentralized Federated Learning has emerged as an alternative to centralized architectures due to its faster training, privacy preservation, and reduced communication overhead. In decentralized communication, the server aggregation phase in Centralized Federated Learning shifts to the client side, which means that clients connect with each other in a peer-to-peer manner. However, compared to the centralized mode, data heterogeneity in Decentralized Federated Learning will cause larger variances between aggregated models, which leads to slow convergence in training and poor generalization performance in tests. To address these issues, we introduce Catalyst Acceleration and propose an acceleration Decentralized Federated Learning algorithm called DFedCata. It consists of two main components: the Moreau envelope function, which primarily addresses parameter inconsistencies among clients caused by data heterogeneity, and Nesterov's extrapolation step, which accelerates the aggregation phase. Theoretically, we prove the optimization error bound and generalization error bound of the algorithm, providing a further understanding of the nature of the algorithm and the theoretical perspectives on the hyperparameter choice. Empirically, we demonstrate the advantages of the proposed algorithm in both convergence speed and generalization performance on CIFAR10/100 with various non-iid data distributions. Furthermore, we also experimentally verify the theoretical properties of DFedCata.
A Utility-Mining-Driven Active Learning Approach for Analyzing Clickstream Sequences
Authors: Danny Y. C. Wang, Lars Arne Jordanger, Jerry Chun-Wei Lin
Date: 2024-10-09 00:00:00
Tasks: Active Learning, Sequential Pattern Mining

In rapidly evolving e-commerce industry, the capability of selecting high-quality data for model training is essential. This study introduces the High-Utility Sequential Pattern Mining using SHAP values (HUSPM-SHAP) model, a utility mining-based active learning strategy to tackle this challenge. We found that the parameter settings for positive and negative SHAP values impact the model's mining outcomes, introducing a key consideration into the active learning framework. Through extensive experiments aimed at predicting behaviors that do lead to purchases or not, the designed HUSPM-SHAP model demonstrates its superiority across diverse scenarios. The model's ability to mitigate labeling needs while maintaining high predictive performance is highlighted. Our findings demonstrate the model's capability to refine e-commerce data processing, steering towards more streamlined, cost-effective prediction modeling.
The trade-off between data minimization and fairness in collaborative filtering
Authors: Nasim Sonboli, Sipei Li, Mehdi Elahi, Asia Biega
Date: 2024-09-21 00:00:00
Tasks: Active Learning, Collaborative Filtering, Fairness, Recommendation Systems

General Data Protection Regulations (GDPR) aim to safeguard individuals' personal information from harm. While full compliance is mandatory in the European Union and the California Privacy Rights Act (CPRA), it is not in other places. GDPR requires simultaneous compliance with all the principles such as fairness, accuracy, and data minimization. However, it overlooks the potential contradictions within its principles. This matter gets even more complex when compliance is required from decision-making systems. Therefore, it is essential to investigate the feasibility of simultaneously achieving the goals of GDPR and machine learning, and the potential tradeoffs that might be forced upon us. This paper studies the relationship between the principles of data minimization and fairness in recommender systems. We operationalize data minimization via active learning (AL) because, unlike many other methods, it can preserve a high accuracy while allowing for strategic data collection, hence minimizing the amount of data collection. We have implemented several active learning strategies (personalized and non-personalized) and conducted a comparative analysis focusing on accuracy and fairness on two publicly available datasets. The results demonstrate that different AL strategies may have different impacts on the accuracy of recommender systems with nearly all strategies negatively impacting fairness. There has been no to very limited work on the trade-off between data minimization and fairness, the pros and cons of active learning methods as tools for implementing data minimization, and the potential impacts of AL on fairness. By exploring these critical aspects, we offer valuable insights for developing recommender systems that are GDPR compliant.
Dual Stream Graph Transformer Fusion Networks for Enhanced Brain Decoding
Authors: Lucas Goene, Siamak Mehrkanoon
Date: 2024-09-23 00:00:00
Tasks: Brain Decoding, Graph Attention

This paper presents the novel Dual Stream Graph-Transformer Fusion (DS-GTF) architecture designed specifically for classifying task-based Magnetoencephalography (MEG) data. In the spatial stream, inputs are initially represented as graphs, which are then passed through graph attention networks (GAT) to extract spatial patterns. Two methods, TopK and Thresholded Adjacency are introduced for initializing the adjacency matrix used in the GAT. In the temporal stream, the Transformer Encoder receives concatenated windowed input MEG data and learns new temporal representations. The learned temporal and spatial representations from both streams are fused before reaching the output layer. Experimental results demonstrate an enhancement in classification performance and a reduction in standard deviation across multiple test subjects compared to other examined models.
Designing Pre-training Datasets from Unlabeled Data for EEG Classification with Transformers
Authors: Tim Bary, Benoit Macq
Date: 2024-09-23 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Seizure Detection

Transformer neural networks require a large amount of labeled data to train effectively. Such data is often scarce in electroencephalography, as annotations made by medical experts are costly. This is why self-supervised training, using unlabeled data, has to be performed beforehand. In this paper, we present a way to design several labeled datasets from unlabeled electroencephalogram (EEG) data. These can then be used to pre-train transformers to learn representations of EEG signals. We tested this method on an epileptic seizure forecasting task on the Temple University Seizure Detection Corpus using a Multi-channel Vision Transformer. Our results suggest that 1) Models pre-trained using our approach demonstrate significantly faster training times, reducing fine-tuning duration by more than 50% for the specific task, and 2) Pre-trained models exhibit improved accuracy, with an increase from 90.93% to 92.16%, as well as a higher AUC, rising from 0.9648 to 0.9702 when compared to non-pre-trained models.
Curb Your Attention: Causal Attention Gating for Robust Trajectory Prediction in Autonomous Driving
Authors: Ehsan Ahmadi, Ray Mercurius, Soheil Alizadeh, Kasra Rezaee, Amir Rasouli
Date: 2024-09-23 00:00:00
Tasks: Autonomous Driving, Causal Discovery, Prediction, Trajectory Prediction

Trajectory prediction models in autonomous driving are vulnerable to perturbations from non-causal agents whose actions should not affect the ego-agent's behavior. Such perturbations can lead to incorrect predictions of other agents' trajectories, potentially compromising the safety and efficiency of the ego-vehicle's decision-making process. Motivated by this challenge, we propose $\textit{Causal tRajecTory predICtion}$ $\textbf{(CRiTIC)}$, a novel model that utilizes a $\textit{Causal Discovery Network}$ to identify inter-agent causal relations over a window of past time steps. To incorporate discovered causal relationships, we propose a novel $\textit{Causal Attention Gating}$ mechanism to selectively filter information in the proposed Transformer-based architecture. We conduct extensive experiments on two autonomous driving benchmark datasets to evaluate the robustness of our model against non-causal perturbations and its generalization capacity. Our results indicate that the robustness of predictions can be improved by up to $\textbf{54%}$ without a significant detriment to prediction accuracy. Lastly, we demonstrate the superior domain generalizability of the proposed model, which achieves up to $\textbf{29%}$ improvement in cross-domain performance. These results underscore the potential of our model to enhance both robustness and generalization capacity for trajectory prediction in diverse autonomous driving domains. Further details can be found on our project page: https://ehsan-ami.github.io/critic.