pwc-archive/papers-with-abstracts huggingface dataset:
An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models
Authors: Qi Liu, Gang Guo, Jiaxin Mao, Zhicheng Dou, Ji-Rong Wen, Hao Jiang, Xinyu Zhang, Zhao Cao
Date: 2024-03-20 00:00:00
Tasks: Retrieval

With the development of pre-trained language models, the dense retrieval models have become promising alternatives to the traditional retrieval models that rely on exact match and sparse bag-of-words representations. Different from most dense retrieval models using a bi-encoder to encode each query or document into a dense vector, the recently proposed late-interaction multi-vector models (i.e., ColBERT and COIL) achieve state-of-the-art retrieval effectiveness by using all token embeddings to represent documents and queries and modeling their relevance with a sum-of-max operation. However, these fine-grained representations may cause unacceptable storage overhead for practical search systems. In this study, we systematically analyze the matching mechanism of these late-interaction models and show that the sum-of-max operation heavily relies on the co-occurrence signals and some important words in the document. Based on these findings, we then propose several simple document pruning methods to reduce the storage overhead and compare the effectiveness of different pruning methods on different late-interaction models. We also leverage query pruning methods to further reduce the retrieval latency. We conduct extensive experiments on both in-domain and out-domain datasets and show that some of the used pruning methods can significantly improve the efficiency of these late-interaction models without substantially hurting their retrieval effectiveness.
Improving Legal Case Retrieval with Brain Signals
Authors: Ruizhe Zhang, Qingyao Ai, Ziyi Ye, Yueyue Wu, Xiaohui Xie, Yiqun Liu
Date: 2024-03-20 00:00:00
Tasks: EEG, Retrieval

The tasks of legal case retrieval have received growing attention from the IR community in the last decade. Relevance feedback techniques with implicit user feedback (e.g., clicks) have been demonstrated to be effective in traditional search tasks (e.g., Web search). In legal case retrieval, however, collecting relevance feedback faces a couple of challenges that are difficult to resolve under existing feedback paradigms. First, legal case retrieval is a complex task as users often need to understand the relationship between legal cases in detail to correctly judge their relevance. Traditional feedback signal such as clicks is too coarse to use as they do not reflect any fine-grained relevance information. Second, legal case documents are usually long, users often need even tens of minutes to read and understand them. Simple behavior signal such as clicks and eye-tracking fixations can hardly be useful when users almost click and examine every part of the document. In this paper, we explore the possibility of solving the feedback problem in legal case retrieval with brain signal. Recent advances in brain signal processing have shown that human emotional can be collected in fine grains through Brain-Machine Interfaces (BMI) without interrupting the users in their tasks. Therefore, we propose a framework for legal case retrieval that uses EEG signal to optimize retrieval results. We collected and create a legal case retrieval dataset with users EEG signal and propose several methods to extract effective EEG features for relevance feedback. Our proposed features achieve a 71% accuracy for feedback prediction with an SVM-RFE model, and our proposed ranking method that takes into account the diverse needs of users can significantly improve user satisfaction for legal case retrieval. Experiment results show that re-ranked result list make user more satisfied.
How Features Benefit: Parallel Series Embedding for Multivariate Time Series Forecasting with Transformer
Authors: Xuande Feng, Zonglin Lyu
Date: 2022-10-31 00:00:00
Proceeding: ICTAI 2022 10
Tasks: Multivariate Time Series Forecasting, Time Series, Time Series Forecasting

Forecasting time series is an engaging and vital
mathematical topic. Theories and applications in related fields
have been studied for decades, and deep learning has provided
reliable tools in recent years. Transformer, capable to capture
longer sequence dependencies, was exploited as a powerful architecture in time series forecasting. While existing work majorly
contributed to breaking memory bottleneck of Trasnformer, how
to effectively leverage multivariate time series remains barely
focused. In this work, a novel architecture utilizing a primary
Transformer is proposed to conduct multivariate time series
predictions. Our proposed architecture has two main advantages. Firstly, it accurately predicts multivariate time series with
shorter or longer sequence lengths and steps. We benchmark
our proposed model with various baseline architectures on realworld datasets, and our model improved their performances
significantly. Secondly, it can easily be leveraged in Transformerbased variants,
ViT-MUL: A Baseline Study on Recent Machine Unlearning Methods Applied to Vision Transformers
Authors: Ikhyun Cho, Changyeon Park, Julia Hockenmaier
Date: 2024-02-07 00:00:00
Tasks: Machine Unlearning

Machine unlearning (MUL) is an arising field in machine learning that seeks to erase the learned information of specific training data points from a trained model. Despite the recent active research in MUL within computer vision, the majority of work has focused on ResNet-based models. Given that Vision Transformers (ViT) have become the predominant model architecture, a detailed study of MUL specifically tailored to ViT is essential. In this paper, we present comprehensive experiments on ViTs using recent MUL algorithms and datasets. We anticipate that our experiments, ablation studies, and findings could provide valuable insights and inspire further research in this field.
Distributed Cooperative Formation Control of Nonlinear Multi-Agent System (UGV) Using Neural Network
Authors: Si Kheang Moeurn
Date: 2024-03-20 00:00:00

The paper presented in this article deals with the issue of distributed cooperative formation of multi-agent systems (MASs). It proposes the use of appropriate neural network control methods to address formation requirements (uncertainties dynamic model). It considers an adaptive leader-follower distributed cooperative formation control based on neural networks (NNs) developed for a class of second-order nonlinear multi-agent systems and neural networks Neural networks are used to compute system data that inputs layer (position, velocity), hidden layers, and output layer. Through collaboration between leader-follower approaches and neural networks with complex systems or complex conditions receive an effective cooperative formation control method. The sufficient conditions for the system stability were derived using Lyapunov stability theory, graph theory, and state space methods. By simulation, the results of this study can be obtained from the main data of the multi-agent system in formation control and verified that the system can process consistency, stability, reliability, and accuracy in cooperative formation.
Adaptive Fast XGBoost for Multiclass Classification
Authors: Baldo, Fabiano; Grando, Julia; Correa, Yuji Yamada; Policarpo, Deividy Amorim
Date: 2023-10-31 00:00:00
Proceeding: JOURNAL OF INFORMATION AND DATA MANAGEMENT 2023 10
Tasks: Classification

The popularization of sensoring and connectivity technologies like 5G and IoT are boosting the generation of data streams. Such kinds of data are one of the last frontiers of data mining applications. However, data streams are massive and unbounded sequences of non-stationary data objects that are continuously generated at rapid rates. To deal with these challenges, the learning algorithms should analyze the data just once and update their classifiers to handle the concept drifts. The literature presents some algorithms to deal with the classification of multiclass data streams. However, most of them have high processing time. Therefore, this work proposes a XGBoost-based classifier called AFXGB-MC to fast classify non-stationary data streams with multiple classes. We compared it with the six state-of-the-art algorithms for multiclass classification found in the literature. The results pointed out that AFXGB-MC presents similar accuracy performance, but with faster processing time, being twice faster than the second fastest algorithm from the literature, and having fast drift recovery time.
When SMILES have Language: Drug Classification using Text Classification Methods on Drug SMILES Strings
Authors: Azmine Toushik Wasi, Šerbetar Karlo, Raima Islam, Taki Hasan Rafi, Dong-Kyu Chae
Date: 2024-03-03 00:00:00
Tasks: Classification, Sentence, text-classification, Text Classification

Complex chemical structures, like drugs, are usually defined by SMILES strings as a sequence of molecules and bonds. These SMILES strings are used in different complex machine learning-based drug-related research and representation works. Escaping from complex representation, in this work, we pose a single question: What if we treat drug SMILES as conventional sentences and engage in text classification for drug classification? Our experiments affirm the possibility with very competitive scores. The study explores the notion of viewing each atom and bond as sentence components, employing basic NLP methods to categorize drug types, proving that complex problems can also be solved with simpler perspectives. The data and code are available here: https://github.com/azminewasi/Drug-Classification-NLP.
Reading Between the Leads: Local Lead-Attention Based Classification of Electrocardiogram Signals
Authors: Gouthamaan Manimaran, Sadasivan Puthusserypady, Helena Dominguez, Jakob E Bardram
Date: 2023-12-26 00:00:00
Proceeding: Computing in Cardiology (CinC) 2023 12
Tasks: ECG Classification, Multi-class Classification

Self-attention models have emerged as powerful tools in both computer vision and Natural Language Processing (NLP) domains. However, their application in timedomain Electrocardiogram (ECG) signal analysis has been limited, primarily due to the lesser need for global receptive fields. In this study, we present a novel approach utilizing local self-attention to address multi-class classification tasks using the PhysioNet/Computing in Cardiology Challenge 2021 dataset, encompassing 26 distinct classes across six different datasets. We introduce an innovative concept called “local lead-attention” to capture features within a single lead and across multiple configurable leads. The proposed architecture achieves an F1 score of 0.521 on the challenge’s validation set, marking a 5.67% improvement over the winning solution. Remarkably, our model accomplishes this performance boost with only one-third of the total parameter size, amounting to 2.4 million parameters.
Learning Generalized Segmentation for Foggy-scenes by Bi-directional Wavelet Guidance
Authors: Qi Bi, ShaoDi You, Theo Gevers
Date: 2024-02-28 00:00:00
Proceeding: Association for the Advancement of Artificial Intelligence (AAAI) 2024 2
Tasks: Autonomous Driving, Domain Adaptation, Domain Generalization, Foggy Scene Segmentation, Scene Segmentation, Segmentation, Unsupervised Domain Adaptation

Learning scene semantics that can be well generalized to foggy conditions is important for safety-crucial applications
such as autonomous driving. Existing methods need both annotated clear images and foggy images to train a curriculum domain adaptation model. Unfortunately, these methods can only generalize to the target foggy domain that has
seen in the training stage, but the foggy domains vary a lot in both urban-scene styles and fog styles. In this paper, we propose to learn scene segmentation well generalized to foggy-scenes under the domain generalization setting, which does not involve any foggy images in the training stage and can generalize to any arbitrary unseen foggy scenes. We argue that an ideal segmentation model that can be well generalized to foggy-scenes need to simultaneously enhance the content, de-correlate the urban-scene style and de-correlate the fog style. As the content (e.g., scene semantic) rests more in low-frequency features while the style of urban-scene and fog rests more in high-frequency features, we propose a novel bi-directional wavelet guidance (BWG) mechanism to realize the above three objectives in a divideand-conquer manner. With the aid of Haar wavelet transformation, the low frequency component is concentrated on the content enhancement self-attention, while the high frequency component is shifted to the style and fog self-attention for de-correlation purpose. It is integrated into existing masklevel Transformer segmentation pipelines in a learnable fashion. Large-scale experiments are conducted on four foggyscene segmentation datasets under a variety of interesting settings. The proposed method significantly outperforms existing directly-supervised, curriculum domain adaptation and domain generalization segmentation methods. Source code is available at https://github.com/BiQiWHU/BWG.
Interactive Learning of Intrinsic and Extrinsic Properties for All-day Semantic Segmentation
Authors: Qi Bi, ShaoDi You, Theo Gevers
Date: 2023-07-07 00:00:00
Proceeding: IEEE Transactions on Image Processing 2023 7
Tasks: All, All-day Semantic Segmentation, Domain Adaptation, Semantic Segmentation

Scene appearance changes drastically throughout the day. Existing semantic segmentation methods mainly focus on well-lit daytime scenarios and are not well designed to cope with such great appearance changes. Naively using domain adaption
does not solve this problem because it usually learns a fixed mapping between the source and target domain and thus have
limited generalization capability on all-day scenarios (i.e., from dawn to night).
In this paper, in contrast to existing methods, we tackle this challenge from the perspective of image formulation itself, where the image appearance is determined by both intrinsic (e.g., semantic category, structure) and extrinsic (e.g., lighting)
properties. To this end, we propose a novel intrinsic-extrinsic interactive learning strategy. The key idea is to interact between intrinsic and extrinsic representations during the learning process under spatial-wise guidance. In this way, the intrinsic representation becomes more stable and, at the same time, the extrinsic representation gets better at depicting the changes. Consequently, the refined image representation is more robust to generate pixel-wise predictions for all-day scenarios. To achieve this, we propose an All-in-One Segmentation Network (AO-SegNet) in an end-to-end manner.
Large scale experiments are conducted on three real datasets (Mapillary, BDD100K and ACDC) and our proposed synthetic All-day CityScapes dataset. The proposed AO-SegNet shows a significant performance gain against the state-of-the-art under a variety of CNN and ViT backbones on all the datasets.
Local semantic enhanced convnet for aerial scene recognition
Authors: Qi Bi, Kun Qin, Han Zhang, Gui-Song Xia
Date: 2021-07-08 00:00:00
Proceeding: IEEE Transactions on Image Processing 2021 7
Tasks: Aerial Scene Classification, Image Classification, Scene Classification, Scene Recognition

Aerial scene recognition is challenging due to the complicated object distribution and spatial arrangement in a large-scale aerial image. Recent studies attempt to explore the local semantic representation capability of deep learning models, but how to exactly perceive the key local regions remains to be handled. In this paper, we present a local semantic enhanced ConvNet (LSE-Net) for aerial scene recognition, which mimics the human visual perception of key local regions in aerial scenes, in the hope of building a discriminative local semantic representation. Our LSE-Net consists of a context enhanced convolutional feature extractor, a local semantic perception module and a classification layer. Firstly, we design a multi-scale dilated convolution operators to fuse multi-level and multi-scale convolutional features in a trainable manner in order to fully receive the local feature responses in an aerial scene. Then, these features are fed into our two-branch local semantic perception module. In this module, we design a context-aware class peak response (CACPR) measurement to precisely depict the visual impulse of key local regions and the corresponding context information. Also, a spatial attention weight matrix is extracted to describe the importance of each key local region for the aerial scene. Finally, the refined class confidence maps are fed into the classification layer. Exhaustive experiments on three aerial scene classification benchmarks indicate that our LSE-Net achieves the state-of-the-art performance, which validates the effectiveness of our local semantic perception module and CACPR measurement.
A multiple-instance densely-connected ConvNet for aerial scene classification
Authors: Qi Bi, Kun Qin, Zhili Li, Han Zhang, Kai Xu, Gui-Song Xia
Date: 2020-03-03 00:00:00
Proceeding: IEEE Transactions on Image Processing 2020 3
Tasks: Aerial Scene Classification, Classification, Image Classification, Multiple Instance Learning, Scene Classification, Scene Recognition

In contrast with nature scenes, aerial scenes are often composed of many objects crowdedly distributed on the surface in bird’s view, the description of which usually demands more discriminative features as well as local semantics. However, when applied to scene classification, most of the existing convolution neural networks (ConvNets) tend to depict global semantics of images, and the loss of low- and mid-level features can hardly be avoided, especially when the model goes deeper. To tackle these challenges, in this paper, we propose a multiple-instance densely-connected ConvNet (MIDC-Net) for aerial scene classification. It regards aerial scene classification as a multiple-instance learning problem so that local semantics can be further investigated. Our classification model consists of an instance-level classifier, a multiple instance pooling and followed by a bag-level classification layer. In the instance-level classifier, we propose a simplified dense connection structure to effectively preserve features from different levels. The extracted convolution features are further converted into instance feature vectors. Then, we propose a trainable attention-based multiple instance pooling. It highlights the local semantics relevant to the scene label and outputs the bag-level probability directly. Finally, with our bag-level classification layer, this multiple instance learning framework is under the direct supervision of bag labels. Experiments on three widely-utilized aerial scene benchmarks demonstrate that our proposed method outperforms many state-of-the-art methods by a large margin with much fewer parameters.
Music ControlNet: A model similar to SD ControlNetD that can accurately control music generation
Authors: Wu, Shih-Lun and Donahue, Chris and Watanabe, Shinji and Bryan, Nicholas J.
Date: 2023-11-07 00:00:00
Proceeding: . 2023 11
Tasks: Music Generation, Rhythm, Text-to-Music Generation

Text-to-music generation models are now capable of generating high-quality music audio in broad styles. However, text control is primarily suitable for the manipulation of global musical attributes like genre, mood, and tempo, and is less suitable for precise control over time-varying attributes such as the positions of beats in time or the changing dynamics of the music. We propose Music ControlNet, a diffusion-based music generation model that offers multiple precise, time-varying controls over generated audio. To imbue text-to-music models with time-varying control, we propose an approach analogous to pixel-wise control of the image-domain ControlNet method. Specifically, we extract controls from training audio yielding paired data, and fine-tune a diffusion-based conditional generative model over audio spectrograms given melody, dynamics, and rhythm controls. While the image-domain Uni-ControlNet method already allows generation with any subset of controls, we devise a new strategy to allow creators to input controls that are only partially specified in time. We evaluate both on controls extracted from audio and controls we expect creators to provide, demonstrating that we can generate realistic music that corresponds to control inputs in both settings. While few comparable music generation models exist, we benchmark against MusicGen, a recent model that accepts text and melody input, and show that our model generates music that is 49% more faithful to input melodies despite having 35x fewer parameters, training on 11x less data, and enabling two additional forms of time-varying control.
All grains, one scheme (AGOS): Learning multigrain instance representation for aerial scene classification
Authors: Qi Bi, Beichen Zhou, Kun Qin, Qinghao Ye, Gui-Song Xia
Date: 2022-08-25 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2022 8
Tasks: Aerial Scene Classification, All, Multiple Instance Learning, Scene Classification

Aerial scene classification remains challenging as: 1) the size of key objects in determining the scene scheme varies greatly and 2) many objects irrelevant to the scene scheme are often flooded in the image. Hence, how to effectively perceive the region of interests (RoIs) from a variety of sizes and build more discriminative representation from such complicated object distribution is vital to understand an aerial scene. In this article, we propose a novel all grains, one scheme (AGOS) framework to tackle these challenges. To the best of our knowledge , it is the first work to extend the classic multiple instance learning (MIL) into multigrain formulation. Specifically, it consists of a multigrain perception (MGP) module, a multibranch multi-instance representation (MBMIR) module, and a self-aligned semantic fusion (SSF) module. First, our MGP module preserves the differential dilated convolutional features from the backbone, which magnifies the discriminative information from multigrains. Then, our MBMIR module highlights the key instances in the multigrain representation under the MIL formulation. Finally, our SSF module allows our framework to learn the same scene scheme from multigrain instance representations and fuses them, so that the entire framework is optimized as a whole. Notably, our AGOS is flexible and can be easily adapted to existing convolutional neural networks (CNNs) in a plug-and-play manner. Extensive experiments on UCM, aerial image dataset (AID), and Northwestern Polytechnical University (NWPU) benchmarks demonstrate that our AGOS achieves a comparable performance against the state-of-the-art methods.