pwc-archive/papers-with-abstracts huggingface dataset:
RSAFormer: A method of polyp segmentation with region self-attention transformer
Authors: Xuehui Yin, Jun Zeng, Tianxiao Hou, Chao Tang, Chenquan Gan, Deepak Kumar Jain, Salvador García
Date: 2024-04-01 00:00:00
Proceeding: Computers in Biology and Medicine 2024 4
Tasks: Decoder, Medical Image Segmentation

Colonoscopy has attached great importance to early screening and clinical diagnosis of colon cancer. It remains a challenging task to achieve fine segmentation of polyps. However, existing State-of-the-art models still have limited segmentation ability due to the lack of clear and highly similar boundaries between normal tissue and polyps. To deal with this problem, we propose a region self-attention enhancement network (RSAFormer) with a transformer encoder to capture more robust features. Different from other excellent methods, RSAFormer uniquely employs a dual decoder structure to generate various feature maps. Contrasting with traditional methods that typically employ a single decoder, it offers more flexibility and detail in feature extraction. RSAFormer also introduces a region self-attention enhancement module (RSA) to acquire more accurate feature information and foster a stronger interplay between low-level and high-level features. This module enhances uncertain areas to extract more precise boundary information, these areas being signified by regional context. Extensive experiments were conducted on five prevalent polyp datasets to demonstrate RSAFormer’s proficiency. It achieves 92.2% and 83.5% mean Dice on Kvasir and ETIS, respectively, which outperformed most of the state-of-the-art models.
Technical Report for CVPR 2024 WeatherProof Dataset Challenge: Semantic Segmentation on Paired Real Data
Authors: Guojin Cao, Jiaxu Li, Jia He, Ying Min, Yunhao Zhang
Date: 2024-06-09 00:00:00
Tasks: Semantic Segmentation

This technical report presents the implementation details of 2nd winning for CVPR'24 UG2 WeatherProof Dataset Challenge. This challenge aims at semantic segmentation of images degraded by various degrees of weather from all around the world. We addressed this problem by introducing a pre-trained large-scale vision foundation model: InternImage, and trained it using images with different levels of noise. Besides, we did not use additional datasets in the training procedure and utilized dense-CRF as post-processing in the final testing procedure. As a result, we achieved 2nd place in the challenge with 45.1 mIOU and fewer submissions than the other winners.
MambaNUT: Nighttime UAV Tracking via Mamba-based Adaptive Curriculum Learning
Authors: You Wu, Xiangyang Yang, Xucheng Wang, Hengzhou Ye, Dan Zeng, Shuiwang Li
Date: 2024-12-01 00:00:00
Tasks: Domain Adaptation, Image Enhancement, Mamba

Harnessing low-light enhancement and domain adaptation, nighttime UAV tracking has made substantial strides. However, over-reliance on image enhancement, limited high-quality nighttime data, and a lack of integration between daytime and nighttime trackers hinder the development of an end-to-end trainable framework. Additionally, current ViT-based trackers demand heavy computational resources due to their reliance on the self-attention mechanism. In this paper, we propose a novel pure Mamba-based tracking framework (MambaNUT) that employs a state space model with linear complexity as its backbone, incorporating a single-stream architecture that integrates feature learning and template-search coupling within Vision Mamba. We introduce an adaptive curriculum learning (ACL) approach that dynamically adjusts sampling strategies and loss weights, thereby improving the model's ability of generalization. Our ACL is composed of two levels of curriculum schedulers: (1) sampling scheduler that transforms the data distribution from imbalanced to balanced, as well as from easier (daytime) to harder (nighttime) samples; (2) loss scheduler that dynamically assigns weights based on the size of the training set and IoU of individual instances. Exhaustive experiments on multiple nighttime UAV tracking benchmarks demonstrate that the proposed MambaNUT achieves state-of-the-art performance while requiring lower computational costs. The code will be available at https://github.com/wuyou3474/MambaNUT.
Nonparametric Teaching for Multiple Learners
Authors: Chen Zhang, Xiaofeng Cao, Weiyang Liu, Ivor Tsang, James Kwok
Date: 2023-11-17 00:00:00
Proceeding: NeurIPS 2023 11

We study the problem of teaching multiple learners simultaneously in the nonparametric iterative teaching setting, where the teacher iteratively provides examples to the learner for accelerating the acquisition of a target concept. This problem is motivated by the gap between current single-learner teaching setting and the real-world scenario of human instruction where a teacher typically imparts knowledge to multiple students. Under the new problem formulation, we introduce a novel framework -- Multi-learner Nonparametric Teaching (MINT). In MINT, the teacher aims to instruct multiple learners, with each learner focusing on learning a scalar-valued target model. To achieve this, we frame the problem as teaching a vector-valued target model and extend the target model space from a scalar-valued reproducing kernel Hilbert space used in single-learner scenarios to a vector-valued space. Furthermore, we demonstrate that MINT offers significant teaching speed-up over repeated single-learner teaching, particularly when the multiple learners can communicate with each other. Lastly, we conduct extensive experiments to validate the practicality and efficiency of MINT.
T-3DGS: Removing Transient Objects for 3D Scene Reconstruction
Authors: Vadim Pryadilshchikov, Alexander Markin, Artem Komarichev, Ruslan Rakhimov, Peter Wonka, Evgeny Burnaev
Date: 2024-11-29 00:00:00
Tasks: 3DGS, 3D Scene Reconstruction

We propose a novel framework to remove transient objects from input videos for 3D scene reconstruction using Gaussian Splatting. Our framework consists of the following steps. In the first step, we propose an unsupervised training strategy for a classification network to distinguish between transient objects and static scene parts based on their different training behavior inside the 3D Gaussian Splatting reconstruction. In the second step, we improve the boundary quality and stability of the detected transients by combining our results from the first step with an off-the-shelf segmentation method. We also propose a simple and effective strategy to track objects in the input video forward and backward in time. Our results show an improvement over the current state of the art in existing sparsely captured datasets and significant improvements in a newly proposed densely captured (video) dataset. More results and code are available at https://transient-3dgs.github.io.
Video Set Distillation: Information Diversification and Temporal Densification
Authors: Yinjie Zhao, Heng Zhao, Bihan Wen, Yew-Soon Ong, Joey Tianyi Zhou
Date: 2024-11-28 00:00:00
Tasks: Dataset Distillation

The rapid development of AI models has led to a growing emphasis on enhancing their capabilities for complex input data such as videos. While large-scale video datasets have been introduced to support this growth, the unique challenges of reducing redundancies in video \textbf{sets} have not been explored. Compared to image datasets or individual videos, video \textbf{sets} have a two-layer nested structure, where the outer layer is the collection of individual videos, and the inner layer contains the correlations among frame-level data points to provide temporal information. Video \textbf{sets} have two dimensions of redundancies: within-sample and inter-sample redundancies. Existing methods like key frame selection, dataset pruning or dataset distillation are not addressing the unique challenge of video sets since they aimed at reducing redundancies in only one of the dimensions. In this work, we are the first to study Video Set Distillation, which synthesizes optimized video data by jointly addressing within-sample and inter-sample redundancies. Our Information Diversification and Temporal Densification (IDTD) method jointly reduces redundancies across both dimensions. This is achieved through a Feature Pool and Feature Selectors mechanism to preserve inter-sample diversity, alongside a Temporal Fusor that maintains temporal information density within synthesized videos. Our method achieves state-of-the-art results in Video Dataset Distillation, paving the way for more effective redundancy reduction and efficient AI model training on video datasets.
Masking The Bias : From Echo Chambers to Large Scale Aspect-Based Sentiment Analysis
Authors: Yeonjung Lee, Yusuf Mücahit Çetinkaya, Emre Külah, İsmail Hakkı Toroslu, Hasan Davulcu
Date: 2024-09-02 00:00:00
Proceeding: IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) 2024 9
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Community Detection, Sentiment Analysis, Sentiment Classification

Aspect-based sentiment analysis (ABSA) is a natural language processing (NLP) task, ascribing precise sentiment linkages to specific entities and issues in text data. This paper addresses critical shortcomings in current ABSA methods, particularly the issues of limited aspects, training set biases, and lack of comprehensive stance-coded datasets. First, we develop a scalable MaskedABSA approach that masks aspect terms in training sentences to enable unbiased sentiment inference from the context alone. We show that the proposed method surpasses the state-of-the-art solutions in accuracy for the aspect term sentiment classification task, as verified by the SemEval datasets. Furthermore, we tackle the perennial challenges of limited training resources and the prohibitive costs of manual annotation in ABSA dataset creation by introducing an innovative weak supervision technique capitalizing on the inherent community clustering properties found within social media datasets. We utilize community detection algorithms to partition a share network into polarized groups with homogeneous adversarial stances, allowing large-scale aspect-based sentiment analysis dataset curation without labor intensive manual labeling. Our methodology is also validated using a real-world polarized dataset comprising diverse aspects and stances to showcase its efficacy and scalability.
A Surprisingly Simple Approach to Generalized Few-Shot Semantic Segmentation
Authors: Tomoya Sakai, Haoxiang Qiu, Takayuki Katsuki, Daiki Kimura, Takayuki Osogami, Tadanobu Inoue
Date: 2024-09-26 00:00:00
Proceeding: NeurIPS 2024 9
Tasks: Few-Shot Semantic Segmentation, Generalized Few-Shot Semantic Segmentation, Inductive Learning, Segmentation, Semantic Segmentation

The goal of generalized few-shot semantic segmentation (GFSS) is to recognize novel-class objects through training with a few annotated examples and the base-class model that learned the knowledge about the base classes. Unlike the classic few-shot semantic segmentation, GFSS aims to classify pixels into both base and novel classes, meaning it is a more practical setting. Current GFSS methods rely on several techniques such as using combinations of customized modules, carefully designed loss functions, meta-learning, and transductive learning. However, we found that a simple rule and standard supervised learning substantially improve the GFSS performance. In this paper, we propose a simple yet effective method for GFSS that does not use the techniques mentioned above. Also, we theoretically show that our method perfectly maintains the segmentation performance of the base-class model over most of the base classes. Through numerical experiments, we demonstrated the effectiveness of our method. It improved in novel-class segmentation performance in the 1-shot scenario by 6.1% on the PASCAL-$5^i$ dataset, 4.7% on the PASCAL-$10^i$ dataset, and 1.0% on the COCO-$20^i$ dataset.
PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusion
Authors: Avinash Paliwal, Xilong Zhou, Andrii Tsarov, Nima Khademi Kalantari
Date: 2024-12-06 00:00:00
Tasks: 3D Scene Reconstruction, Depth Estimation

In this paper, we present PanoDreamer, a novel method for producing a coherent 360{\deg} 3D scene from a single input image. Unlike existing methods that generate the scene sequentially, we frame the problem as single-image panorama and depth estimation. Once the coherent panoramic image and its corresponding depth are obtained, the scene can be reconstructed by inpainting the small occluded regions and projecting them into 3D space. Our key contribution is formulating single-image panorama and depth estimation as two optimization tasks and introducing alternating minimization strategies to effectively solve their objectives. We demonstrate that our approach outperforms existing techniques in single-image 360{\deg} 3D scene reconstruction in terms of consistency and overall quality.
DrIFT: Autonomous Drone Dataset with Integrated Real and Synthetic Data, Flexible Views, and Transformed Domains
Authors: Fardad Dadboud, Hamid Azad, Varun Mehta, Miodrag Bolic, Iraj Mntegh
Date: 2024-12-06 00:00:00
Tasks: Domain Adaptation, Unsupervised Domain Adaptation

Dependable visual drone detection is crucial for the secure integration of drones into the airspace. However, drone detection accuracy is significantly affected by domain shifts due to environmental changes, varied points of view, and background shifts. To address these challenges, we present the DrIFT dataset, specifically developed for visual drone detection under domain shifts. DrIFT includes fourteen distinct domains, each characterized by shifts in point of view, synthetic-to-real data, season, and adverse weather. DrIFT uniquely emphasizes background shift by providing background segmentation maps to enable background-wise metrics and evaluation. Our new uncertainty estimation metric, MCDO-map, features lower postprocessing complexity, surpassing traditional methods. We use the MCDO-map in our uncertainty-aware unsupervised domain adaptation method, demonstrating superior performance to SOTA unsupervised domain adaptation techniques. The dataset is available at: https://github.com/CARG-uOttawa/DrIFT.git.
Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery Games
Authors: Ryota Nonomura, Hiroki Mori
Date: 2024-12-06 00:00:00
Tasks: Logical Reasoning

Multi-agent systems utilizing large language models (LLMs) have shown great promise in achieving natural dialogue. However, smooth dialogue control and autonomous decision making among agents still remain challenges. In this study, we focus on conversational norms such as adjacency pairs and turn-taking found in conversation analysis and propose a new framework called "Murder Mystery Agents" that applies these norms to AI agents' dialogue control. As an evaluation target, we employed the "Murder Mystery" game, a reasoning-type table-top role-playing game that requires complex social reasoning and information manipulation. In this game, players need to unravel the truth of the case based on fragmentary information through cooperation and bargaining. The proposed framework integrates next speaker selection based on adjacency pairs and a self-selection mechanism that takes agents' internal states into account to achieve more natural and strategic dialogue. To verify the effectiveness of this new approach, we analyzed utterances that led to dialogue breakdowns and conducted automatic evaluation using LLMs, as well as human evaluation using evaluation criteria developed for the Murder Mystery game. Experimental results showed that the implementation of the next speaker selection mechanism significantly reduced dialogue breakdowns and improved the ability of agents to share information and perform logical reasoning. The results of this study demonstrate that the systematics of turn-taking in human conversation are also effective in controlling dialogue among AI agents, and provide design guidelines for more advanced multi-agent dialogue systems.
Compound Gaussian Radar Clutter Model With Positive Tempered Alpha-Stable Texture
Authors: Xingxing Liao, Junhao Xie, Jie zhou
Date: 2024-12-06 00:00:00

The compound Gaussian (CG) family of distributions has achieved great success in modeling sea clutter. This work develops a flexible-tailed CG model to improve generality in clutter modeling, by introducing the positive tempered $\alpha$-stable (PT$\alpha$S) distribution to model clutter texture. The PT$\alpha$S distribution exhibits widely tunable tails by tempering the heavy tails of the positive $\alpha$-stable (P$\alpha$S) distribution, thus providing greater flexibility in texture modeling. Specifically, we first develop a bivariate isotropic CG-PT$\alpha$S complex clutter model that is defined by an explicit characteristic function, based on which the corresponding amplitude model is derived. Then, we prove that the amplitude model can be expressed as a scale mixture of Rayleighs, just as the successful compound K and Pareto models. Furthermore, a characteristic function-based method is developed to estimate the parameters of the amplitude model. Finally, real-world sea clutter data analysis indicates the amplitude model's flexibility in modeling clutter data with various tail behaviors.
Two stages domain invariant representation learners solve the large co-variate shift in unsupervised domain adaptation with two dimensional data domains
Authors: Hisashi Oshima, Tsuyoshi Ishizone, Tomoyuki Higuchi
Date: 2024-12-06 00:00:00
Tasks: Domain Adaptation, Representation Learning, Unsupervised Domain Adaptation

Recent developments in the unsupervised domain adaptation (UDA) enable the unsupervised machine learning (ML) prediction for target data, thus this will accelerate real world applications with ML models such as image recognition tasks in self-driving. Researchers have reported the UDA techniques are not working well under large co-variate shift problems where e.g. supervised source data consists of handwritten digits data in monotone color and unsupervised target data colored digits data from the street view. Thus there is a need for a method to resolve co-variate shift and transfer source labelling rules under this dynamics. We perform two stages domain invariant representation learning to bridge the gap between source and target with semantic intermediate data (unsupervised). The proposed method can learn domain invariant features simultaneously between source and intermediate also intermediate and target. Finally this achieves good domain invariant representation between source and target plus task discriminability owing to source labels. This induction for the gradient descent search greatly eases learning convergence in terms of classification performance for target data even when large co-variate shift. We also derive a theorem for measuring the gap between trained models and unsupervised target labelling rules, which is necessary for the free parameters optimization. Finally we demonstrate that proposing method is superiority to previous UDA methods using 4 representative ML classification datasets including 38 UDA tasks. Our experiment will be a basis for challenging UDA problems with large co-variate shift.
Anamorphic transformation and its application to time-bandwidth compression
Authors: Mohammad H. Asghari, Bahram Jalali
Date: 2013-08-19 00:00:00
Tasks: Data Compression

A general method for compressing the modulation time-bandwidth product of analog signals is introduced and experimentally demonstrated. As one of its applications, this physics-based signal grooming performs feature-selective stretch, enabling a conventional digitizer to capture fast temporal features that were beyond its bandwidth. At the same time, the total digital data size is reduced. The compression is lossless and is achieved through a same-domain transformation of the signal's complex field, performed in the analog domain prior to digitization. Our method is inspired by operation of Fovea centralis in the human eye and by anamorphic transformation in visual arts. The proposed transform can also be performed in the digital domain as a digital data compression algorithm to alleviate the storage and transmission bottlenecks associated with "big data".
Stag-1: Towards Realistic 4D Driving Simulation with Video Generation Model
Authors: Lening Wang, Wenzhao Zheng, Dalong Du, Yunpeng Zhang, Yilong Ren, Han Jiang, Zhiyong Cui, Haiyang Yu, Jie zhou, Jiwen Lu, Shanghang Zhang
Date: 2024-12-06 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, Scene Understanding, Video Generation

4D driving simulation is essential for developing realistic autonomous driving simulators. Despite advancements in existing methods for generating driving scenes, significant challenges remain in view transformation and spatial-temporal dynamic modeling. To address these limitations, we propose a Spatial-Temporal simulAtion for drivinG (Stag-1) model to reconstruct real-world scenes and design a controllable generative network to achieve 4D simulation. Stag-1 constructs continuous 4D point cloud scenes using surround-view data from autonomous vehicles. It decouples spatial-temporal relationships and produces coherent keyframe videos. Additionally, Stag-1 leverages video generation models to obtain photo-realistic and controllable 4D driving simulation videos from any perspective. To expand the range of view generation, we train vehicle motion videos based on decomposed camera poses, enhancing modeling capabilities for distant scenes. Furthermore, we reconstruct vehicle camera trajectories to integrate 3D points across consecutive views, enabling comprehensive scene understanding along the temporal dimension. Following extensive multi-level scene training, Stag-1 can simulate from any desired viewpoint and achieve a deep understanding of scene evolution under static spatial-temporal conditions. Compared to existing methods, our approach shows promising performance in multi-view scene consistency, background coherence, and accuracy, and contributes to the ongoing advancements in realistic autonomous driving simulation. Code: https://github.com/wzzheng/Stag.
MotionFlow: Attention-Driven Motion Transfer in Video Diffusion Models
Authors: Tuna Han Salih Meral, Hidir Yesiltepe, Connor Dunlop, Pinar Yanardag
Date: 2024-12-06 00:00:00

Text-to-video models have demonstrated impressive capabilities in producing diverse and captivating video content, showcasing a notable advancement in generative AI. However, these models generally lack fine-grained control over motion patterns, limiting their practical applicability. We introduce MotionFlow, a novel framework designed for motion transfer in video diffusion models. Our method utilizes cross-attention maps to accurately capture and manipulate spatial and temporal dynamics, enabling seamless motion transfers across various contexts. Our approach does not require training and works on test-time by leveraging the inherent capabilities of pre-trained video diffusion models. In contrast to traditional approaches, which struggle with comprehensive scene changes while maintaining consistent motion, MotionFlow successfully handles such complex transformations through its attention-based mechanism. Our qualitative and quantitative experiments demonstrate that MotionFlow significantly outperforms existing models in both fidelity and versatility even during drastic scene alterations.