pwc-archive/papers-with-abstracts huggingface dataset:
Mitigating Test-Time Bias for Fair Image Retrieval
Authors: Fanjie Kong, Shuai Yuan, Weituo Hao, Ricardo Henao
Date: 2023-05-23 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Image Retrieval, Language Modeling, Language Modelling, Retrieval

We address the challenge of generating fair and unbiased image retrieval results given neutral textual queries (with no explicit gender or race connotations), while maintaining the utility (performance) of the underlying vision-language (VL) model. Previous methods aim to disentangle learned representations of images and text queries from gender and racial characteristics. However, we show these are inadequate at alleviating bias for the desired equal representation result, as there usually exists test-time bias in the target retrieval set. So motivated, we introduce a straightforward technique, Post-hoc Bias Mitigation (PBM), that post-processes the outputs from the pre-trained vision-language model. We evaluate our algorithm on real-world image search datasets, Occupation 1 and 2, as well as two large-scale image-text datasets, MS-COCO and Flickr30k. Our approach achieves the lowest bias, compared with various existing bias-mitigation methods, in text-based image retrieval result while maintaining satisfactory retrieval performance. The source code is publicly available at \url{https://anonymous.4open.science/r/Fair_Text_based_Image_Retrieval-D8B2}.
CORRECT: Context- and Reference-Augmented Reasoning and Prompting for Fact-Checking
Authors: Delvin Ce Zhang, Dongwon Lee
Date: 2025-02-09 00:00:00
Tasks: Fact Checking

Fact-checking the truthfulness of claims usually requires reasoning over multiple evidence sentences. Oftentimes, evidence sentences may not be always self-contained, and may require additional contexts and references from elsewhere to understand coreferential expressions, acronyms, and the scope of a reported finding. For example, evidence sentences from an academic paper may need contextual sentences in the paper and descriptions in its cited papers to determine the scope of a research discovery. However, most fact-checking models mainly focus on the reasoning within evidence sentences, and ignore the auxiliary contexts and references. To address this problem, we propose a novel method, Context- and Reference-augmented Reasoning and Prompting. For evidence reasoning, we construct a three-layer evidence graph with evidence, context, and reference layers. We design intra- and cross-layer reasoning to integrate three graph layers into a unified evidence embedding. For verdict prediction, we design evidence-conditioned prompt encoder, which produces unique prompt embeddings for each claim. These evidence-conditioned prompt embeddings and claims are unified for fact-checking. Experiments verify the strength of our model.
Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model
Authors: Guoqing Ma, Haoyang Huang, Kun Yan, Liangyu Chen, Nan Duan, Shengming Yin, Changyi Wan, Ranchen Ming, Xiaoniu Song, Xing Chen, Yu Zhou, Deshan Sun, Deyu Zhou, Jian Zhou, Kaijun Tan, Kang An, Mei Chen, Wei Ji, Qiling Wu, Wen Sun, Xin Han, Yanan Wei, Zheng Ge, Aojie Li, Bin Wang, Bizhu Huang, Bo wang, Brian Li, Changxing Miao, Chen Xu, Chenfei Wu, Chenguang Yu, Dapeng Shi, Dingyuan Hu, Enle Liu, Gang Yu, Ge Yang, Guanzhe Huang, Gulin Yan, Haiyang Feng, Hao Nie, Haonan Jia, Hanpeng Hu, Hanqi Chen, Haolong Yan, Heng Wang, Hongcheng Guo, Huilin Xiong, Huixin Xiong, Jiahao Gong, Jianchang Wu, Jiaoren Wu, Jie Wu, Jie Yang, Jiashuai Liu, Jiashuo Li, Jingyang Zhang, Junjing Guo, Junzhe Lin, Kaixiang Li, Lei Liu, Lei Xia, Liang Zhao, Liguo Tan, Liwen Huang, Liying Shi, Ming Li, Mingliang Li, Muhua Cheng, Na Wang, Qiaohui Chen, Qinglin He, Qiuyan Liang, Quan Sun, Ran Sun, Rui Wang, Shaoliang Pang, Shiliang Yang, Sitong Liu, SiQi Liu, Shuli Gao, Tiancheng Cao, Tianyu Wang, Weipeng Ming, Wenqing He, Xu Zhao, Xuelin Zhang, Xianfang Zeng, Xiaojia Liu, Xuan Yang, Yaqi Dai, Yanbo Yu, Yang Li, Yineng Deng, Yingming Wang, Yilei Wang, Yuanwei Lu, Yu Chen, Yu Luo, Yuchu Luo, Yuhe Yin, Yuheng Feng, Yuxiang Yang, Zecheng Tang, Zekai Zhang, Zidong Yang, Binxing Jiao, Jiansheng Chen, Jing Li, Shuchang Zhou, Xiangyu Zhang, Xinhao Zhang, Yibo Zhu, Heung-Yeung Shum, Daxin Jiang
Date: 2025-02-14 00:00:00
Tasks: Video Generation, Video Reconstruction

We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model with 30B parameters and the ability to generate videos up to 204 frames in length. A deep compression Variational Autoencoder, Video-VAE, is designed for video generation tasks, achieving 16x16 spatial and 8x temporal compression ratios, while maintaining exceptional video reconstruction quality. User prompts are encoded using two bilingual text encoders to handle both English and Chinese. A DiT with 3D full attention is trained using Flow Matching and is employed to denoise input noise into latent frames. A video-based DPO approach, Video-DPO, is applied to reduce artifacts and improve the visual quality of the generated videos. We also detail our training strategies and share key observations and insights. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, Step-Video-T2V-Eval, demonstrating its state-of-the-art text-to-video quality when compared with both open-source and commercial engines. Additionally, we discuss the limitations of current diffusion-based model paradigm and outline future directions for video foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval available at https://github.com/stepfun-ai/Step-Video-T2V. The online version can be accessed from https://yuewen.cn/videos as well. Our goal is to accelerate the innovation of video foundation models and empower video content creators.
Cascading Bandits Robust to Adversarial Corruptions
Authors: Jize Xie, Cheng Chen, Zhiyong Wang, Shuai Li
Date: 2025-02-12 00:00:00
Tasks: Learning-To-Rank

Online learning to rank sequentially recommends a small list of items to users from a large candidate set and receives the users' click feedback. In many real-world scenarios, users browse the recommended list in order and click the first attractive item without checking the rest. Such behaviors are usually formulated as the cascade model. Many recent works study algorithms for cascading bandits, an online learning to rank framework in the cascade model. However, the performance of existing methods may drop significantly if part of the user feedback is adversarially corrupted (e.g., click fraud). In this work, we study how to resist adversarial corruptions in cascading bandits. We first formulate the ``\textit{Cascading Bandits with Adversarial Corruptions}" (CBAC) problem, which assumes that there is an adaptive adversary that may manipulate the user feedback. Then we propose two robust algorithms for this problem, which assume the corruption level is known and agnostic, respectively. We show that both algorithms can achieve logarithmic regret when the algorithm is not under attack, and the regret increases linearly with the corruption level. The experimental results also verify the robustness of our methods.
Amnesia as a Catalyst for Enhancing Black Box Pixel Attacks in Image Classification and Object Detection
Authors: Dongsu Song, Daehwa Ko, Jay Hoon Jung
Date: 2025-02-10 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection

It is well known that query-based attacks tend to have relatively higher success rates in adversarial black-box attacks. While research on black-box attacks is actively being conducted, relatively few studies have focused on pixel attacks that target only a limited number of pixels. In image classification, query-based pixel attacks often rely on patches, which heavily depend on randomness and neglect the fact that scattered pixels are more suitable for adversarial attacks. Moreover, to the best of our knowledge, query-based pixel attacks have not been explored in the field of object detection. To address these issues, we propose a novel pixel-based black-box attack called Remember and Forget Pixel Attack using Reinforcement Learning(RFPAR), consisting of two main components: the Remember and Forget processes. RFPAR mitigates randomness and avoids patch dependency by leveraging rewards generated through a one-step RL algorithm to perturb pixels. RFPAR effectively creates perturbed images that minimize the confidence scores while adhering to limited pixel constraints. Furthermore, we advance our proposed attack beyond image classification to object detection, where RFPAR reduces the confidence scores of detected objects to avoid detection. Experiments on the ImageNet-1K dataset for classification show that RFPAR outperformed state-of-the-art query-based pixel attacks. For object detection, using the MSCOCO dataset with YOLOv8 and DDQ, RFPAR demonstrates comparable mAP reduction to state-of-the-art query-based attack while requiring fewer query. Further experiments on the Argoverse dataset using YOLOv8 confirm that RFPAR effectively removed objects on a larger scale dataset. Our code is available at https://github.com/KAU-QuantumAILab/RFPAR.
A GAN-based Denoising Method for Chinese Stele and Rubbing Calligraphic Image
Authors: Xuanhong Wang, Kun Wu, Ying Zhang, Yun Xiao & Pengfei Xu
Date: 2022-02-09 00:00:00
Proceeding: Springer Nature 2022 2
Tasks: Denoising, Generative Adversarial Network, Image Denoising, SSIM

Chinese calligraphic images have important artistic and historical values. However, subjected to hundreds of years of natural weathering, corrosion and man-made destruction, Chinese calligraphic images inevitably contain some special noise, such as dotted noise, flake corrosion noise and scratch noise. How to denoise this special noise is a challenge for digital preservation of Chinese calligraphic. In this paper, we propose an end-to-end calligraphic image denoising algorithm based on a well-designed generative adversarial network. The generator contains a recurrent network and a denoising autoencoder. By introducing an attention mechanism, we use a recurrent network with multiple progressive network units to generate a noise attention map. Through the noise attention map, the denoising autoencoder can restore the noisy calligraphic image into a clean image with reduced noise or even no noise. The extensive experiments results show that the results of our method are better than those of other comparison methods in terms of visual effects, PSNR and SSIM.
Robust Anomaly Detection via Tensor Chidori Pseudoskeleton Decomposition
Authors: Bowen Su
Date: 2025-02-14 00:00:00

Anomaly detection plays a critical role in modern data-driven applications, from identifying fraudulent transactions and safeguarding network infrastructure to monitoring sensor systems for irregular patterns. Traditional approaches, such as distance, density, or cluster-based methods, face significant challenges when applied to high dimensional tensor data, where complex interdependencies across dimensions amplify noise and computational complexity. To address these limitations, this paper leverages Tensor Chidori pseudoskeleton decomposition within a tensor-robust principal component analysis framework to extract low Tucker rank structure while isolating sparse anomalies, ensuring robustness to anomaly detection. We establish theoretical results regarding convergence, and estimation error, demonstrating the stability and accuracy of the proposed approach. Numerical experiments on real-world spatiotemporal data from New York City taxi trip records validate the superiority of the proposed method in detecting anomalous urban events compared to existing benchmark methods. The results underscore the potential of Tensor Chidori pseudoskeleton decomposition to enhance anomaly detection for large-scale, high-dimensional data.
QMaxViT-Unet+: A Query-Based MaxViT-Unet with Edge Enhancement for Scribble-Supervised Segmentation of Medical Images
Authors: Thien B. Nguyen-Tat, Hoang-An Vo, Phuoc-Sang Dang
Date: 2025-02-14 00:00:00
Tasks: Decoder, Image Segmentation, Medical Image Analysis, Medical Image Segmentation, Semantic Segmentation, Weakly-supervised Learning

The deployment of advanced deep learning models for medical image segmentation is often constrained by the requirement for extensively annotated datasets. Weakly-supervised learning, which allows less precise labels, has become a promising solution to this challenge. Building on this approach, we propose QMaxViT-Unet+, a novel framework for scribble-supervised medical image segmentation. This framework is built on the U-Net architecture, with the encoder and decoder replaced by Multi-Axis Vision Transformer (MaxViT) blocks. These blocks enhance the model's ability to learn local and global features efficiently. Additionally, our approach integrates a query-based Transformer decoder to refine features and an edge enhancement module to compensate for the limited boundary information in the scribble label. We evaluate the proposed QMaxViT-Unet+ on four public datasets focused on cardiac structures, colorectal polyps, and breast cancer: ACDC, MS-CMRSeg, SUN-SEG, and BUSI. Evaluation metrics include the Dice similarity coefficient (DSC) and the 95th percentile of Hausdorff distance (HD95). Experimental results show that QMaxViT-Unet+ achieves 89.1\% DSC and 1.316mm HD95 on ACDC, 88.4\% DSC and 2.226mm HD95 on MS-CMRSeg, 71.4\% DSC and 4.996mm HD95 on SUN-SEG, and 69.4\% DSC and 50.122mm HD95 on BUSI. These results demonstrate that our method outperforms existing approaches in terms of accuracy, robustness, and efficiency while remaining competitive with fully-supervised learning approaches. This makes it ideal for medical image analysis, where high-quality annotations are often scarce and require significant effort and expense. The code is available at: https://github.com/anpc849/QMaxViT-Unet
Compress image to patches for Vision Transformer
Authors: Xinfeng Zhao, Yaoru Sun
Date: 2025-02-14 00:00:00
Tasks: Image Classification, Inductive Bias

The Vision Transformer (ViT) has made significant strides in the field of computer vision. However, as the depth of the model and the resolution of the input images increase, the computational cost associated with training and running ViT models has surged dramatically.This paper proposes a hybrid model based on CNN and Vision Transformer, named CI2P-ViT. The model incorporates a module called CI2P, which utilizes the CompressAI encoder to compress images and subsequently generates a sequence of patches through a series of convolutions. CI2P can replace the Patch Embedding component in the ViT model, enabling seamless integration into existing ViT models.Compared to ViT-B/16, CI2P-ViT has the number of patches input to the self-attention layer reduced to a quarter of the original.This design not only significantly reduces the computational cost of the ViT model but also effectively enhances the model's accuracy by introducing the inductive bias properties of CNN.The ViT model's precision is markedly enhanced.When trained from the ground up on the Animals-10 dataset, CI2P-ViT achieved an accuracy rate of 92.37%, representing a 3.3% improvement over the ViT-B/16 baseline. Additionally, the model's computational operations, measured in floating-point operations per second (FLOPs), were diminished by 63.35%, and it exhibited a 2-fold increase in training velocity on identical hardware configurations.
Efficient Beam Search for Large Language Models Using Trie-Based Decoding
Authors: Brian J Chan, Jui-Hung Cheng, Mao Xun Huang, Chao-Ting Chen, Hen-Hsen Huang
Date: 2025-01-31 00:00:00

In Transformer-based sequence-to-sequence generation, beam search has proven effective in enhancing the quality of generated sequences compared to greedy decoding. Conventional beam search methods typically adopt either a sequential or batch-based approach. The sequential approach, while memory-efficient, requires multiple decoding passes to construct a complete search tree, leading to significantly slower inference. On the other hand, the batch-based approach enables parallel computation across beams, but at the expense of high memory consumption due to the need to maintain separate key-value (KV) caches for each beam. In this study, we introduce a novel trie (prefix-tree)-based parallel decoding method that addresses the memory inefficiency of batch-based beam search. By sharing a single KV cache among all beams that share the same prefix, the proposed method not only reduces memory consumption dramatically but also enables parallel decoding across all branches. This innovative use of a prefix tree offers an efficient alternative for beam search, achieving significant memory savings while preserving inference speed, making it particularly well-suited for memory-constrained environments or large-scale model deployments.
Object Detection and Tracking
Authors: Md Pranto, Omar Faruk
Date: 2025-02-14 00:00:00
Tasks: Deep Learning, Object, object-detection, Object Detection

Efficient and accurate object detection is an important topic in the development of computer vision systems. With the advent of deep learning techniques, the accuracy of object detection has increased significantly. The project aims to integrate a modern technique for object detection with the aim of achieving high accuracy with real-time performance. The reliance on other computer vision algorithms in many object identification systems, which results in poor and ineffective performance, is a significant obstacle. In this research, we solve the end-to-end object detection problem entirely using deep learning techniques. The network is trained using the most difficult publicly available dataset, which is used for an annual item detection challenge. Applications that need object detection can benefit the system's quick and precise finding.
A fuzzy rank-based ensemble of CNN models for MRI segmentation
Authors: Daria Valenkova, Asya Lyanova, Aleksandr Sinitca, Ram Sarkar, Dmitrii Kaplun
Date: 2024-12-28 00:00:00
Proceeding: Biomedical Signal Processing and Control 2024 12
Tasks: Decision Making, MRI segmentation, Segmentation, Tumor Segmentation

Glioblastoma is the most common subtype of malignant tumors of the central nervous system. Segmentation of the brain tumor image is crucial for accelerating the diagnosis and treatment of a patient. In the paper, an advanced neural network ensemble based on a fuzzy ranking approach for tumor segmentation is presented using a combination of convolutional neural network (CNN) architectures, namely SegResNet, UNETR, and SwinUNETR. The proposed method uses fuzzy rank-based unification of deep learners by considering two nonlinear functions in decision-making, which helps to take into account the confidence in the predictions of the three base models. The proposed method is evaluated using the BRATS 2023 MRI dataset and outperforms the state-of-the-art methods, achieving an average Dice score of 0.885+/-0.134. The statistical significance of the differences between the models and the ensemble is confirmed based on the Wilcoxon signed rank test, and the p-value is below 0.005.
CISSIR: Beam Codebooks with Self-Interference Reduction Guarantees for Integrated Sensing and Communication Beyond 5G
Authors: Rodrigo Hernangómez, Jochen Fink, Renato L. G. Cavalcante, Sławomir Stańczak
Date: 2025-02-14 00:00:00
Tasks: Integrated sensing and communication, ISAC, Quantization

We propose a beam codebook design to reduce self-interference (SI) in integrated sensing and communication (ISAC) systems. Our optimization methods, which can be applied to both tapered beamforming and phased arrays, adapt the codebooks to the SI channel such that a certain SI level is achieved. Furthermore, we derive an upper bound on the quantization noise in terms of the achieved SI level, which provides guidelines to pose the optimization problem in order to obtain performance guarantees for sensing. By selecting standard reference codebooks in our simulations, we show substantially improved sensing quality with little impact on 5G-NR communication. Our proposed method is not only less dependent on hyperparameters than other approaches in the literature, but it can also reduce SI further, and thus deliver better sensing and communication performance.
Safe Reinforcement Learning-based Control for Hydrogen Diesel Dual-Fuel Engines
Authors: Vasu Sharma, Alexander Winkler, Armin Norouzi, Jakob Andert, David Gordon, Hongsheng Guo
Date: 2025-02-13 00:00:00
Tasks: Model Predictive Control, Reinforcement Learning (RL), Safe Reinforcement Learning

The urgent energy transition requirements towards a sustainable future stretch across various industries and are a significant challenge facing humanity. Hydrogen promises a clean, carbon-free future, with the opportunity to integrate with existing solutions in the transportation sector. However, adding hydrogen to existing technologies such as diesel engines requires additional modeling effort. Reinforcement Learning (RL) enables interactive data-driven learning that eliminates the need for mathematical modeling. The algorithms, however, may not be real-time capable and need large amounts of data to work in practice. This paper presents a novel approach which uses offline model learning with RL to demonstrate safe control of a 4.5 L Hydrogen Diesel Dual-Fuel (H2DF) engine. The controllers are demonstrated to be constraint compliant and can leverage a novel state-augmentation approach for sample-efficient learning. The offline policy is subsequently experimentally validated on the real engine where the control algorithm is executed on a Raspberry Pi controller and requires 6 times less computation time compared to online Model Predictive Control (MPC) optimization.
Data Valuation using Neural Networks for Efficient Instruction Fine-Tuning
Authors: Ishika Agarwal, Dilek Hakkani-Tur
Date: 2025-02-14 00:00:00
Tasks: Data Valuation

Influence functions provide crucial insights into model training, but existing methods suffer from large computational costs and limited generalization. Particularly, recent works have proposed various metrics and algorithms to calculate the influence of data using language models, which do not scale well with large models and datasets. This is because of the expensive forward and backward passes required for computation, substantial memory requirements to store large models, and poor generalization of influence estimates to new data. In this paper, we explore the use of small neural networks -- which we refer to as the InfluenceNetwork -- to estimate influence values, achieving up to 99% cost reduction. Our evaluation demonstrates that influence values can be estimated with models just 0.0027% the size of full language models (we use 7B and 8B versions). We apply our algorithm of estimating influence values (called NN-CIFT: Neural Networks for effiCient Instruction Fine-Tuning) to the downstream task of subset selection for general instruction fine-tuning. In our study, we include four state-of-the-art influence functions and show no compromise in performance, despite large speedups, between NN-CIFT and the original influence functions. We provide an in-depth hyperparameter analyses of NN-CIFT. The code for our method can be found here: https://github.com/agarwalishika/NN-CIFT.