pwc-archive/papers-with-abstracts huggingface dataset:
Development of Context-Sensitive Formulas to Obtain Constant Luminance Perception for a Foreground Object in Front of Backgrounds of Varying Luminance
Authors: Ergun Akleman, Bekir Tevfik Akgun, Adil Alpkocak
Date: 2024-02-28 00:00:00

In this article, we present a framework for developing context-sensitive luminance correction formulas that can produce constant luminance perception for foreground objects. Our formulas make the foreground object slightly translucent to mix with the blurred version of the background. This mix can quickly produce any desired illusion of luminance in foreground objects based on the luminance of the background. The translucency formula has only one parameter; the relative size of the foreground object, which is a number between zero and one. We have identified the general structure of the translucency formulas as a power function of the relative size of the foreground object. We have implemented a web-based interactive program in Shadertoy. Using this program, we determined the coefficients of the polynomial exponents of the power function. To intuitively control the coefficients of the polynomial functions, we have used a B\'{e}zier form. Our final translucency formula uses a quadratic polynomial and requires only three coefficients. We also identified a simpler affine formula, which requires only two coefficients. We made our program publicly available in Shadertoy so that anyone can access and improve it. In this article, we also explain how to intuitively change the polynomial part of the formula. Using our explanation, users change the polynomial part of the formula to obtain their own perceptively constant luminance. This can be used as a crowd-sourcing experiment for further improvement of the formula.
Windowed-FourierMixer: Enhancing Clutter-Free Room Modeling with Fourier Transform
Authors: Bruno Henriques, Benjamin Allaert, Jean-Philippe Vandeborre
Date: 2024-02-28 00:00:00

With the growing demand for immersive digital applications, the need to understand and reconstruct 3D scenes has significantly increased. In this context, inpainting indoor environments from a single image plays a crucial role in modeling the internal structure of interior spaces as it enables the creation of textured and clutter-free reconstructions. While recent methods have shown significant progress in room modeling, they rely on constraining layout estimators to guide the reconstruction process. These methods are highly dependent on the performance of the structure estimator and its generative ability in heavily occluded environments. In response to these issues, we propose an innovative approach based on a U-Former architecture and a new Windowed-FourierMixer block, resulting in a unified, single-phase network capable of effectively handle human-made periodic structures such as indoor spaces. This new architecture proves advantageous for tasks involving indoor scenes where symmetry is prevalent, allowing the model to effectively capture features such as horizon/ceiling height lines and cuboid-shaped rooms. Experiments show the proposed approach outperforms current state-of-the-art methods on the Structured3D dataset demonstrating superior performance in both quantitative metrics and qualitative results. Code and models will be made publicly available.
EAN-MapNet: Efficient Vectorized HD Map Construction with Anchor Neighborhoods
Authors: Huiyuan Xiong, Jun Shen, Taohong Zhu, Yuelong Pan
Date: 2024-02-28 00:00:00
Tasks: Autonomous Driving, Decoder

High-definition (HD) map is crucial for autonomous driving systems. Most existing works design map elements detection heads based on the DETR decoder. However, the initial queries lack explicit incorporation of physical positional information, and vanilla self-attention entails high computational complexity. Therefore, we propose EAN-MapNet for Efficiently constructing HD map using Anchor Neighborhoods. Firstly, we design query units based on the anchor neighborhoods, allowing non-neighborhood central anchors to effectively assist in fitting the neighborhood central anchors to the target points representing map elements. Then, we propose grouped local self-attention (GL-SA) by leveraging the relative instance relationship among the queries. This facilitates direct feature interaction among queries of the same instances, while innovatively employing local queries as intermediaries for interaction among queries from different instances. Consequently, GL-SA significantly reduces the computational complexity of self-attention while ensuring ample feature interaction among queries. On the nuScenes dataset, EAN-MapNet achieves a state-of-the-art performance with 63.0 mAP after training for 24 epochs, surpassing MapTR by 12.7 mAP. Furthermore, it considerably reduces memory consumption by 8198M compared to MapTRv2.
Attentive Illumination Decomposition Model for Multi-Illuminant White Balancing
Authors: Dongyoung Kim, Jinwoo Kim, Junsang Yu, Seon Joo Kim
Date: 2024-02-28 00:00:00
Proceeding: CVPR 2024 1

White balance (WB) algorithms in many commercial cameras assume single and uniform illumination, leading to undesirable results when multiple lighting sources with different chromaticities exist in the scene. Prior research on multi-illuminant WB typically predicts illumination at the pixel level without fully grasping the scene's actual lighting conditions, including the number and color of light sources. This often results in unnatural outcomes lacking in overall consistency. To handle this problem, we present a deep white balancing model that leverages the slot attention, where each slot is in charge of representing individual illuminants. This design enables the model to generate chromaticities and weight maps for individual illuminants, which are then fused to compose the final illumination map. Furthermore, we propose the centroid-matching loss, which regulates the activation of each slot based on the color range, thereby enhancing the model to separate illumination more effectively. Our method achieves the state-of-the-art performance on both single- and multi-illuminant WB benchmarks, and also offers additional information such as the number of illuminants in the scene and their chromaticity. This capability allows for illumination editing, an application not feasible with prior methods.
Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding
Authors: Hongshen Xu, Lu Chen, Zihan Zhao, Da Ma, Ruisheng Cao, Zichen Zhu, Kai Yu
Date: 2024-02-28 00:00:00
Tasks: document understanding, Information Retrieval, Retrieval

The growing prevalence of visually rich documents, such as webpages and scanned/digital-born documents (images, PDFs, etc.), has led to increased interest in automatic document understanding and information extraction across academia and industry. Although various document modalities, including image, text, layout, and structure, facilitate human information retrieval, the interconnected nature of these modalities presents challenges for neural networks. In this paper, we introduce WebLM, a multimodal pre-training network designed to address the limitations of solely modeling text and structure modalities of HTML in webpages. Instead of processing document images as unified natural images, WebLM integrates the hierarchical structure of document images to enhance the understanding of markup-language-based documents. Additionally, we propose several pre-training tasks to model the interaction among text, structure, and image modalities effectively. Empirical results demonstrate that the pre-trained WebLM significantly surpasses previous state-of-the-art pre-trained models across several webpage understanding tasks. The pre-trained models and code are available at https://github.com/X-LANCE/weblm.
On the Accuracy of Edge Detectors in Number Plate Extraction
Authors: Bashir Olaniyi Sadiq
Date: 2024-02-28 00:00:00
Tasks: Edge Detection

Edge detection as a pre-processing stage is a fundamental and important aspect of the number plate extraction system. This is due to the fact that the identification of a particular vehicle is achievable using the number plate because each number plate is unique to a vehicle. As such, the characters of a number plate system that differ in lines and shapes can be extracted using the principle of edge detection. This paper presents a method of number plate extraction using edge detection technique. Edges in number plates are identified with changes in the intensity of pixel values. Therefore, these edges are identified using a single based pixel or collection of pixel-based approach. The efficiency of these approaches of edge detection algorithms in number plate extraction in both noisy and clean environment are experimented. Experimental results are achieved in MATLAB 2017b using the Pratt Figure of Merit (PFOM) as a performance metric
Image2Flow: A hybrid image and graph convolutional neural network for rapid patient-specific pulmonary artery segmentation and CFD flow field calculation from 3D cardiac MRI data
Authors: Tina Yao, Endrit Pajaziti, Michael Quail, Silvia Schievano, Jennifer A Steeden, Vivek Muthurangu
Date: 2024-02-28 00:00:00
Tasks: Anatomy, Segmentation

Computational fluid dynamics (CFD) can be used for evaluation of hemodynamics. However, its routine use is limited by labor-intensive manual segmentation, CFD mesh creation, and time-consuming simulation. This study aims to train a deep learning model to both generate patient-specific volume-meshes of the pulmonary artery from 3D cardiac MRI data and directly estimate CFD flow fields. This study used 135 3D cardiac MRIs from both a public and private dataset. The pulmonary arteries in the MRIs were manually segmented and converted into volume-meshes. CFD simulations were performed on ground truth meshes and interpolated onto point-point correspondent meshes to create the ground truth dataset. The dataset was split 85/10/15 for training, validation and testing. Image2Flow, a hybrid image and graph convolutional neural network, was trained to transform a pulmonary artery template to patient-specific anatomy and CFD values. Image2Flow was evaluated in terms of segmentation and accuracy of CFD predicted was assessed using node-wise comparisons. Centerline comparisons of Image2Flow and CFD simulations performed using machine learning segmentation were also performed. Image2Flow achieved excellent segmentation accuracy with a median Dice score of 0.9 (IQR: 0.86-0.92). The median node-wise normalized absolute error for pressure and velocity magnitude was 11.98% (IQR: 9.44-17.90%) and 8.06% (IQR: 7.54-10.41), respectively. Centerline analysis showed no significant difference between the Image2Flow and conventional CFD simulated on machine learning-generated volume-meshes. This proof-of-concept study has shown it is possible to simultaneously perform patient specific volume-mesh based segmentation and pressure and flow field estimation. Image2Flow completes segmentation and CFD in ~205ms, which ~7000 times faster than manual methods, making it more feasible in a clinical environment.
Zero-Shot Aerial Object Detection with Visual Description Regularization
Authors: Zhengqing Zang, Chenyu Lin, Chenwei Tang, Tao Wang, Jiancheng Lv
Date: 2024-02-28 00:00:00
Tasks: Object, object-detection, Object Detection, Triplet

Existing object detection models are mainly trained on large-scale labeled datasets. However, annotating data for novel aerial object classes is expensive since it is time-consuming and may require expert knowledge. Thus, it is desirable to study label-efficient object detection methods on aerial images. In this work, we propose a zero-shot method for aerial object detection named visual Description Regularization, or DescReg. Concretely, we identify the weak semantic-visual correlation of the aerial objects and aim to address the challenge with prior descriptions of their visual appearance. Instead of directly encoding the descriptions into class embedding space which suffers from the representation gap problem, we propose to infuse the prior inter-class visual similarity conveyed in the descriptions into the embedding learning. The infusion process is accomplished with a newly designed similarity-aware triplet loss which incorporates structured regularization on the representation space. We conduct extensive experiments with three challenging aerial object detection datasets, including DIOR, xView, and DOTA. The results demonstrate that DescReg significantly outperforms the state-of-the-art ZSD methods with complex projection designs and generative frameworks, e.g., DescReg outperforms best reported ZSD method on DIOR by 4.5 mAP on unseen classes and 8.1 in HM. We further show the generalizability of DescReg by integrating it into generative ZSD methods as well as varying the detection architecture.
Region-Aware Exposure Consistency Network for Mixed Exposure Correction
Authors: Jin Liu, Huiyuan Fu, Chuanming Wang, Huadong Ma
Date: 2024-02-28 00:00:00
Tasks: Exposure Correction

Exposure correction aims to enhance images suffering from improper exposure to achieve satisfactory visual effects. Despite recent progress, existing methods generally mitigate either overexposure or underexposure in input images, and they still struggle to handle images with mixed exposure, i.e., one image incorporates both overexposed and underexposed regions. The mixed exposure distribution is non-uniform and leads to varying representation, which makes it challenging to address in a unified process. In this paper, we introduce an effective Region-aware Exposure Correction Network (RECNet) that can handle mixed exposure by adaptively learning and bridging different regional exposure representations. Specifically, to address the challenge posed by mixed exposure disparities, we develop a region-aware de-exposure module that effectively translates regional features of mixed exposure scenarios into an exposure-invariant feature space. Simultaneously, as de-exposure operation inevitably reduces discriminative information, we introduce a mixed-scale restoration unit that integrates exposure-invariant features and unprocessed features to recover local information. To further achieve a uniform exposure distribution in the global image, we propose an exposure contrastive regularization strategy under the constraints of intra-regional exposure consistency and inter-regional exposure continuity. Extensive experiments are conducted on various datasets, and the experimental results demonstrate the superiority and generalization of our proposed method. The code is released at: https://github.com/kravrolens/RECNet.
Multi-objective Differentiable Neural Architecture Search
Authors: Rhea Sanjay Sukthanker, Arber Zela, Benedikt Staffler, Samuel Dooley, Josif Grabocka, Frank Hutter
Date: 2024-02-28 00:00:00
Tasks: Decoder, Language Modelling, Machine Translation, Neural Architecture Search

Pareto front profiling in multi-objective optimization (MOO), i.e. finding a diverse set of Pareto optimal solutions, is challenging, especially with expensive objectives like neural network training. Typically, in MOO neural architecture search (NAS), we aim to balance performance and hardware metrics across devices. Prior NAS approaches simplify this task by incorporating hardware constraints into the objective function, but profiling the Pareto front necessitates a computationally expensive search for each constraint. In this work, we propose a novel NAS algorithm that encodes user preferences for the trade-off between performance and hardware metrics, and yields representative and diverse architectures across multiple devices in just one search run. To this end, we parameterize the joint architectural distribution across devices and multiple objectives via a hypernetwork that can be conditioned on hardware features and preference vectors, enabling zero-shot transferability to new devices. Extensive experiments with up to 19 hardware devices and 3 objectives showcase the effectiveness and scalability of our method. Finally, we show that, without extra costs, our method outperforms existing MOO NAS methods across a broad range of qualitatively different search spaces and datasets, including MobileNetV3 on ImageNet-1k, an encoder-decoder transformer space for machine translation and a decoder-only transformer space for language modelling.
Balancing Act: Distribution-Guided Debiasing in Diffusion Models
Authors: Rishubh Parihar, Abhijnya Bhat, Abhipsa Basu, Saswat Mallick, Jogendra Nath Kundu, R. Venkatesh Babu
Date: 2024-02-28 00:00:00
Proceeding: CVPR 2024 1
Tasks: Attribute, Data Augmentation, Denoising, Image Generation

Diffusion Models (DMs) have emerged as powerful generative models with unprecedented image generation capability. These models are widely used for data augmentation and creative applications. However, DMs reflect the biases present in the training datasets. This is especially concerning in the context of faces, where the DM prefers one demographic subgroup vs others (eg. female vs male). In this work, we present a method for debiasing DMs without relying on additional data or model retraining. Specifically, we propose Distribution Guidance, which enforces the generated images to follow the prescribed attribute distribution. To realize this, we build on the key insight that the latent features of denoising UNet hold rich demographic semantics, and the same can be leveraged to guide debiased generation. We train Attribute Distribution Predictor (ADP) - a small mlp that maps the latent features to the distribution of attributes. ADP is trained with pseudo labels generated from existing attribute classifiers. The proposed Distribution Guidance with ADP enables us to do fair generation. Our method reduces bias across single/multiple attributes and outperforms the baseline by a significant margin for unconditional and text-conditional diffusion models. Further, we present a downstream task of training a fair attribute classifier by rebalancing the training set with our generated data.
Oil Spill Drone: A Dataset of Drone-Captured, Segmented RGB Images for Oil Spill Detection in Port Environments
Authors: T. De Kerf, S. Sels, S. Samsonova, S. Vanlanduit
Date: 2024-02-28 00:00:00
Tasks: Edge-computing

The high incidence of oil spills in port areas poses a serious threat to the environment, prompting the need for efficient detection mechanisms. Utilizing automated drones for this purpose can significantly improve the speed and accuracy of oil spill detection. Such advancements not only expedite cleanup operations, reducing environmental harm but also enhance polluter accountability, potentially deterring future incidents. Currently, there's a scarcity of datasets employing RGB images for oil spill detection in maritime settings. This paper presents a unique, annotated dataset aimed at addressing this gap, leveraging a neural network for analysis on both desktop and edge computing platforms. The dataset, captured via drone, comprises 1268 images categorized into oil, water, and other, with a convolutional neural network trained using an Unet model architecture achieving an F1 score of 0.71 for oil detection. This underscores the dataset's practicality for real-world applications, offering crucial resources for environmental conservation in port environments.
Learning Invariant Inter-pixel Correlations for Superpixel Generation
Authors: Sen Xu, Shikui Wei, Tao Ruan, Lixin Liao
Date: 2024-02-28 00:00:00

Deep superpixel algorithms have made remarkable strides by substituting hand-crafted features with learnable ones. Nevertheless, we observe that existing deep superpixel methods, serving as mid-level representation operations, remain sensitive to the statistical properties (e.g., color distribution, high-level semantics) embedded within the training dataset. Consequently, learnable features exhibit constrained discriminative capability, resulting in unsatisfactory pixel grouping performance, particularly in untrainable application scenarios. To address this issue, we propose the Content Disentangle Superpixel (CDS) algorithm to selectively separate the invariant inter-pixel correlations and statistical properties, i.e., style noise. Specifically, We first construct auxiliary modalities that are homologous to the original RGB image but have substantial stylistic variations. Then, driven by mutual information, we propose the local-grid correlation alignment across modalities to reduce the distribution discrepancy of adaptively selected features and learn invariant inter-pixel correlations. Afterwards, we perform global-style mutual information minimization to enforce the separation of invariant content and train data styles. The experimental results on four benchmark datasets demonstrate the superiority of our approach to existing state-of-the-art methods, regarding boundary adherence, generalization, and efficiency. Code and pre-trained model are available at https://github.com/rookiie/CDSpixel.
NToP: NeRF-Powered Large-scale Dataset Generation for 2D and 3D Human Pose Estimation in Top-View Fisheye Images
Authors: Jingrui Yu, Dipankar Nandi, Roman Seidel, Gangolf Hirtz
Date: 2024-02-28 00:00:00
Tasks: 3D Human Pose Estimation, Dataset Generation, NeRF, Pose Estimation

Human pose estimation (HPE) in the top-view using fisheye cameras presents a promising and innovative application domain. However, the availability of datasets capturing this viewpoint is extremely limited, especially those with high-quality 2D and 3D keypoint annotations. Addressing this gap, we leverage the capabilities of Neural Radiance Fields (NeRF) technique to establish a comprehensive pipeline for generating human pose datasets from existing 2D and 3D datasets, specifically tailored for the top-view fisheye perspective. Through this pipeline, we create a novel dataset NToP570K (NeRF-powered Top-view human Pose dataset for fisheye cameras with over 570 thousand images), and conduct an extensive evaluation of its efficacy in enhancing neural networks for 2D and 3D top-view human pose estimation. A pretrained ViTPose-B model achieves an improvement in AP of 33.3 % on our validation set for 2D HPE after finetuning on our training set. A similarly finetuned HybrIK-Transformer model gains 53.7 mm reduction in PA-MPJPE for 3D HPE on the validation set.
Misalignment-Robust Frequency Distribution Loss for Image Transformation
Authors: Zhangkai Ni, Juncheng Wu, Zian Wang, Wenhan Yang, Hanli Wang, Lin Ma
Date: 2024-02-28 00:00:00
Proceeding: CVPR 2024 1
Tasks: Image Enhancement, Style Transfer, Super-Resolution

This paper aims to address a common challenge in deep learning-based image transformation methods, such as image enhancement and super-resolution, which heavily rely on precisely aligned paired datasets with pixel-level alignments. However, creating precisely aligned paired images presents significant challenges and hinders the advancement of methods trained on such data. To overcome this challenge, this paper introduces a novel and simple Frequency Distribution Loss (FDL) for computing distribution distance within the frequency domain. Specifically, we transform image features into the frequency domain using Discrete Fourier Transformation (DFT). Subsequently, frequency components (amplitude and phase) are processed separately to form the FDL loss function. Our method is empirically proven effective as a training constraint due to the thoughtful utilization of global information in the frequency domain. Extensive experimental evaluations, focusing on image enhancement and super-resolution tasks, demonstrate that FDL outperforms existing misalignment-robust loss functions. Furthermore, we explore the potential of our FDL for image style transfer that relies solely on completely misaligned data. Our code is available at: https://github.com/eezkni/FDL