pwc-archive/papers-with-abstracts huggingface dataset:
TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation
Authors: Hyesu Lim, Byeonggeun Kim, Jaegul Choo, Sungha Choi
Date: 2023-02-10 00:00:00
Tasks: Test-time Adaptation

This paper proposes a novel batch normalization strategy for test-time adaptation. Recent test-time adaptation methods heavily rely on the modified batch normalization, i.e., transductive batch normalization (TBN), which calculates the mean and the variance from the current test batch rather than using the running mean and variance obtained from the source data, i.e., conventional batch normalization (CBN). Adopting TBN that employs test batch statistics mitigates the performance degradation caused by the domain shift. However, re-estimating normalization statistics using test data depends on impractical assumptions that a test batch should be large enough and be drawn from i.i.d. stream, and we observed that the previous methods with TBN show critical performance drop without the assumptions. In this paper, we identify that CBN and TBN are in a trade-off relationship and present a new test-time normalization (TTN) method that interpolates the statistics by adjusting the importance between CBN and TBN according to the domain-shift sensitivity of each BN layer. Our proposed TTN improves model robustness to shifted domains across a wide range of batch sizes and in various realistic evaluation scenarios. TTN is widely applicable to other test-time adaptation methods that rely on updating model parameters via backpropagation. We demonstrate that adopting TTN further improves their performance and achieves state-of-the-art performance in various standard benchmarks.
Virtually increasing the measurement frequency of LIDAR sensor utilizing a single RGB camera
Authors: Zoltan Rozsa, Tamas Sziranyi
Date: 2023-02-10 00:00:00

The frame rates of most 3D LIDAR sensors used in intelligent vehicles are substantially lower than current cameras installed in the same vehicle. This research suggests using a mono camera to virtually enhance the frame rate of LIDARs, allowing the more frequent monitoring of dynamic objects in the surroundings that move quickly. As a first step, dynamic object candidates are identified and tracked in the camera frames. Following that, the LIDAR measurement points of these items are found by clustering in the frustums of 2D bounding boxes. Projecting these to the camera and tracking them to the next camera frame can be used to create 3D-2D correspondences between different timesteps. These correspondences between the last LIDAR frame and the actual camera frame are used to solve the PnP (Perspective-n-Point) problem. Finally, the estimated transformations are applied to the previously measured points to generate virtual measurements. With the proposed estimation, if the ego movement is known, not just static object position can be determined at timesteps where camera measurement is available, but positions of dynamic objects as well. We achieve state-of-the-art performance on large public datasets in terms of accuracy and similarity to real measurements.
End-to-end Semantic Object Detection with Cross-Modal Alignment
Authors: Silvan Ferreira, Allan Martins, Ivanovitch Silva
Date: 2023-02-10 00:00:00
Tasks: Contrastive Learning, cross-modal alignment, Image Retrieval, Object, object-detection, Object Detection, Region Proposal

Traditional semantic image search methods aim to retrieve images that match the meaning of the text query. However, these methods typically search for objects on the whole image, without considering the localization of objects within the image. This paper presents an extension of existing object detection models for semantic image search that considers the semantic alignment between object proposals and text queries, with a focus on searching for objects within images. The proposed model uses a single feature extractor, a pre-trained Convolutional Neural Network, and a transformer encoder to encode the text query. Proposal-text alignment is performed using contrastive learning, producing a score for each proposal that reflects its semantic alignment with the text query. The Region Proposal Network (RPN) is used to generate object proposals, and the end-to-end training process allows for an efficient and effective solution for semantic image search. The proposed model was trained end-to-end, providing a promising solution for semantic image search that retrieves images that match the meaning of the text query and generates semantically relevant object proposals.
PointWavelet: Learning in Spectral Domain for 3D Point Cloud Analysis
Authors: Cheng Wen, Jianzhi Long, Baosheng Yu, DaCheng Tao
Date: 2023-02-10 00:00:00
Tasks: Autonomous Driving, Deep Learning, Point Cloud Classification

With recent success of deep learning in 2D visual recognition, deep learning-based 3D point cloud analysis has received increasing attention from the community, especially due to the rapid development of autonomous driving technologies. However, most existing methods directly learn point features in the spatial domain, leaving the local structures in the spectral domain poorly investigated. In this paper, we introduce a new method, PointWavelet, to explore local graphs in the spectral domain via a learnable graph wavelet transform. Specifically, we first introduce the graph wavelet transform to form multi-scale spectral graph convolution to learn effective local structural representations. To avoid the time-consuming spectral decomposition, we then devise a learnable graph wavelet transform, which significantly accelerates the overall training process. Extensive experiments on four popular point cloud datasets, ModelNet40, ScanObjectNN, ShapeNet-Part, and S3DIS, demonstrate the effectiveness of the proposed method on point cloud classification and segmentation.
CGA-PoseNet: Camera Pose Regression via a 1D-Up Approach to Conformal Geometric Algebra
Authors: Alberto Pepe, Joan Lasenby
Date: 2023-02-10 00:00:00
Tasks: regression

We introduce CGA-PoseNet, which uses the 1D-Up approach to Conformal Geometric Algebra (CGA) to represent rotations and translations with a single mathematical object, the motor, for camera pose regression. We do so starting from PoseNet, which successfully predicts camera poses from small datasets of RGB frames. State-of-the-art methods, however, require expensive tuning to balance the orientational and translational components of the camera pose.This is usually done through complex, ad-hoc loss function to be minimized, and in some cases also requires 3D points as well as images. Our approach has the advantage of unifying the camera position and orientation through the motor. Consequently, the network searches for a single object which lives in a well-behaved 4D space with a Euclidean signature. This means that we can address the case of image-only datasets and work efficiently with a simple loss function, namely the mean squared error (MSE) between the predicted and ground truth motors. We show that it is possible to achieve high accuracy camera pose regression with a significantly simpler problem formulation. This 1D-Up approach to CGA can be employed to overcome the dichotomy between translational and orientational components in camera pose regression in a compact and elegant way.
CEN-HDR: Computationally Efficient neural Network for real-time High Dynamic Range imaging
Authors: Steven Tel, Barthélémy Heyrman, Dominique Ginhac
Date: 2023-02-10 00:00:00
Tasks: Efficient Neural Network, Knowledge Distillation

High dynamic range (HDR) imaging is still a challenging task in modern digital photography. Recent research proposes solutions that provide high-quality acquisition but at the cost of a very large number of operations and a slow inference time that prevent the implementation of these solutions on lightweight real-time systems. In this paper, we propose CEN-HDR, a new computationally efficient neural network by providing a novel architecture based on a light attention mechanism and sub-pixel convolution operations for real-time HDR imaging. We also provide an efficient training scheme by applying network compression using knowledge distillation. We performed extensive qualitative and quantitative comparisons to show that our approach produces competitive results in image quality while being faster than state-of-the-art solutions, allowing it to be practically deployed under real-time constraints. Experimental results show our method obtains a score of 43.04 mu-PSNR on the Kalantari2017 dataset with a framerate of 33 FPS using a Macbook M1 NPU.
A Novel Improved Mask RCNN for Multiple Targets Detection in the Indoor Complex Scenes
Authors: Zongmin Liu, Jirui Wang, Jie Li, Pengda Liu, Kai Ren
Date: 2023-01-07 00:00:00

With the expansive aging of global population, service robot with living assistance applied in indoor scenes will serve as a crucial role in the field of elderly care and health in the future. Service robots need to detect multiple targets when completing auxiliary tasks. However, indoor scenes are usually complex and there are many types of interference factors, leading to great challenges in the multiple targets detection. To overcome this technical difficulty, a novel improved Mask RCNN method for multiple targets detection in the indoor complex scenes is proposed in this paper. The improved model utilizes Mask RCNN as the network framework. On this basis, Convolutional Block Attention Module (CBAM) with channel mechanism and space mechanism is integrated, and the influence of different background, distance, angle and interference factors are comprehensively considered. Meanwhile, in order to evaluate the detection and identification effects of the established model, a comprehensive evaluation system based on loss function and Mean Average Precision (mAP) is established. For verification, experiments on the detection and identification effects under different distances, backgrounds, angles and interference factors were conducted. The results show that designed model improves the accuracy to a higher level and has a better anti-interference ability than other methods when the detection speed was nearly the same.
Objective Evaluation-based High-efficiency Learning Framework for Hyperspectral Image Classification
Authors: Xuming Zhang, Jian Yan, Jia Tian, Wei Li, Xingfa Gu, Qingjiu Tian
Date: 2023-01-10 00:00:00
Tasks: Hyperspectral Image Classification, image-classification, Image Classification, Vocal Bursts Intensity Prediction

Deep learning methods have been successfully applied to hyperspectral image (HSI) classification with remarkable performance. Because of limited labelled HSI data, earlier studies primarily adopted a patch-based classification framework, which divides images into overlapping patches for training and testing. However, this approach results in redundant computations and possible information leakage. In this study, we propose an objective evaluation-based high-efficiency learning framework for tiny HSI classification. This framework comprises two main parts: (i) a leakage-free balanced sampling strategy, and (ii) a modified end-to-end fully convolutional network (FCN) architecture that optimizes the trade-off between accuracy and efficiency. The leakage-free balanced sampling strategy generates balanced and non-overlapping training and testing data by partitioning an HSI and the ground truth image into small windows, each of which corresponds to one training or testing sample. The proposed high-efficiency FCN exhibits a pixel-to-pixel architecture with modifications aimed at faster inference speed and improved parameter efficiency. Experiments conducted on four representative datasets demonstrated that the proposed sampling strategy can provide objective performance evaluation and that the proposed network outperformed many state-of-the-art approaches with respect to the speed/accuracy tradeoff. Our source code is available at https://github.com/xmzhang2018.
Leveraging Inpainting for Single-Image Shadow Removal
Authors: Xiaoguang Li, Qing Guo, Rabab Abdelfattah, Di Lin, Wei Feng, Ivor Tsang, Song Wang
Date: 2023-02-10 00:00:00
Proceeding: ICCV 2023 1
Tasks: Decoder, Image Inpainting, Image Shadow Removal, Shadow Removal

Fully-supervised shadow removal methods achieve the best restoration qualities on public datasets but still generate some shadow remnants. One of the reasons is the lack of large-scale shadow & shadow-free image pairs. Unsupervised methods can alleviate the issue but their restoration qualities are much lower than those of fully-supervised methods. In this work, we find that pretraining shadow removal networks on the image inpainting dataset can reduce the shadow remnants significantly: a naive encoder-decoder network gets competitive restoration quality w.r.t. the state-of-the-art methods via only 10% shadow & shadow-free image pairs. After analyzing networks with/without inpainting pre-training via the information stored in the weight (IIW), we find that inpainting pretraining improves restoration quality in non-shadow regions and enhances the generalization ability of networks significantly. Additionally, shadow removal fine-tuning enables networks to fill in the details of shadow regions. Inspired by these observations we formulate shadow removal as an adaptive fusion task that takes advantage of both shadow removal and image inpainting. Specifically, we develop an adaptive fusion network consisting of two encoders, an adaptive fusion block, and a decoder. The two encoders are responsible for extracting the feature from the shadow image and the shadow-masked image respectively. The adaptive fusion block is responsible for combining these features in an adaptive manner. Finally, the decoder converts the adaptive fused features to the desired shadow-free result. The extensive experiments show that our method empowered with inpainting outperforms all state-of-the-art methods.
A function space perspective on stochastic shape evolution
Authors: Elizabeth Baker, Thomas Besnier, Stefan Sommer
Date: 2023-02-10 00:00:00

Modelling randomness in shape data, for example, the evolution of shapes of organisms in biology, requires stochastic models of shapes. This paper presents a new stochastic shape model based on a description of shapes as functions in a Sobolev space. Using an explicit orthonormal basis as a reference frame for the noise, the model is independent of the parameterisation of the mesh. We define the stochastic model, explore its properties, and illustrate examples of stochastic shape evolutions using the resulting numerical framework.
Deep Learning Based Object Tracking in Walking Droplet and Granular Intruder Experiments
Authors: Erdi Kara, George Zhang, Joseph J. Williams, Gonzalo Ferrandez-Quinto, Leviticus J. Rhoden, Maximilian Kim, J. Nathan Kutz, Aminur Rahman
Date: 2023-01-27 00:00:00
Tasks: object-detection, Object Detection, Object Tracking

We present a deep-learning based tracking objects of interest in walking droplet and granular intruder experiments. In a typical walking droplet experiment, a liquid droplet, known as \textit{walker}, propels itself laterally on the free surface of a vibrating bath of the same liquid. This motion is the result of the interaction between the droplets and the surface waves generated by the droplet itself after each successive bounce. A walker can exhibit a highly irregular trajectory over the course of its motion, including rapid acceleration and complex interactions with the other walkers present in the same bath. In analogy with the hydrodynamic experiments, the granular matter experiments consist of a vibrating bath of very small solid particles and a larger solid \textit{intruder}. Like the fluid droplets, the intruder interacts with and travels the domain due to the waves of the bath but tends to move much slower and much less smoothly than the droplets. When multiple intruders are introduced, they also exhibit complex interactions with each other. We leverage the state-of-art object detection model YOLO and the Hungarian Algorithm to accurately extract the trajectory of a walker or intruder in real-time. Our proposed methodology is capable of tracking individual walker(s) or intruder(s) in digital images acquired from a broad spectrum of experimental settings and does not suffer from any identity-switch issues. Thus, the deep learning approach developed in this work could be used to automatize the efficient, fast and accurate extraction of observables of interests in walking droplet and granular flow experiments. Such extraction capabilities are critically enabling for downstream tasks such as building data-driven dynamical models for the coarse-grained dynamics and interactions of the objects of interest.
A deep convolutional neural network for salt-and-pepper noise removal using selective convolutional blocks
Authors: Ahmad Ali Rafiee, Mahmoud Farhang
Date: 2023-02-10 00:00:00
Tasks: Denoising, Image Denoising, Salt-And-Pepper Noise Removal

In recent years, there has been an unprecedented upsurge in applying deep learning approaches, specifically convolutional neural networks (CNNs), to solve image denoising problems, owing to their superior performance. However, CNNs mostly rely on Gaussian noise, and there is a conspicuous lack of exploiting CNNs for salt-and-pepper (SAP) noise reduction. In this paper, we proposed a deep CNN model, namely SeConvNet, to suppress SAP noise in gray-scale and color images. To meet this objective, we introduce a new selective convolutional (SeConv) block. SeConvNet is compared to state-of-the-art SAP denoising methods using extensive experiments on various common datasets. The results illustrate that the proposed SeConvNet model effectively restores images corrupted by SAP noise and surpasses all its counterparts at both quantitative criteria and visual effects, especially at high and very high noise densities.
Deep Learning on Implicit Neural Representations of Shapes
Authors: Luca De Luigi, Adriano Cardace, Riccardo Spezialetti, Pierluigi Zama Ramirez, Samuele Salti, Luigi Di Stefano
Date: 2023-02-10 00:00:00
Tasks: Deep Learning

Implicit Neural Representations (INRs) have emerged in the last few years as a powerful tool to encode continuously a variety of different signals like images, videos, audio and 3D shapes. When applied to 3D shapes, INRs allow to overcome the fragmentation and shortcomings of the popular discrete representations used so far. Yet, considering that INRs consist in neural networks, it is not clear whether and how it may be possible to feed them into deep learning pipelines aimed at solving a downstream task. In this paper, we put forward this research problem and propose inr2vec, a framework that can compute a compact latent representation for an input INR in a single inference pass. We verify that inr2vec can embed effectively the 3D shapes represented by the input INRs and show how the produced embeddings can be fed into deep learning pipelines to solve several tasks by processing exclusively INRs.
EVC: Towards Real-Time Neural Image Compression with Mask Decay
Authors: Guo-Hua Wang, Jiahao Li, Bin Li, Yan Lu
Date: 2023-02-10 00:00:00
Tasks: Decoder, Image Compression, Representation Learning

Neural image compression has surpassed state-of-the-art traditional codecs (H.266/VVC) for rate-distortion (RD) performance, but suffers from large complexity and separate models for different rate-distortion trade-offs. In this paper, we propose an Efficient single-model Variable-bit-rate Codec (EVC), which is able to run at 30 FPS with 768x512 input images and still outperforms VVC for the RD performance. By further reducing both encoder and decoder complexities, our small model even achieves 30 FPS with 1920x1080 input images. To bridge the performance gap between our different capacities models, we meticulously design the mask decay, which transforms the large model's parameters into the small model automatically. And a novel sparsity regularization loss is proposed to mitigate shortcomings of $L_p$ regularization. Our algorithm significantly narrows the performance gap by 50% and 30% for our medium and small models, respectively. At last, we advocate the scalable encoder for neural image compression. The encoding complexity is dynamic to meet different latency requirements. We propose decaying the large encoder multiple times to reduce the residual representation progressively. Both mask decay and residual representation learning greatly improve the RD performance of our scalable encoder. Our code is at https://github.com/microsoft/DCVC.
Self-supervised learning-based cervical cytology for the triage of HPV-positive women in resource-limited settings and low-data regime
Authors: Thomas Stegmüller, Christian Abbet, Behzad Bozorgtabar, Holly Clarke, Patrick Petignat, Pierre Vassilakos, Jean-Philippe Thiran
Date: 2023-02-10 00:00:00
Tasks: Multiple Instance Learning, Self-Supervised Learning, Transfer Learning, whole slide images

Screening Papanicolaou test samples has proven to be highly effective in reducing cervical cancer-related mortality. However, the lack of trained cytopathologists hinders its widespread implementation in low-resource settings. Deep learning-based telecytology diagnosis emerges as an appealing alternative, but it requires the collection of large annotated training datasets, which is costly and time-consuming. In this paper, we demonstrate that the abundance of unlabeled images that can be extracted from Pap smear test whole slide images presents a fertile ground for self-supervised learning methods, yielding performance improvements relative to readily available pre-trained models for various downstream tasks. In particular, we propose \textbf{C}ervical \textbf{C}ell \textbf{C}opy-\textbf{P}asting ($\texttt{C}^{3}\texttt{P}$) as an effective augmentation method, which enables knowledge transfer from open-source and labeled single-cell datasets to unlabeled tiles. Not only does $\texttt{C}^{3}\texttt{P}$ outperforms naive transfer from single-cell images, but we also demonstrate its advantageous integration into multiple instance learning methods. Importantly, all our experiments are conducted on our introduced \textit{in-house} dataset comprising liquid-based cytology Pap smear images obtained using low-cost technologies. This aligns with our objective of leveraging deep learning-based telecytology for diagnosis in low-resource settings.
The LuViRA Dataset: Synchronized Vision, Radio, and Audio Sensors for Indoor Localization
Authors: Ilayda Yaman, Guoda Tian, Martin Larsson, Patrik Persson, Michiel Sandra, Alexander Dürr, Erik Tegler, Nikhil Challa, Henrik Garde, Fredrik Tufvesson, Kalle Åström, Ove Edfors, Steffen Malkowsky, Liang Liu
Date: 2023-02-10 00:00:00
Tasks: image-classification, Image Classification, Indoor Localization, Sensor Fusion

We present a synchronized multisensory dataset for accurate and robust indoor localization: the Lund University Vision, Radio, and Audio (LuViRA) Dataset. The dataset includes color images, corresponding depth maps, inertial measurement unit (IMU) readings, channel response between a 5G massive multiple-input and multiple-output (MIMO) testbed and user equipment, audio recorded by 12 microphones, and accurate six degrees of freedom (6DOF) pose ground truth of 0.5 mm. We synchronize these sensors to ensure that all data is recorded simultaneously. A camera, speaker, and transmit antenna are placed on top of a slowly moving service robot, and 89 trajectories are recorded. Each trajectory includes 20 to 50 seconds of recorded sensor data and ground truth labels. Data from different sensors can be used separately or jointly to perform localization tasks, and data from the motion capture (mocap) system is used to verify the results obtained by the localization algorithms. The main aim of this dataset is to enable research on sensor fusion with the most commonly used sensors for localization tasks. Moreover, the full dataset or some parts of it can also be used for other research areas such as channel estimation, image classification, etc. Our dataset is available at: https://github.com/ilaydayaman/LuViRA_Dataset