pwc-archive/papers-with-abstracts huggingface dataset:
Semi-Supervised Deep Regression with Uncertainty Consistency and Variational Model Ensembling via Bayesian Neural Networks
Authors: Weihang Dai, Xiaomeng Li, Kwang-Ting Cheng
Date: 2023-02-15 00:00:00
Tasks: Age Estimation, regression

Deep regression is an important problem with numerous applications. These range from computer vision tasks such as age estimation from photographs, to medical tasks such as ejection fraction estimation from echocardiograms for disease tracking. Semi-supervised approaches for deep regression are notably under-explored compared to classification and segmentation tasks, however. Unlike classification tasks, which rely on thresholding functions for generating class pseudo-labels, regression tasks use real number target predictions directly as pseudo-labels, making them more sensitive to prediction quality. In this work, we propose a novel approach to semi-supervised regression, namely Uncertainty-Consistent Variational Model Ensembling (UCVME), which improves training by generating high-quality pseudo-labels and uncertainty estimates for heteroscedastic regression. Given that aleatoric uncertainty is only dependent on input data by definition and should be equal for the same inputs, we present a novel uncertainty consistency loss for co-trained models. Our consistency loss significantly improves uncertainty estimates and allows higher quality pseudo-labels to be assigned greater importance under heteroscedastic regression. Furthermore, we introduce a novel variational model ensembling approach to reduce prediction noise and generate more robust pseudo-labels. We analytically show our method generates higher quality targets for unlabeled data and further improves training. Experiments show that our method outperforms state-of-the-art alternatives on different tasks and can be competitive with supervised methods that use full labels. Our code is available at https://github.com/xmed-lab/UCVME.
ForceFormer: Exploring Social Force and Transformer for Pedestrian Trajectory Prediction
Authors: Weicheng Zhang, Hao Cheng, Fatema T. Johora, Monika Sester
Date: 2023-02-15 00:00:00
Tasks: Autonomous Driving, Pedestrian Trajectory Prediction, Trajectory Prediction

Predicting trajectories of pedestrians based on goal information in highly interactive scenes is a crucial step toward Intelligent Transportation Systems and Autonomous Driving. The challenges of this task come from two key sources: (1) complex social interactions in high pedestrian density scenarios and (2) limited utilization of goal information to effectively associate with past motion information. To address these difficulties, we integrate social forces into a Transformer-based stochastic generative model backbone and propose a new goal-based trajectory predictor called ForceFormer. Differentiating from most prior works that simply use the destination position as an input feature, we leverage the driving force from the destination to efficiently simulate the guidance of a target on a pedestrian. Additionally, repulsive forces are used as another input feature to describe the avoidance action among neighboring pedestrians. Extensive experiments show that our proposed method achieves on-par performance measured by distance errors with the state-of-the-art models but evidently decreases collisions, especially in dense pedestrian scenarios on widely used pedestrian datasets.
Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Clouds
Authors: Tiago Cortinhal, Eren Erdal Aksoy
Date: 2023-02-15 00:00:00
Tasks: Autonomous Vehicles, Image-to-Image Translation

This work presents a new depth- and semantics-aware conditional generative model, named TITAN-Next, for cross-domain image-to-image translation in a multi-modal setup between LiDAR and camera sensors. The proposed model leverages scene semantics as a mid-level representation and is able to translate raw LiDAR point clouds to RGB-D camera images by solely relying on semantic scene segments. We claim that this is the first framework of its kind and it has practical applications in autonomous vehicles such as providing a fail-safe mechanism and augmenting available data in the target image domain. The proposed model is evaluated on the large-scale and challenging Semantic-KITTI dataset, and experimental findings show that it considerably outperforms the original TITAN-Net and other strong baselines by 23.7$\%$ margin in terms of IoU.
DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes
Authors: Shenghao Hao, Peiyuan Liu, Yibing Zhan, Kaixun Jin, Zuozhu Liu, Mingli Song, Jenq-Neng Hwang, Gaoang Wang
Date: 2023-02-15 00:00:00
Tasks: Multi-Object Tracking, Object, object-detection, Object Detection, Object Tracking

Cross-view multi-object tracking aims to link objects between frames and camera views with substantial overlaps. Although cross-view multi-object tracking has received increased attention in recent years, existing datasets still have several issues, including 1) missing real-world scenarios, 2) lacking diverse scenes, 3) owning a limited number of tracks, 4) comprising only static cameras, and 5) lacking standard benchmarks, which hinder the investigation and comparison of cross-view tracking methods. To solve the aforementioned issues, we introduce DIVOTrack: a new cross-view multi-object tracking dataset for DIVerse Open scenes with dense tracking pedestrians in realistic and non-experimental environments. Our DIVOTrack has fifteen distinct scenarios and 953 cross-view tracks, surpassing all cross-view multi-object tracking datasets currently available. Furthermore, we provide a novel baseline cross-view tracking method with a unified joint detection and cross-view tracking framework named CrossMOT, which learns object detection, single-view association, and cross-view matching with an all-in-one embedding model. Finally, we present a summary of current methodologies and a set of standard benchmarks with our DIVOTrack to provide a fair comparison and conduct a comprehensive analysis of current approaches and our proposed CrossMOT. The dataset and code are available at https://github.com/shengyuhao/DIVOTrack.
Event-guided Multi-patch Network with Self-supervision for Non-uniform Motion Deblurring
Authors: Hongguang Zhang, Limeng Zhang, Yuchao Dai, Hongdong Li, Piotr Koniusz
Date: 2023-02-14 00:00:00
Tasks: Deblurring

Contemporary deep learning multi-scale deblurring models suffer from many issues: 1) They perform poorly on non-uniformly blurred images/videos; 2) Simply increasing the model depth with finer-scale levels cannot improve deblurring; 3) Individual RGB frames contain a limited motion information for deblurring; 4) Previous models have a limited robustness to spatial transformations and noise. Below, we extend the DMPHN model by several mechanisms to address the above issues: I) We present a novel self-supervised event-guided deep hierarchical Multi-patch Network (MPN) to deal with blurry images and videos via fine-to-coarse hierarchical localized representations; II) We propose a novel stacked pipeline, StackMPN, to improve the deblurring performance under the increased network depth; III) We propose an event-guided architecture to exploit motion cues contained in videos to tackle complex blur in videos; IV) We propose a novel self-supervised step to expose the model to random transformations (rotations, scale changes), and make it robust to Gaussian noises. Our MPN achieves the state of the art on the GoPro and VideoDeblur datasets with a 40x faster runtime compared to current multi-scale methods. With 30ms to process an image at 1280x720 resolution, it is the first real-time deep motion deblurring model for 720p images at 30fps. For StackMPN, we obtain significant improvements over 1.2dB on the GoPro dataset by increasing the network depth. Utilizing the event information and self-supervision further boost results to 33.83dB.
Fine-tuning of sign language recognition models: a technical report
Authors: Maxim Novopoltsev, Leonid Verkhovtsev, Ruslan Murtazin, Dmitriy Milevich, Iuliia Zemtsova
Date: 2023-02-15 00:00:00
Tasks: Gesture Recognition, GPU, Sign Language Recognition

Sign Language Recognition (SLR) is an essential yet challenging task since sign language is performed with the fast and complex movement of hand gestures, body posture, and even facial expressions. %Skeleton Aware Multi-modal Sign Language Recognition In this work, we focused on investigating two questions: how fine-tuning on datasets from other sign languages helps improve sign recognition quality, and whether sign recognition is possible in real-time without using GPU. Three different languages datasets (American sign language WLASL, Turkish - AUTSL, Russian - RSL) have been used to validate the models. The average speed of this system has reached 3 predictions per second, which meets the requirements for the real-time scenario. This model (prototype) will benefit speech or hearing impaired people talk with other trough internet. We also investigated how the additional training of the model in another sign language affects the quality of recognition. The results show that further training of the model on the data of another sign language almost always leads to an improvement in the quality of gesture recognition. We also provide code for reproducing model training experiments, converting models to ONNX format, and inference for real-time gesture recognition.
Audio-Visual Contrastive Learning with Temporal Self-Supervision
Authors: Simon Jenni, Alexander Black, John Collomosse
Date: 2023-02-15 00:00:00
Tasks: Action Recognition, Audio Classification, Contrastive Learning, Retrieval, Self-Supervised Learning

We propose a self-supervised learning approach for videos that learns representations of both the RGB frames and the accompanying audio without human supervision. In contrast to images that capture the static scene appearance, videos also contain sound and temporal scene dynamics. To leverage the temporal and aural dimension inherent to videos, our method extends temporal self-supervision to the audio-visual setting and integrates it with multi-modal contrastive objectives. As temporal self-supervision, we pose playback speed and direction recognition in both modalities and propose intra- and inter-modal temporal ordering tasks. Furthermore, we design a novel contrastive objective in which the usual pairs are supplemented with additional sample-dependent positives and negatives sampled from the evolving feature space. In our model, we apply such losses among video clips and between videos and their temporally corresponding audio clips. We verify our model design in extensive ablation experiments and evaluate the video and audio representations in transfer experiments to action recognition and retrieval on UCF101 and HMBD51, audio classification on ESC50, and robust video fingerprinting on VGG-Sound, with state-of-the-art results.
One-Shot Face Video Re-enactment using Hybrid Latent Spaces of StyleGAN2
Authors: Trevine Oorloff, Yaser Yacoob
Date: 2023-02-15 00:00:00
Tasks: Attribute, Disentanglement, Optical Flow Estimation, Video Generation

While recent research has progressively overcome the low-resolution constraint of one-shot face video re-enactment with the help of StyleGAN's high-fidelity portrait generation, these approaches rely on at least one of the following: explicit 2D/3D priors, optical flow based warping as motion descriptors, off-the-shelf encoders, etc., which constrain their performance (e.g., inconsistent predictions, inability to capture fine facial details and accessories, poor generalization, artifacts). We propose an end-to-end framework for simultaneously supporting face attribute edits, facial motions and deformations, and facial identity control for video generation. It employs a hybrid latent-space that encodes a given frame into a pair of latents: Identity latent, $\mathcal{W}_{ID}$, and Facial deformation latent, $\mathcal{S}_F$, that respectively reside in the $W+$ and $SS$ spaces of StyleGAN2. Thereby, incorporating the impressive editability-distortion trade-off of $W+$ and the high disentanglement properties of $SS$. These hybrid latents employ the StyleGAN2 generator to achieve high-fidelity face video re-enactment at $1024^2$. Furthermore, the model supports the generation of realistic re-enactment videos with other latent-based semantic edits (e.g., beard, age, make-up, etc.). Qualitative and quantitative analyses performed against state-of-the-art methods demonstrate the superiority of the proposed approach.
Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild
Authors: Hshmat Sahak, Daniel Watson, Chitwan Saharia, David Fleet
Date: 2023-02-15 00:00:00
Tasks: Blind Super-Resolution, Denoising, Image Super-Resolution, Image-to-Image Translation, Super-Resolution

Diffusion models have shown promising results on single-image super-resolution and other image- to-image translation tasks. Despite this success, they have not outperformed state-of-the-art GAN models on the more challenging blind super-resolution task, where the input images are out of distribution, with unknown degradations. This paper introduces SR3+, a diffusion-based model for blind super-resolution, establishing a new state-of-the-art. To this end, we advocate self-supervised training with a combination of composite, parameterized degradations for self-supervised training, and noise-conditioing augmentation during training and testing. With these innovations, a large-scale convolutional architecture, and large-scale datasets, SR3+ greatly outperforms SR3. It outperforms Real-ESRGAN when trained on the same data, with a DRealSR FID score of 36.82 vs. 37.22, which further improves to FID of 32.37 with larger models, and further still with larger training sets.
Edge-weighted pFISTA-Net for MRI Reconstruction
Authors: Jianpeng Cao
Date: 2023-02-15 00:00:00
Tasks: Deep Learning, Edge Detection, MRI Reconstruction, Segmentation

Deep learning based on unrolled algorithm has served as an effective method for accelerated magnetic resonance imaging (MRI). However, many methods ignore the direct use of edge information to assist MRI reconstruction. In this work, we present the edge-weighted pFISTA-Net that directly applies the detected edge map to the soft-thresholding part of pFISTA-Net. The soft-thresholding value of different regions will be adjusted according to the edge map. Experimental results of a public brain dataset show that the proposed yields reconstructions with lower error and better artifact suppression compared with the state-of-the-art deep learning-based methods. The edge-weighted pFISTA-Net also shows robustness for different undersampling masks and edge detection operators. In addition, we extend the edge weighted structure to joint reconstruction and segmentation network and obtain improved reconstruction performance and more accurate segmentation results.
Super-Resolution of BVOC Maps by Adapting Deep Learning Methods
Authors: Antonio Giganti, Sara Mandelli, Paolo Bestagini, Marco Marcon, Stefano Tubaro
Date: 2023-02-15 00:00:00
Tasks: Deep Learning, Image Super-Resolution, Super-Resolution

Biogenic Volatile Organic Compounds (BVOCs) play a critical role in biosphere-atmosphere interactions, being a key factor in the physical and chemical properties of the atmosphere and climate. Acquiring large and fine-grained BVOC emission maps is expensive and time-consuming, so most available BVOC data are obtained on a loose and sparse sampling grid or on small regions. However, high-resolution BVOC data are desirable in many applications, such as air quality, atmospheric chemistry, and climate monitoring. In this work, we investigate the possibility of enhancing BVOC acquisitions, further explaining the relationships between the environment and these compounds. We do so by comparing the performances of several state-of-the-art neural networks proposed for image Super-Resolution (SR), adapting them to overcome the challenges posed by the large dynamic range of the emission and reduce the impact of outliers in the prediction. Moreover, we also consider realistic scenarios, considering both temporal and geographical constraints. Finally, we present possible future developments regarding SR generalization, considering the scale-invariance property and super-resolving emissions from unseen compounds.
'Aariz: A Benchmark Dataset for Automatic Cephalometric Landmark Detection and CVM Stage Classification
Authors: Muhammad Anwaar Khalid, Kanwal Zulfiqar, Ulfat Bashir, Areeba Shaheen, Rida Iqbal, Zarnab Rizwan, Ghina Rizwan, Muhammad Moazam Fraz
Date: 2023-02-15 00:00:00

The accurate identification and precise localization of cephalometric landmarks enable the classification and quantification of anatomical abnormalities. The traditional way of marking cephalometric landmarks on lateral cephalograms is a monotonous and time-consuming job. Endeavours to develop automated landmark detection systems have persistently been made, however, they are inadequate for orthodontic applications due to unavailability of a reliable dataset. We proposed a new state-of-the-art dataset to facilitate the development of robust AI solutions for quantitative morphometric analysis. The dataset includes 1000 lateral cephalometric radiographs (LCRs) obtained from 7 different radiographic imaging devices with varying resolutions, making it the most diverse and comprehensive cephalometric dataset to date. The clinical experts of our team meticulously annotated each radiograph with 29 cephalometric landmarks, including the most significant soft tissue landmarks ever marked in any publicly available dataset. Additionally, our experts also labelled the cervical vertebral maturation (CVM) stage of the patient in a radiograph, making this dataset the first standard resource for CVM classification. We believe that this dataset will be instrumental in the development of reliable automated landmark detection frameworks for use in orthodontics and beyond.
Efficiency in European Air Traffic Management -- A Fundamental Analysis of Data, Models, and Methods
Authors: Thomas Standfuss, Georg Hirte, Michael Schultz, Hartmut Fricke
Date: 2023-02-15 00:00:00
Tasks: Benchmarking, Decision Making, Management

We systematically study cornerstones that must be solved to define an air traffic control benchmarking system based on a Data Envelopment Analysis. Primarily, we examine the appropriate decision-making units, what to consider and what to avoid when choosing inputs and outputs in the case that several countries are included, and how we can identify and deal with outliers, like the Maastricht Service Provider. We argue that Air Navigation Service Providers would be a good choice of decision units within the European context. Based on that, we discuss candidates for DEA inputs and outputs and emphasize that monetary values should be excluded. We, further suggest to use super-efficiency DEA for eliminating outliers. In this context, we compare different DEA approaches and find that standard DEA is performing well.
The Effect of Points Dispersion on the k-nn Search in Random Projection Forests
Authors: Mashaan Alshammari, John Stavrakakis, Adel F. Ahmed, Masahiro Takatsuka
Date: 2022-08-01 00:00:00
Proceeding: IEEE Access 2022 8
Tasks: Quantization

Partitioning trees are efficient data structures for k -nearest neighbor search. Machine learning libraries commonly use a special type of partitioning trees called kd -trees to perform k -nn search. Unfortunately, kd -trees can be ineffective in high dimensions because they need more tree levels to decrease the vector quantization (VQ) error. Random projection trees rpTrees solve this scalability problem by using random directions to split the data. A collection of rpTrees is called rpForest. k -nn search in an rpForest is influenced by two factors: 1) the dispersion of points along the random direction and 2) the number of rpTrees in the rpForest. In this study, we investigate how these two factors affect the k -nn search with varying k values and different datasets. We found that with larger number of trees, the dispersion of points has a very limited effect on the k -nn search. One should use the original rpTree algorithm by picking a random direction regardless of the dispersion of points.
Selective Noise Suppression Methods Using Random SVPWM to Shape the Noise Spectrum of PMSMs
Authors: Jian Wen, Xiaobin Cheng, Peifeng Ji, Jun Yang, Feng Zhao
Date: 2023-02-16 00:00:00
Tasks: Position

Random pulse width modulation techniques are used in AC motors powered by two-level three-phase inverters, which cause a broadband spectrum of voltage, current, and electromagnetic force. The voltage distribution across a wide range of frequencies may increase the vibration and acoustic noise of motors. This study proposes two selective noise suppression (SNS) methods to eliminate voltage harmonics for specified frequencies. In the first method, the switching frequency is constant. The pulse position is calculated by the duty cycle of the current switching cycle. Both the pulse position and switching frequency are randomized in the second method. This involves creating a unique relationship among the switching frequency, pulse position, and duty cycle to shape the noise spectrum. Computer simulation and experimental results show that both methods effectively perform selective noise suppression at a specific frequency. The power spectrum density (PSD) using the second SNS method is more uniform near integer multiples of the switching frequency than that using random pulse width modulation techniques or the first SNS method. These methods provide a valuable reference for eliminating electromagnetic and acoustic noises at resonant frequencies in motors.
WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks
Authors: Shaowu Chen, Weize Sun, Lei Huang
Date: 2023-02-16 00:00:00
Tasks: Classification, Network Pruning, Neural Network Compression

Filter pruning has attracted increasing attention in recent years for its capacity in compressing and accelerating convolutional neural networks. Various data-independent criteria, including norm-based and relationship-based ones, were proposed to prune the most unimportant filters. However, these state-of-the-art criteria fail to fully consider the dissimilarity of filters, and thus might lead to performance degradation. In this paper, we first analyze the limitation of relationship-based criteria with examples, and then introduce a new data-independent criterion, Weighted Hybrid Criterion (WHC), to tackle the problems of both norm-based and relationship-based criteria. By taking the magnitude of each filter and the linear dependence between filters into consideration, WHC can robustly recognize the most redundant filters, which can be safely pruned without introducing severe performance degradation to networks. Extensive pruning experiments in a simple one-shot manner demonstrate the effectiveness of the proposed WHC. In particular, WHC can prune ResNet-50 on ImageNet with more than 42% of floating point operations reduced without any performance loss in top-5 accuracy.