pwc-archive/papers-with-abstracts huggingface dataset:
Digital Fingerprinting of Microstructures
Authors: Michael D. White, Alexander Tarakanov, Christopher P. Race, Philip J. Withers, Kody J. H. Law
Date: 2022-03-25 00:00:00
Tasks: Dimensionality Reduction, Transfer Learning

Finding efficient means of fingerprinting microstructural information is a critical step towards harnessing data-centric machine learning approaches. A statistical framework is systematically developed for compressed characterisation of a population of images, which includes some classical computer vision methods as special cases. The focus is on materials microstructure. The ultimate purpose is to rapidly fingerprint sample images in the context of various high-throughput design/make/test scenarios. This includes, but is not limited to, quantification of the disparity between microstructures for quality control, classifying microstructures, predicting materials properties from image data and identifying potential processing routes to engineer new materials with specific properties. Here, we consider microstructure classification and utilise the resulting features over a range of related machine learning tasks, namely supervised, semi-supervised, and unsupervised learning. The approach is applied to two distinct datasets to illustrate various aspects and some recommendations are made based on the findings. In particular, methods that leverage transfer learning with convolutional neural networks (CNNs), pretrained on the ImageNet dataset, are generally shown to outperform other methods. Additionally, dimensionality reduction of these CNN-based fingerprints is shown to have negligible impact on classification accuracy for the supervised learning approaches considered. In situations where there is a large dataset with only a handful of images labelled, graph-based label propagation to unlabelled data is shown to be favourable over discarding unlabelled data and performing supervised learning. In particular, label propagation by Poisson learning is shown to be highly effective at low label rates.
Salt Detection Using Segmentation of Seismic Image
Authors: Mrinmoy Sarkar
Date: 2022-03-25 00:00:00
Tasks: Object Recognition, Seismic Imaging

In this project, a state-of-the-art deep convolution neural network (DCNN) is presented to segment seismic images for salt detection below the earth's surface. Detection of salt location is very important for starting mining. Hence, a seismic image is used to detect the exact salt location under the earth's surface. However, precisely detecting the exact location of salt deposits is difficult. Therefore, professional seismic imaging still requires expert human interpretation of salt bodies. This leads to very subjective, highly variable renderings. Hence, to create the most accurate seismic images and 3D renderings, we need a robust algorithm that automatically and accurately identifies if a surface target is a salt or not. Since the performance of DCNN is well-known and well-established for object recognition in images, DCNN is a very good choice for this particular problem and being successfully applied to a dataset of seismic images in which each pixel is labeled as salt or not. The result of this algorithm is promising.
FReSCO: Flow Reconstruction and Segmentation for low latency Cardiac Output monitoring using deep artifact suppression and segmentation
Authors: Olivier Jaubert, Javier Montalt-Tordera, James Brown, Daniel Knight, Simon Arridge, Jennifer Steeden, Vivek Muthurangu
Date: 2022-03-25 00:00:00
Tasks: compressed sensing, Segmentation

Purpose: Real-time monitoring of cardiac output (CO) requires low latency reconstruction and segmentation of real-time phase contrast MR (PCMR), which has previously been difficult to perform. Here we propose a deep learning framework for 'Flow Reconstruction and Segmentation for low latency Cardiac Output monitoring' (FReSCO). Methods: Deep artifact suppression and segmentation U-Nets were independently trained. Breath hold spiral PCMR data (n=516) was synthetically undersampled using a variable density spiral sampling pattern and gridded to create aliased data for training of the artifact suppression U-net. A subset of the data (n=96) was segmented and used to train the segmentation U-net. Real-time spiral PCMR was prospectively acquired and then reconstructed and segmented using the trained models (FReSCO) at low latency at the scanner in 10 healthy subjects during rest, exercise and recovery periods. CO obtained via FReSCO was compared to a reference rest CO and rest and exercise Compressed Sensing (CS) CO. Results: FReSCO was demonstrated prospectively at the scanner. Beat-to-beat heartrate, stroke volume and CO could be visualized with a mean latency of 622ms. No significant differences were noted when compared to reference at rest (Bias = -0.21+-0.50 L/min, p=0.246) or CS at peak exercise (Bias=0.12+-0.48 L/min, p=0.458). Conclusion: FReSCO was successfully demonstrated for real-time monitoring of CO during exercise and could provide a convenient tool for assessment of the hemodynamic response to a range of stressors.
Analysis of the use of color and its emotional relationship in visual creations based on experiences during the context of the COVID-19 pandemic
Authors: César González-Martín, Miguel Carrasco, Germán Oviedo
Date: 2022-03-25 00:00:00

Color is a complex communicative element that helps us understand and evaluate our environment. At the level of artistic creation, this component influences both the formal aspects of the composition and the symbolic weight, directly affecting the construction and transmission of the message that you want to communicate, creating a specific emotional reaction. During the COVID-19 pandemic, people generated countless images transmitting this event's subjective experiences. Using the repository of images created in the Instagram account CAM (The COVID Art Museum), we propose a methodology to understand the use of color and its emotional relationship in this context. The process considers two stages in parallel that are then combined. First, emotions are extracted and classified from the CAM dataset images through a convolutional neural network. Second, we extract the colors and their harmonies through a clustering process. Once both processes are completed, we combine the results generating an expanded discussion on the usage of color, harmonies, and emotion. The results indicate that warm colors are prevalent in the sample, with a preference for analog compositions over complementary ones. The relationship between emotions and these compositions shows a trend in positive emotions, reinforced by the results of the algorithm a priori and the emotional relationship analysis of the attributes of color (hue, chroma, and lighting).
Continuous Dynamic-NeRF: Spline-NeRF
Authors: Julian Knodt
Date: 2022-03-25 00:00:00
Tasks: NeRF

The problem of reconstructing continuous functions over time is important for problems such as reconstructing moving scenes, and interpolating between time steps. Previous approaches that use deep-learning rely on regularization to ensure that reconstructions are approximately continuous, which works well on short sequences. As sequence length grows, though, it becomes more difficult to regularize, and it becomes less feasible to learn only through regularization. We propose a new architecture for function reconstruction based on classical Bezier splines, which ensures $C^0$ and $C^1$-continuity, where $C^0$ continuity is that $\forall c:\lim\limits_{x\to c} f(x) = f(c)$, or more intuitively that there are no breaks at any point in the function. In order to demonstrate our architecture, we reconstruct dynamic scenes using Neural Radiance Fields, but hope it is clear that our approach is general and can be applied to a variety of problems. We recover a Bezier spline $B(\beta, t\in[0,1])$, parametrized by the control points $\beta$. Using Bezier splines ensures reconstructions have $C^0$ and $C^1$ continuity, allowing for guaranteed interpolation over time. We reconstruct $\beta$ with a multi-layer perceptron (MLP), blending machine learning with classical animation techniques. All code is available at https://github.com/JulianKnodt/nerf_atlas, and datasets are from prior work.
Playing Lottery Tickets in Style Transfer Models
Authors: Meihao Kong, Jing Huo, Wenbin Li, Jing Wu, Yu-Kun Lai, Yang Gao
Date: 2022-03-25 00:00:00
Tasks: Style Transfer

Style transfer has achieved great success and attracted a wide range of attention from both academic and industrial communities due to its flexible application scenarios. However, the dependence on a pretty large VGG-based autoencoder leads to existing style transfer models having high parameter complexities, which limits their applications on resource-constrained devices. Compared with many other tasks, the compression of style transfer models has been less explored. Recently, the lottery ticket hypothesis (LTH) has shown great potential in finding extremely sparse matching subnetworks which can achieve on par or even better performance than the original full networks when trained in isolation. In this work, we for the first time perform an empirical study to verify whether such trainable matching subnetworks also exist in style transfer models. Specifically, we take two most popular style transfer models, i.e., AdaIN and SANet, as the main testbeds, which represent global and local transformation based style transfer methods respectively. We carry out extensive experiments and comprehensive analysis, and draw the following conclusions. (1) Compared with fixing the VGG encoder, style transfer models can benefit more from training the whole network together. (2) Using iterative magnitude pruning, we find the matching subnetworks at 89.2% sparsity in AdaIN and 73.7% sparsity in SANet, which demonstrates that style transfer models can play lottery tickets too. (3) The feature transformation module should also be pruned to obtain a much sparser model without affecting the existence and quality of the matching subnetworks. (4) Besides AdaIN and SANet, other models such as LST, MANet, AdaAttN and MCCNet can also play lottery tickets, which shows that LTH can be generalized to various style transfer models.
Spatially Multi-conditional Image Generation
Authors: Ritika Chakraborty, Nikola Popovic, Danda Pani Paudel, Thomas Probst, Luc van Gool
Date: 2022-03-25 00:00:00
Tasks: Conditional Image Generation, Image Generation, Missing Labels

In most scenarios, conditional image generation can be thought of as an inversion of the image understanding process. Since generic image understanding involves solving multiple tasks, it is natural to aim at generating images via multi-conditioning. However, multi-conditional image generation is a very challenging problem due to the heterogeneity and the sparsity of the (in practice) available conditioning labels. In this work, we propose a novel neural architecture to address the problem of heterogeneity and sparsity of the spatially multi-conditional labels. Our choice of spatial conditioning, such as by semantics and depth, is driven by the promise it holds for better control of the image generation process. The proposed method uses a transformer-like architecture operating pixel-wise, which receives the available labels as input tokens to merge them in a learned homogeneous space of labels. The merged labels are then used for image generation via conditional generative adversarial training. In this process, the sparsity of the labels is handled by simply dropping the input tokens corresponding to the missing labels at the desired locations, thanks to the proposed pixel-wise operating architecture. Our experiments on three benchmark datasets demonstrate the clear superiority of our method over the state-of-the-art and compared baselines. The source code will be made publicly available.
Versatile Multi-Modal Pre-Training for Human-Centric Perception
Authors: Fangzhou Hong, Liang Pan, Zhongang Cai, Ziwei Liu
Date: 2022-03-25 00:00:00
Proceeding: CVPR 2022 1
Tasks: Contrastive Learning, Human Parsing, Representation Learning

Human-centric perception plays a vital role in vision and graphics. But their data annotations are prohibitively expensive. Therefore, it is desirable to have a versatile pre-train model that serves as a foundation for data-efficient downstream tasks transfer. To this end, we propose the Human-Centric Multi-Modal Contrastive Learning framework HCMoCo that leverages the multi-modal nature of human data (e.g. RGB, depth, 2D keypoints) for effective representation learning. The objective comes with two main challenges: dense pre-train for multi-modality data, efficient usage of sparse human priors. To tackle the challenges, we design the novel Dense Intra-sample Contrastive Learning and Sparse Structure-aware Contrastive Learning targets by hierarchically learning a modal-invariant latent space featured with continuous and ordinal feature distribution and structure-aware semantic consistency. HCMoCo provides pre-train for different modalities by combining heterogeneous datasets, which allows efficient usage of existing task-specific human data. Extensive experiments on four downstream tasks of different modalities demonstrate the effectiveness of HCMoCo, especially under data-efficient settings (7.16% and 12% improvement on DensePose Estimation and Human Parsing). Moreover, we demonstrate the versatility of HCMoCo by exploring cross-modality supervision and missing-modality inference, validating its strong ability in cross-modal association and reasoning.
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
Authors: Ziqian Bai, Timur Bagautdinov, Javier Romero, Michael Zollhöfer, Ping Tan, Shunsuke Saito
Date: 2022-03-25 00:00:00
Tasks: 3D Human Dynamics, 3D Human Reconstruction, Human Dynamics

Neural fields such as implicit surfaces have recently enabled avatar modeling from raw scans without explicit temporal correspondences. In this work, we exploit autoregressive modeling to further extend this notion to capture dynamic effects, such as soft-tissue deformations. Although autoregressive models are naturally capable of handling dynamics, it is non-trivial to apply them to implicit representations, as explicit state decoding is infeasible due to prohibitive memory requirements. In this work, for the first time, we enable autoregressive modeling of implicit avatars. To reduce the memory bottleneck and efficiently model dynamic implicit surfaces, we introduce the notion of articulated observer points, which relate implicit states to the explicit surface of a parametric human body model. We demonstrate that encoding implicit surfaces as a set of height fields defined on articulated observer points leads to significantly better generalization compared to a latent representation. The experiments show that our approach outperforms the state of the art, achieving plausible dynamic deformations even for unseen motions. https://zqbai-jeremy.github.io/autoavatar
Deep learning for laboratory earthquake prediction and autoregressive forecasting of fault zone stress
Authors: Laura Laurenti, Elisa Tinti, Fabio Galasso, Luca Franco, Chris Marone
Date: 2022-03-24 00:00:00
Tasks: Earthquake prediction, Prediction

Earthquake forecasting and prediction have long and in some cases sordid histories but recent work has rekindled interest based on advances in early warning, hazard assessment for induced seismicity and successful prediction of laboratory earthquakes. In the lab, frictional stick-slip events provide an analog for earthquakes and the seismic cycle. Labquakes are ideal targets for machine learning (ML) because they can be produced in long sequences under controlled conditions. Recent works show that ML can predict several aspects of labquakes using fault zone acoustic emissions. Here, we generalize these results and explore deep learning (DL) methods for labquake prediction and autoregressive (AR) forecasting. DL improves existing ML methods of labquake prediction. AR methods allow forecasting at future horizons via iterative predictions. We demonstrate that DL models based on Long-Short Term Memory (LSTM) and Convolution Neural Networks predict labquakes under several conditions, and that fault zone stress can be predicted with fidelity, confirming that acoustic energy is a fingerprint of fault zone stress. We predict also time to start of failure (TTsF) and time to the end of Failure (TTeF) for labquakes. Interestingly, TTeF is successfully predicted in all seismic cycles, while the TTsF prediction varies with the amount of preseismic fault creep. We report AR methods to forecast the evolution of fault stress using three sequence modeling frameworks: LSTM, Temporal Convolution Network and Transformer Network. AR forecasting is distinct from existing predictive models, which predict only a target variable at a specific time. The results for forecasting beyond a single seismic cycle are limited but encouraging. Our ML/DL models outperform the state-of-the-art and our autoregressive model represents a novel framework that could enhance current methods of earthquake forecasting.
Microstructure Surface Reconstruction from SEM Images: An Alternative to Digital Image Correlation (DIC)
Authors: Khalid El-Awady
Date: 2022-03-25 00:00:00
Tasks: Optical Flow Estimation, Surface Reconstruction

We reconstruct a 3D model of the surface of a material undergoing fatigue testing and experiencing cracking. Specifically we reconstruct the surface depth (out of plane intrusions and extrusions) and lateral (in-plane) motion from multiple views of the sample at the end of the experiment, combined with a reverse optical flow propagation backwards in time that utilizes interim single view images. These measurements can be mapped to a material strain tensor which helps to understand material life and predict failure. This approach offers an alternative to the commonly used Digital Image Correlation (DIC) technique which relies on tracking a speckle pattern applied to the material surface. DIC only produces in-plane (2D) measurements whereas our approach is 3D and non-invasive (requires no pattern being applied to the material).
RD-Optimized Trit-Plane Coding of Deep Compressed Image Latent Tensors
Authors: Seungmin Jeon, Jae-Han Lee, Chang-Su Kim
Date: 2022-03-25 00:00:00
Tasks: Decoder

DPICT is the first learning-based image codec supporting fine granular scalability. In this paper, we describe how to implement two key components of DPICT efficiently: trit-plane slicing and rate-distortion-optimized (RD-optimized) coding. In DPICT, we transform an image into a latent tensor, represent the tensor in ternary digits (trits), and encode the trits in the decreasing order of significance. For entropy encoding, it is necessary to compute the probability of each trit, which demands high time complexity in both the encoder and the decoder. To reduce the complexity, we develop a parallel computing scheme for the probabilities, which is described in detail with pseudo-codes. Moreover, we compare the trit-plane slicing in DPICT with the alternative bit-plane slicing. Experimental results show that the time complexity is reduced significantly by the parallel computing and that the trit-plane slicing provides better RD performances than the bit-plane slicing.
SeCo: Separating Unknown Musical Visual Sounds with Consistency Guidance
Authors: Xinchi Zhou, Dongzhan Zhou, Wanli Ouyang, Hang Zhou, Ziwei Liu, Di Hu
Date: 2022-03-25 00:00:00

Recent years have witnessed the success of deep learning on the visual sound separation task. However, existing works follow similar settings where the training and testing datasets share the same musical instrument categories, which to some extent limits the versatility of this task. In this work, we focus on a more general and challenging scenario, namely the separation of unknown musical instruments, where the categories in training and testing phases have no overlap with each other. To tackle this new setting, we propose the Separation-with-Consistency (SeCo) framework, which can accomplish the separation on unknown categories by exploiting the consistency constraints. Furthermore, to capture richer characteristics of the novel melodies, we devise an online matching strategy, which can bring stable enhancements with no cost of extra parameters. Experiments demonstrate that our SeCo framework exhibits strong adaptation ability on the novel musical categories and outperforms the baseline methods by a significant margin.
On the performance of preconditioned methods to solve \(L^p\)-norm phase unwrapping
Authors: Ricardo Legarda-Saenz, Carlos Brito-Loeza, Arturo Espinosa-Romero
Date: 2022-03-25 00:00:00

In this paper, we analyze and evaluate suitable preconditioning techniques to improve the performance of the $L^p$-norm phase unwrapping method. We consider five preconditioning techniques commonly found in the literature, and analyze their performance with different sizes of wrapped-phase maps. Keywords.- Phase unwrapping, $L^p$-norm based method, Preconditioning techniques.
Visual-based Safe Landing for UAVs in Populated Areas: Real-time Validation in Virtual Environments
Authors: Hector Tovanche-Picon, Javier Gonzalez-Trejo, Angel Flores-Abad, Diego Mercado-Ravell
Date: 2022-03-25 00:00:00

Safe autonomous landing for Unmanned Aerial Vehicles (UAVs) in populated areas is a crucial aspect for successful urban deployment, particularly in emergency landing situations. Nonetheless, validating autonomous landing in real scenarios is a challenging task involving a high risk of injuring people. In this work, we propose a framework for real-time safe and thorough evaluation of vision-based autonomous landing in populated scenarios, using photo-realistic virtual environments. We propose to use the Unreal graphics engine coupled with the AirSim plugin for drone's simulation, and evaluate autonomous landing strategies based on visual detection of Safe Landing Zones (SLZ) in populated scenarios. Then, we study two different criteria for selecting the "best" SLZ, and evaluate them during autonomous landing of a virtual drone in different scenarios and conditions, under different distributions of people in urban scenes, including moving people. We evaluate different metrics to quantify the performance of the landing strategies, establishing a baseline for comparison with future works in this challenging task, and analyze them through an important number of randomized iterations. The study suggests that the use of the autonomous landing algorithms considerably helps to prevent accidents involving humans, which may allow to unleash the full potential of drones in urban environments near to people.
A Single Image Dehazing Technique Using the Dual Transmission Maps Strategy and Gradient-Domain Guided Image Filtering
Authors: S. M. Ehsan, M. Imran, A. Ullah and E. Elbasi
Date: 2021-07-17 00:00:00
Proceeding: journal 2021 7
Tasks: Image Dehazing, Single Image Dehazing

In this paper, a single image dehazing technique using dual transmission maps strategy and gradient-domain guided image filtering is presented. A new strategy is adopted to compute the dual transmission maps using the dark channel and atmospheric light. Further, the transmission maps are refined to remove any remaining ill effects using the gradient-domain-guided filter. Finally, using the dark channel, atmospheric light, and refined transmission map, the haze-free image is obtained. The dual transmission maps strategy not only removes halo artifacts and reduces the saturation but also ensures the natural appearance in the recovered images. Furthermore, the proposed scheme is evaluated using a wide range of images and compared with state-of-the-art schemes. The comparison shows the superiority of the proposed technique in terms of recovering haze-free images.