pwc-archive/papers-with-abstracts huggingface dataset:
Unified Pretraining Framework for Document Understanding
Authors: Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Nikolaos Barmpalios, Rajiv Jain, Ani Nenkova, Tong Sun
Date: 2022-04-22 00:00:00
Tasks: Document Layout Analysis, document understanding, Self-Supervised Learning

Document intelligence automates the extraction of information from documents and supports many business applications. Recent self-supervised learning methods on large-scale unlabeled document datasets have opened up promising directions towards reducing annotation efforts by training models with self-supervised objectives. However, most of the existing document pretraining methods are still language-dominated. We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. Each input element is composed of words and visual features from a semantic region of the input document image. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses, encouraging the representation to model sentences, learn similarities, and align modalities. Extensive empirical analysis demonstrates that the pretraining procedure learns better joint representations and leads to improvements in downstream tasks.
A Multi-level Alignment Training Scheme for Video-and-Language Grounding
Authors: Yubo Zhang, Feiyang Niu, Qing Ping, Govind Thattai
Date: 2022-04-22 00:00:00
Tasks: Retrieval, Semantic Similarity, Semantic Textual Similarity

To solve video-and-language grounding tasks, the key is for the network to understand the connection between the two modalities. For a pair of video and language description, their semantic relation is reflected by their encodings' similarity. A good multi-modality encoder should be able to well capture both inputs' semantics and encode them in the shared feature space where embedding distance gets properly translated into their semantic similarity. In this work, we focused on this semantic connection between video and language, and developed a multi-level alignment training scheme to directly shape the encoding process. Global and segment levels of video-language alignment pairs were designed, based on the information similarity ranging from high-level context to fine-grained semantics. The contrastive loss was used to contrast the encodings' similarities between the positive and negative alignment pairs, and to ensure the network is trained in such a way that similar information is encoded closely in the shared feature space while information of different semantics is kept apart. Our multi-level alignment training can be applied to various video-and-language grounding tasks. Together with the task-specific training loss, our framework achieved comparable performance to previous state-of-the-arts on multiple video QA and retrieval datasets.
Revealing Occlusions with 4D Neural Fields
Authors: Basile Van Hoorick, Purva Tendulka, Didac Suris, Dennis Park, Simon Stent, Carl Vondrick
Date: 2022-04-22 00:00:00
Proceeding: CVPR 2022 1
Tasks: Video Understanding

For computer vision systems to operate in dynamic situations, they need to be able to represent and reason about object permanence. We introduce a framework for learning to estimate 4D visual representations from monocular RGB-D, which is able to persist objects, even once they become obstructed by occlusions. Unlike traditional video representations, we encode point clouds into a continuous representation, which permits the model to attend across the spatiotemporal context to resolve occlusions. On two large video datasets that we release along with this paper, our experiments show that the representation is able to successfully reveal occlusions for several tasks, without any architectural changes. Visualizations show that the attention mechanism automatically learns to follow occluded objects. Since our approach can be trained end-to-end and is easily adaptable, we believe it will be useful for handling occlusions in many video understanding tasks. Data, code, and models are available at https://occlusions.cs.columbia.edu/.
Label a Herd in Minutes: Individual Holstein-Friesian Cattle Identification
Authors: Jing Gao, Tilo Burghardt, Neill W. Campbell
Date: 2022-04-22 00:00:00
Tasks: Active Learning, Metric Learning

We describe a practically evaluated approach for training visual cattle ID systems for a whole farm requiring only ten minutes of labelling effort. In particular, for the task of automatic identification of individual Holstein-Friesians in real-world farm CCTV, we show that self-supervision, metric learning, cluster analysis, and active learning can complement each other to significantly reduce the annotation requirements usually needed to train cattle identification frameworks. Evaluating the approach on the test portion of the publicly available Cows2021 dataset, for training we use 23,350 frames across 435 single individual tracklets generated by automated oriented cattle detection and tracking in operational farm footage. Self-supervised metric learning is first employed to initialise a candidate identity space where each tracklet is considered a distinct entity. Grouping entities into equivalence classes representing cattle identities is then performed by automated merging via cluster analysis and active learning. Critically, we identify the inflection point at which automated choices cannot replicate improvements based on human intervention to reduce annotation to a minimum. Experimental results show that cluster analysis and a few minutes of labelling after automated self-supervision can improve the test identification accuracy of 153 identities to 92.44% (ARI=0.93) from the 74.9% (ARI=0.754) obtained by self-supervision only. These promising results indicate that a tailored combination of human and machine reasoning in visual cattle ID pipelines can be highly effective whilst requiring only minimal labelling effort. We provide all key source code and network weights with this paper for easy result reproduction.
Transfer Learning from Synthetic In-vitro Soybean Pods Dataset for In-situ Segmentation of On-branch Soybean Pod
Authors: Si Yang, Lihua Zheng, Xieyuanli Chen, Laura Zabawa, Man Zhang, Minjuan Wang
Date: 2022-04-22 00:00:00
Tasks: Image Generation, Instance Segmentation, Segmentation, Semantic Segmentation, Transfer Learning

The mature soybean plants are of complex architecture with pods frequently touching each other, posing a challenge for in-situ segmentation of on-branch soybean pods. Deep learning-based methods can achieve accurate training and strong generalization capabilities, but it demands massive labeled data, which is often a limitation, especially for agricultural applications. As lacking the labeled data to train an in-situ segmentation model for on-branch soybean pods, we propose a transfer learning from synthetic in-vitro soybean pods. First, we present a novel automated image generation method to rapidly generate a synthetic in-vitro soybean pods dataset with plenty of annotated samples. The in-vitro soybean pods samples are overlapped to simulate the frequently physically touching of on-branch soybean pods. Then, we design a two-step transfer learning. In the first step, we finetune an instance segmentation network pretrained by a source domain (MS COCO dataset) with a synthetic target domain (in-vitro soybean pods dataset). In the second step, transferring from simulation to reality is performed by finetuning on a few real-world mature soybean plant samples. The experimental results show the effectiveness of the proposed two-step transfer learning method, such that AP$_{50}$ was 0.80 for the real-world mature soybean plant test dataset, which is higher than that of direct adaptation and its AP$_{50}$ was 0.77. Furthermore, the visualizations of in-situ segmentation results of on-branch soybean pods show that our method performs better than other methods, especially when soybean pods overlap densely.
Generative Sampling in Bundle Tractography using Autoencoders (GESTA)
Authors: Jon Haitz Legarreta, Laurent Petit, Pierre-Marc Jodoin, Maxime Descoteaux
Date: 2022-04-22 00:00:00
Tasks: Anatomy

Current tractography methods use the local orientation information to propagate streamlines from seed locations. Many such seeds provide streamlines that stop prematurely or fail to map the true white matter pathways because some bundles are "harder-to-track" than others. This results in tractography reconstructions with poor white and gray matter spatial coverage. In this work, we propose a generative, autoencoder-based method, named GESTA (Generative Sampling in Bundle Tractography using Autoencoders), that produces streamlines achieving better spatial coverage. Compared to other deep learning methods, our autoencoder-based framework uses a single model to generate streamlines in a bundle-wise fashion, and does not require to propagate local orientations. GESTA produces new and complete streamlines for any given white matter bundle, including hard-to-track bundles. Applied on top of a given tractogram, GESTA is shown to be effective in improving the white matter volume coverage in poorly populated bundles, both on synthetic and human brain in vivo data. Our streamline evaluation framework ensures that the streamlines produced by GESTA are anatomically plausible and fit well to the local diffusion signal. The streamline evaluation criteria assess anatomy (white matter coverage), local orientation alignment (direction), and geometry features of streamlines, and optionally, gray matter connectivity. GESTA is thus a novel deep generative bundle tractography method that can be used to improve the tractography reconstruction of the white matter.
Translating Clinical Delineation of Diabetic Foot Ulcers into Machine Interpretable Segmentation
Authors: Connah Kendrick, Bill Cassidy, Joseph M. Pappachan, Claire O'Shea, Cornelious J. Fernandez, Elias Chacko, Koshy Jacob, Neil D. Reeves, Moi Hoon Yap
Date: 2022-04-22 00:00:00
Tasks: Management

Diabetic foot ulcer is a severe condition that requires close monitoring and management. For training machine learning methods to auto-delineate the ulcer, clinical staff must provide ground truth annotations. In this paper, we propose a new diabetic foot ulcers dataset, namely DFUC2022, the largest segmentation dataset where ulcer regions were manually delineated by clinicians. We assess whether the clinical delineations are machine interpretable by deep learning networks or if image processing refined contour should be used. By providing benchmark results using a selection of popular deep learning algorithms, we draw new insights into the limitations of DFU wound delineation and report on the associated issues. This paper provides some observations on baseline models to facilitate DFUC2022 Challenge in conjunction with MICCAI 2022. The leaderboard will be ranked by Dice score, where the best FCN-based method is 0.5708 and DeepLabv3+ achieved the best score of 0.6277. This paper demonstrates that image processing using refined contour as ground truth can provide better agreement with machine predicted results. DFUC2022 will be released on the 27th April 2022.
Exoplanet Cartography using Convolutional Neural Networks
Authors: K. Meinke, D. M. Stam, P. M. Visser
Date: 2022-04-25 00:00:00
Tasks: Retrieval

In the near-future, dedicated telescopes observe Earth-like exoplanets in reflected light, allowing their characterization. Because of the huge distances, every exoplanet will be a single pixel, but temporal variations in its spectral flux hold information about the planet's surface and atmosphere. We test convolutional neural networks for retrieving a planet's rotation axis, surface and cloud map from simulated single-pixel flux and polarization observations. We investigate the assumption that the planets reflect Lambertian in the retrieval while their actual reflection is bidirectional, and of including polarization in retrievals. We simulate observations along a planet's orbit using a radiative transfer algorithm that includes polarization and bidirectional reflection by vegetation, desert, oceans, water clouds, and Rayleigh scattering in 6 spectral bands from 400 to 800 nm, at various photon noise levels. The surface-types and cloud patterns of the facets covering a model planet are based on probability distributions. Our networks are trained with simulated observations of millions of planets before retrieving maps of test planets. The neural networks can constrain rotation axes with a mean squared error (MSE) as small as 0.0097, depending on the orbital inclination. On a bidirectionally reflecting planet, 92% of ocean and 85% of vegetation, desert, and cloud facets are correctly retrieved, in the absence of noise. With realistic noise, it should still be possible to retrieve the main map features with a dedicated telescope. Except for face-on orbits, a network trained with Lambertian reflecting planets, yields significant retrieval errors when given observations of bidirectionally reflecting planets, in particular, brightness artefacts around a planet's pole. Including polarization improves retrieving the rotation axis and the accuracy of the retrieval of ocean and cloud facets.
Online Deep Learning from Doubly-Streaming Data
Authors: Heng Lian, John Scovil Atwood, BoJian Hou, Jian Wu, Yi He
Date: 2022-04-25 00:00:00
Tasks: Deep Learning

This paper investigates a new online learning problem with doubly-streaming data, where the data streams are described by feature spaces that constantly evolve, with new features emerging and old features fading away. The challenges of this problem are two folds: 1) Data samples ceaselessly flowing in may carry shifted patterns over time, requiring learners to update hence adapt on-the-fly. 2) Newly emerging features are described by very few samples, resulting in weak learners that tend to make error predictions. A plausible idea to overcome the challenges is to establish relationship between the pre-and-post evolving feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional media streams with complex feature interplay, which suffers an tradeoff between onlineness (biasing shallow learners) and expressiveness(requiring deep learners). Motivated by this, we propose a novel OLD^3S paradigm, where a shared latent subspace is discovered to summarize information from the old and new feature spaces, building intermediate feature mapping relationship. A key trait of OLD^3S is to treat the model capacity as a learnable semantics, yields optimal model depth and parameters jointly, in accordance with the complexity and non-linearity of the input data streams in an online fashion. Both theoretical analyses and empirical studies substantiate the viability and effectiveness of our proposal.
Can Rationalization Improve Robustness?
Authors: Howard Chen, Jacqueline He, Karthik Narasimhan, Danqi Chen
Date: 2022-04-25 00:00:00
Proceeding: NAACL 2022 7
Tasks: Sentence

A growing line of work has investigated the development of neural NLP models that can produce rationales--subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models can also provide robustness to adversarial attacks in addition to their interpretable nature. Since these models need to first generate rationales ("rationalizer") before making predictions ("predictor"), they have the potential to ignore noise or adversarially added text by simply masking it out of the generated rationale. To this end, we systematically generate various types of 'AddText' attacks for both token and sentence-level rationalization tasks, and perform an extensive empirical evaluation of state-of-the-art rationale models across five different tasks. Our experiments reveal that the rationale models show the promise to improve robustness, while they struggle in certain scenarios--when the rationalizer is sensitive to positional bias or lexical choices of attack text. Further, leveraging human rationale as supervision does not always translate to better performance. Our study is a first step towards exploring the interplay between interpretability and robustness in the rationalize-then-predict framework.
Blind Equalization and Channel Estimation in Coherent Optical Communications Using Variational Autoencoders
Authors: Vincent Lauinger, Fred Buchali, Laurent Schmalen
Date: 2022-04-25 00:00:00
Tasks: Variational Inference

We investigate the potential of adaptive blind equalizers based on variational inference for carrier recovery in optical communications. These equalizers are based on a low-complexity approximation of maximum likelihood channel estimation. We generalize the concept of variational autoencoder (VAE) equalizers to higher order modulation formats encompassing probabilistic constellation shaping (PCS), ubiquitous in optical communications, oversampling at the receiver, and dual-polarization transmission. Besides black-box equalizers based on convolutional neural networks, we propose a model-based equalizer based on a linear butterfly filter and train the filter coefficients using the variational inference paradigm. As a byproduct, the VAE also provides a reliable channel estimation. We analyze the VAE in terms of performance and flexibility over a classical additive white Gaussian noise (AWGN) channel with inter-symbol interference (ISI) and over a dispersive linear optical dual-polarization channel. We show that it can extend the application range of blind adaptive equalizers by outperforming the state-of-the-art constant-modulus algorithm (CMA) for PCS for both fixed but also time-varying channels. The evaluation is accompanied with a hyperparameter analysis.
A feasibility study proposal of the predictive model to enable the prediction of population susceptibility to COVID-19 by analysis of vaccine utilization for advising deployment of a booster dose
Authors: Chottiwatt Jittprasong
Date: 2022-04-25 00:00:00

With the present highly infectious dominant SARS-CoV-2 strain of B1.1.529 or Omicron spreading around the globe, there is concern that the COVID-19 pandemic will not end soon and that it will be a race against time until a more contagious and virulent variant emerges. One of the most promising approaches for preventing virus propagation is to maintain continuous high vaccination efficacy among the population, thereby strengthening the population protective effect and preventing the majority of infection in the vaccinated population, as is known to occur with the Omicron variant frequently. Countries must structure vaccination programs in accordance with their populations' susceptibility to infection, optimizing vaccination efforts by delivering vaccines progressively enough to protect the majority of the population. We present a feasibility study proposal for maintaining optimal continuous vaccination by assessing the susceptible population, the decline of vaccine efficacy in the population, and advising booster dosage deployment to maintain the population's protective efficacy through the use of a predictive model. Numerous studies have been conducted in the direction of analyzing vaccine utilization; however, very little study has been conducted to substantiate the optimal deployment of booster dosage vaccination with the help of a predictive model based on machine learning algorithms.
On the Performance of Machine Learning Methods for Breakthrough Curve Prediction
Authors: Daria Fokina, Oleg Iliev, Pavel Toktaliev, Ivan Oseledets, Felix Schindler
Date: 2022-04-25 00:00:00
Tasks: BIG-bench Machine Learning

Reactive flows are important part of numerous technical and environmental processes. Often monitoring the flow and species concentrations within the domain is not possible or is expensive, in contrast, outlet concentration is straightforward to measure. In connection with reactive flows in porous media, the term breakthrough curve is used to denote the time dependency of the outlet concentration with prescribed conditions at the inlet. In this work we apply several machine learning methods to predict breakthrough curves from the given set of parameters. In our case the parameters are the Damk\"ohler and Peclet numbers. We perform a thorough analysis for the one-dimensional case and also provide the results for the three-dimensional case.
Predicting Real-time Scientific Experiments Using Transformer models and Reinforcement Learning
Authors: Juan Manuel Parrilla-Gutierrez
Date: 2022-04-25 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL), scientific discovery, Scientific Results Extraction

Life and physical sciences have always been quick to adopt the latest advances in machine learning to accelerate scientific discovery. Examples of this are cell segmentation or cancer detection. Nevertheless, these exceptional results are based on mining previously created datasets to discover patterns or trends. Recent advances in AI have been demonstrated in real-time scenarios like self-driving cars or playing video games. However, these new techniques have not seen widespread adoption in life or physical sciences because experimentation can be slow. To tackle this limitation, this work aims to adapt generative learning algorithms to model scientific experiments and accelerate their discovery using in-silico simulations. We particularly focused on real-time experiments, aiming to model how they react to user inputs. To achieve this, here we present an encoder-decoder architecture based on the Transformer model to simulate real-time scientific experimentation, predict its future behaviour and manipulate it on a step-by-step basis. As a proof of concept, this architecture was trained to map a set of mechanical inputs to the oscillations generated by a chemical reaction. The model was paired with a Reinforcement Learning controller to show how the simulated chemistry can be manipulated in real-time towards user-defined behaviours. Our results demonstrate how generative learning can model real-time scientific experimentation to track how it changes through time as the user manipulates it, and how the trained models can be paired with optimisation algorithms to discover new phenomena beyond the physical limitations of lab experimentation. This work paves the way towards building surrogate systems where physical experimentation interacts with machine learning on a step-by-step basis.