OpenRaiser/Intern-Atlas huggingface dataset:
DeOccNet: Learning to See Through Foreground Occlusions in Light Fields
Venue: arXiv cs.CV 2019
Year: 2019

Background objects occluded in some views of a light field (LF) camera can be seen by other views. Consequently, occluded surfaces are possible to be reconstructed from LF images. In this paper, we handle the LF de-occlusion (LF-DeOcc) problem using a deep encoder-decoder network (namely, DeOccNet). In our method, sub-aperture images (SAIs) are first given to the encoder to incorporate both spatial and angular information. The encoded representations are then used by the decoder to render an occlusionfree center-view SAI. To the best of our knowledge, DeOccNet is the first deep learning-based LF-DeOcc method. To handle the insufficiency of training data, we propose an LF synthesis approach to embed selected occlusion masks into existing LF images. Besides, several synthetic and realworld LFs are developed for performance evaluation. Experimental results show that, after training on the generated data, our DeOccNet can effectively remove foreground occlusions and achieves superior performance as compared to other state-of-the-art methods. Source codes are available at: https://github.com/YingqianWang/DeOccNet.
Continuous Domains in Formal Concept Analysis
Venue: arXiv cs.LO 2019
Year: 2019

Formal Concept Analysis has proven to be an effective method of restructuring complete lattices and various algebraic domains. In this paper, the notions of attribute continuous formal context and continuous formal concept are introduced by considering a selection F of fnite subsets of attributes. Our decision of a selection F relies on a kind of generalized interior operators. It is shown that the set of continuous formal concepts forms a continuous domain, and every continuous domain can be obtained in this way. Moreover, an notion of F-morphism is also identified to produce a category equivalent to that of continuous domains with Scott-continuous functions. This paper also consider the representations of various subclasses of continuous domains such as algebraic domains, bounded complete domains and stably continuous semilattices. These results explore the fundamental idea of domain theory in Formal Concept Analysis from a categorical viewpoint.
MDFN: Multi-Scale Deep Feature Learning Network for Object Detection
Venue: arXiv cs.CV 2019
Year: 2019

This paper proposes an innovative object detector by leveraging deep features learned in high-level layers. Compared with features produced in earlier layers, the deep features are better at expressing semantic and contextual information. The proposed deep feature learning scheme shifts the focus from concrete features with details to abstract ones with semantic information. It considers not only individual objects and local contexts but also their relationships by building a multi-scale deep feature learning network (MDFN). MDFN efficiently detects the objects by introducing information square and cubic inception modules into the high-level layers, which employs parameter-sharing to enhance the computational efficiency. MDFN provides a multi-scale object detector by integrating multi-box, multi-scale and multi-level technologies. Although MDFN employs a simple framework with a relatively small base network (VGG-16), it achieves better or competitive detection results than those with a macro hierarchical structure that is either very deep or very wide for stronger ability of feature extraction. The proposed technique is evaluated extensively on KITTI, PASCAL VOC, and COCO datasets, which achieves the best results on KITTI and leading performance on PASCAL VOC and COCO. This study reveals that deep features provide prominent semantic information and a variety of contextual contents, which contribute to its superior performance in detecting small or occluded objects. In addition, the MDFN model is computationally efficient, making a good trade-off between the accuracy and speed.
No-Trick (Treat) Kernel Adaptive Filtering using Deterministic Features
Venue: arXiv cs.LG 2019
Year: 2019

Kernel methods form a powerful, versatile, and theoretically-grounded unifying framework to solve nonlinear problems in signal processing and machine learning. The standard approach relies on the kernel trick to perform pairwise evaluations of a kernel function, which leads to scalability issues for large datasets due to its linear and superlinear growth with respect to the training data. A popular approach to tackle this problem, known as random Fourier features (RFFs), samples from a distribution to obtain the data-independent basis of a higher finite-dimensional feature space, where its dot product approximates the kernel function. Recently, deterministic, rather than random construction has been shown to outperform RFFs, by approximating the kernel in the frequency domain using Gaussian quadrature. In this paper, we view the dot product of these explicit mappings not as an approximation, but as an equivalent positive-definite kernel that induces a new finite-dimensional reproducing kernel Hilbert space (RKHS). This opens the door to no-trick (NT) online kernel adaptive filtering (KAF) that is scalable and robust. Random features are prone to large variances in performance, especially for smaller dimensions. Here, we focus on deterministic feature-map construction based on polynomial-exact solutions and show their superiority over random constructions. Without loss of generality, we apply this approach to classical adaptive filtering algorithms and validate the methodology to show that deterministic features are faster to generate and outperform state-of-the-art kernel methods based on random Fourier features.
RGB Point Cloud Manipulation with Triangular Structures for Artistic Image Recoloring
Venue: arXiv cs.GR 2019
Year: 2019

Usual approaches for image recoloring, such as local filtering by transfer functions and global histogram remapping, lack of accurate control or miss small groups of important pixels. In this paper, we introduce a triangle-based structuring of the colors of an image in the RGB space. We present an analysis of image colors in the RGB space showing the theoretical motivation of our triangular abstraction. We illustrate the usefulness of our structure to recolor images.
Client-side Vulnerabilities in Commercial VPNs
Venue: arXiv cs.CR 2019
Year: 2019

Internet users increasingly rely on commercial virtual private network (VPN) services to protect their security and privacy. The VPN services route the client's traffic over an encrypted tunnel to a VPN gateway in the cloud. Thus, they hide the client's real IP address from online services, and they also shield the user's connections from perceived threats in the access networks. In this paper, we study the security of such commercial VPN services. The focus is on how the client applications set up VPN tunnels, and how the service providers instruct users to configure generic client software. We analyze common VPN protocols and implementations on Windows, macOS and Ubuntu. We find that the VPN clients have various configuration flaws, which an attacker can exploit to strip off traffic encryption or to bypass authentication of the VPN gateway. In some cases, the attacker can also steal the VPN user's username and password. We suggest ways to mitigate each of the discovered vulnerabilities.
Reconstructing Multi-echo Magnetic Resonance Images via Structured Deep Dictionary Learning
Venue: arXiv cs.LG 2019
Year: 2019

Multi-echo magnetic resonance (MR) images are acquired by changing the echo times (for T2 weighted) or relaxation times (for T1 weighted) of scans. The resulting (multi-echo) images are usually used for quantitative MR imaging. Acquiring MR images is a slow process and acquiring multi scans of the same cross section for multi-echo imaging is even slower. In order to accelerate the scan, compressed sensing (CS) based techniques have been advocating partial K-space (Fourier domain) scans; the resulting images are reconstructed via structured CS algorithms. In recent times, it has been shown that instead of using off-the-shelf CS, better results can be obtained by adaptive reconstruction algorithms based on structured dictionary learning. In this work, we show that the reconstruction results can be further improved by using structured deep dictionaries. Experimental results on real datasets show that by using our proposed technique the scan-time can be cut by half compared to the state-of-the-art.
Fraud detection in telephone conversations for financial services using linguistic features
Venue: arXiv cs.CL 2019
Year: 2019

Detecting the elements of deception in a conversation is one of the most challenging problems for the AI community. It becomes even more difficult to design a transparent system, which is fully explainable and satisfies the need for financial and legal services to be deployed. This paper presents an approach for fraud detection in transcribed telephone conversations using linguistic features. The proposed approach exploits the syntactic and semantic information of the transcription to extract both the linguistic markers and the sentiment of the customer's response. We demonstrate the results on real-world financial services data using simple, robust and explainable classifiers such as Naive Bayes, Decision Tree, Nearest Neighbours, and Support Vector Machines.
A Novel Topology for End-to-end Temporal Classification and Segmentation with Recurrent Neural Network
Venue: arXiv cs.CL 2019
Year: 2019

Connectionist temporal classification (CTC) has matured as an alignment free to sequence transduction and shows competitive for end-to-end speech recognition. In the CTC topology, the blank symbol occupies more than half of the state trellis, which results the spike phenomenon of the non-blank symbols. For classification task, the spikes work quite well, but as to the segmentation task it does not provide boundaries information. In this paper, a novel topology is introduced to combine the temporal classification and segmentation ability in one framework.
edBB: Biometrics and Behavior for Assessing Remote Education
Venue: arXiv cs.HC 2019
Year: 2019

We present a platform for student monitoring in remote education consisting of a collection of sensors and software that capture biometric and behavioral data. We define a collection of tasks to acquire behavioral data that can be useful for facing the existing challenges in automatic student monitoring during remote evaluation. Additionally, we release an initial database including data from 20 different users completing these tasks with a set of basic sensors: webcam, microphone, mouse, and keyboard; and also from more advanced sensors: NIR camera, smartwatch, additional RGB cameras, and an EEG band. Information from the computer (e.g. system logs, MAC, IP, or web browsing history) is also stored. During each acquisition session each user completed three different types of tasks generating data of different nature: mouse and keystroke dynamics, face data, and audio data among others. The tasks have been designed with two main goals in mind: i) analyse the capacity of such biometric and behavioral data for detecting anomalies during remote evaluation, and ii) study the capability of these data, i.e. EEG, ECG, or NIR video, for estimating other information about the users such as their attention level, the presence of stress, or their pulse rate.
Ride the Lightning: The Game Theory of Payment Channels
Venue: arXiv cs.GT 2019
Year: 2019

Payment channels were introduced to solve various eminent cryptocurrency scalability issues. Multiple payment channels build a network on top of a blockchain, the so-called layer 2. In this work, we analyze payment networks through the lens of network creation games. We identify betweenness and closeness centrality as central concepts regarding payment networks. We study the topologies that emerge when players act selfishly and determine the parameter space in which they constitute a Nash equilibrium. Moreover, we determine the social optima depending on the correlation of betweenness and closeness centrality. When possible, we bound the price of anarchy. We also briefly discuss the price of stability.
Modelling curvature of a bent paper leaf
Venue: arXiv cs.GR 2019
Year: 2019

In this article, we briefly describe various tools and approaches that algebraic geometry has to offer to straighten bent objects. Throughout this article we will consider a specific example of a bent or curved piece of paper which in our case acts very much like an elastica curve. We conclude this article with a suggestion to algebraic geometry as a viable and fast performance alternative of neural networks in vision and machine learning. The purpose of this article is not to build a full blown framework but to show possibility of using algebraic geometry as an alternative to neural networks for recognizing or extracting features on manifolds.
Learning Domain Adaptive Features with Unlabeled Domain Bridges
Venue: arXiv cs.CV 2019
Year: 2019

Conventional cross-domain image-to-image translation or unsupervised domain adaptation methods assume that the source domain and target domain are closely related. This neglects a practical scenario where the domain discrepancy between the source and target is excessively large. In this paper, we propose a novel approach to learn domain adaptive features between the largely-gapped source and target domains with unlabeled domain bridges. Firstly, we introduce the framework of Cycle-consistency Flow Generative Adversarial Networks (CFGAN) that utilizes domain bridges to perform image-to-image translation between two distantly distributed domains. Secondly, we propose the Prototypical Adversarial Domain Adaptation (PADA) model which utilizes unlabeled bridge domains to align feature distribution between source and target with a large discrepancy. Extensive quantitative and qualitative experiments are conducted to demonstrate the effectiveness of our proposed models.
Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset
Venue: arXiv cs.CV 2019
Year: 2019

Human emotions unfold over time, and more affective computing research has to prioritize capturing this crucial component of real-world affect. Modeling dynamic emotional stimuli requires solving the twin challenges of time-series modeling and of collecting high-quality time-series datasets. We begin by assessing the state-of-the-art in time-series emotion recognition, and we review contemporary time-series approaches in affective computing, including discriminative and generative models. We then introduce the first version of the Stanford Emotional Narratives Dataset (SENDv1): a set of rich, multimodal videos of self-paced, unscripted emotional narratives, annotated for emotional valence over time. The complex narratives and naturalistic expressions in this dataset provide a challenging test for contemporary time-series emotion recognition models. We demonstrate several baseline and state-of-the-art modeling approaches on the SEND, including a Long Short-Term Memory model and a multimodal Variational Recurrent Neural Network, which perform comparably to the human-benchmark. We end by discussing the implications for future research in time-series affective computing.
Line-based Camera Pose Estimation in Point Cloud of Structured Environments
Venue: arXiv cs.CV 2019
Year: 2019

Accurate registration of 2D imagery with point clouds is a key technology for image-LiDAR point cloud fusion, camera to laser scanner calibration and camera localization. Despite continuous improvements, automatic registration of 2D and 3D data without using additional textured information still faces great challenges. In this paper, we propose a new 2D-3D registration method to estimate 2D-3D line feature correspondences and the camera pose in untextured point clouds of structured environments. Specifically, we first use geometric constraints between vanishing points and 3D parallel lines to compute all feasible camera rotations. Then, we utilize a hypothesis testing strategy to estimate the 2D-3D line correspondences and the translation vector. By checking the consistency with computed correspondences, the best rotation matrix can be found. Finally, the camera pose is further refined using non-linear optimization with all the 2D-3D line correspondences. The experimental results demonstrate the effectiveness of the proposed method on the synthetic and real dataset (outdoors and indoors) with repeated structures and rapid depth changes.
Google street view and deep learning: a new ground truthing approach for crop mapping
Venue: arXiv cs.CV 2019
Year: 2019

Ground referencing is essential for supervised crop mapping. However, conventional ground truthing involves extensive field surveys and post processing, which is costly in terms of time and labor. In this study, we applied a convolutional neural network (CNN) model to explore the efficacy of automatic ground truthing via Google street view (GSV) images in two distinct farming regions: central Illinois and southern California. We demonstrated the feasibility and reliability of the new ground referencing technique further by performing pixel-based crop mapping with vegetation indices as the model input. The results were evaluated using the United States Department of Agriculture (USDA) crop data layer (CDL) products. From 8,514 GSV images, the CNN model screened out 2,645 target crop images. These images were well classified into crop types, including alfalfa, almond, corn, cotton, grape, soybean, and pistachio. The overall GSV image classification accuracy reached 93% in California and 97% in Illinois. We then shifted the image geographic coordinates using fixed empirical coefficients to produce 8,173 crop reference points including 1,764 in Illinois and 6,409 in California. Evaluation of these new reference points with CDL products showed satisfactory coherence, with 94 to 97% agreement. CNN-based mapping also captured the general pattern of crop type distributions. The overall differences between CDL products and our mapping results were 4% in California and 5% in Illinois. Thus, using these deep learning and GSV image techniques, we have provided an efficient and cost-effective alternative method for ground referencing and crop mapping.
Detecting Hardly Visible Roads in Low-Resolution Satellite Time Series Data
Venue: arXiv cs.CV 2019
Year: 2019

Massive amounts of satellite data have been gathered over time, holding the potential to unveil a spatiotemporal chronicle of the surface of Earth. These data allow scientists to investigate various important issues, such as land use changes, on a global scale. However, not all land-use phenomena are equally visible on satellite imagery. In particular, the creation of an inventory of the planet's road infrastructure remains a challenge, despite being crucial to analyze urbanization patterns and their impact. Towards this end, this work advances data-driven approaches for the automatic identification of roads based on open satellite data. Given the typical resolutions of these historical satellite data, we observe that there is inherent variation in the visibility of different road types. Based on this observation, we propose two deep learning frameworks that extend state-of-the-art deep learning methods by formalizing road detection as an ordinal classification task. In contrast to related schemes, one of the two models also resorts to satellite time series data that are potentially affected by missing data and cloud occlusion. Taking these time series data into account eliminates the need to manually curate datasets of high-quality image tiles, substantially simplifying the application of such models on a global scale. We evaluate our approaches on a dataset that is based on Sentinel~2 satellite imagery and OpenStreetMap vector data. Our results indicate that the proposed models can successfully identify large and medium-sized roads. We also discuss opportunities and challenges related to the detection of roads and other infrastructure on a global scale.
cFineGAN: Unsupervised multi-conditional fine-grained image generation
Venue: arXiv cs.CV 2019
Year: 2019

We propose an unsupervised multi-conditional image generation pipeline: cFineGAN, that can generate an image conditioned on two input images such that the generated image preserves the texture of one and the shape of the other input. To achieve this goal, we extend upon the recently proposed work of FineGAN \citep{singh2018finegan} and make use of standard as well as shape-biased pre-trained ImageNet models. We demonstrate both qualitatively as well as quantitatively the benefit of using the shape-biased network. We present our image generation result across three benchmark datasets- CUB-200-2011, Stanford Dogs and UT Zappos50k.
Deep Adaptive Wavelet Network
Venue: arXiv cs.CV 2019
Year: 2019

Even though convolutional neural networks have become the method of choice in many fields of computer vision, they still lack interpretability and are usually designed manually in a cumbersome trial-and-error process. This paper aims at overcoming those limitations by proposing a deep neural network, which is designed in a systematic fashion and is interpretable, by integrating multiresolution analysis at the core of the deep neural network design. By using the lifting scheme, it is possible to generate a wavelet representation and design a network capable of learning wavelet coefficients in an end-to-end form. Compared to state-of-the-art architectures, the proposed model requires less hyper-parameter tuning and achieves competitive accuracy in image classification tasks
Form + Function: Optimizing Aesthetic Product Design via Adaptive, Geometrized Preference Elicitation
Venue: arXiv cs.HC 2019
Year: 2019

Visual design is critical to product success, and the subject of intensive marketing research effort. Yet visual elements, due to their holistic and interactive nature, do not lend themselves well to optimization using extant decompositional methods for preference elicitation. Here we present a systematic methodology to incorporate interactive, 3D-rendered product configurations into a conjoint-like framework. The method relies on rapid, scalable machine learning algorithms to adaptively update product designs along with standard information-oriented product attributes. At its heart is a parametric account of a product's geometry, along with a novel, adaptive "bi-level" query task that can estimate individuals' visual design form preferences and their trade-offs against such traditional elements as price and product features. We illustrate the method's performance through extensive simulations and robustness checks, a formal proof of the bi-level query methodology's domain of superiority, and a field test for the design of a mid-priced sedan, using real-time 3D rendering for an online panel. Results indicate not only substantially enhanced predictive accuracy, but two quantities beyond the reach of standard conjoint methods: trade-offs between form and function overall, and willingness-to-pay for specific design elements. Moreover -- and most critically for applications -- the method provides "optimal" visual designs for both individuals and model-derived or analyst-supplied consumer groupings, as well as their sensitivities to form and functional elements.