OpenRaiser/Intern-Atlas huggingface dataset:
Video Stitching for Linear Camera Arrays
Venue: arXiv cs.CV 2019
Year: 2019

Despite the long history of image and video stitching research, existing academic and commercial solutions still produce strong artifacts. In this work, we propose a wide-baseline video stitching algorithm for linear camera arrays that is temporally stable and tolerant to strong parallax. Our key insight is that stitching can be cast as a problem of learning a smooth spatial interpolation between the input videos. To solve this problem, inspired by pushbroom cameras, we introduce a fast pushbroom interpolation layer and propose a novel pushbroom stitching network, which learns a dense flow field to smoothly align the multiple input videos for spatial interpolation. Our approach outperforms the state-of-the-art by a significant margin, as we show with a user study, and has immediate applications in many areas such as virtual reality, immersive telepresence, autonomous driving, and video surveillance.
Disentangled Relational Representations for Explaining and Learning from Demonstration
Venue: arXiv cs.RO 2019
Year: 2019

Learning from demonstration is an effective method for human users to instruct desired robot behaviour. However, for most non-trivial tasks of practical interest, efficient learning from demonstration depends crucially on inductive bias in the chosen structure for rewards/costs and policies. We address the case where this inductive bias comes from an exchange with a human user. We propose a method in which a learning agent utilizes the information bottleneck layer of a high-parameter variational neural model, with auxiliary loss terms, in order to ground abstract concepts such as spatial relations. The concepts are referred to in natural language instructions and are manifested in the high-dimensional sensory input stream the agent receives from the world. We evaluate the properties of the latent space of the learned model in a photorealistic synthetic environment and particularly focus on examining its usability for downstream tasks. Additionally, through a series of controlled table-top manipulation experiments, we demonstrate that the learned manifold can be used to ground demonstrations as symbolic plans, which can then be executed on a PR2 robot.
Quantifying Cochlear Implant Users' Ability for Speaker Identification using CI Auditory Stimuli
Venue: arXiv cs.SD 2019
Year: 2019

Speaker recognition is a biometric modality that uses underlying speech information to determine the identity of the speaker. Speaker Identification (SID) under noisy conditions is one of the challenging topics in the field of speech processing, specifically when it comes to individuals with cochlear implants (CI). This study analyzes and quantifies the ability of CI-users to perform speaker identification based on direct electric auditory stimuli. CI users employ a limited number of frequency bands (8 to 22) and use electrodes to directly stimulate the Basilar Membrane/Cochlear in order to recognize the speech signal. The sparsity of electric stimulation within the CI frequency range is a prime reason for loss in human speech recognition, as well as SID performance. Therefore, it is assumed that CI-users might be unable to recognize and distinguish a speaker given dependent information such as formant frequencies, pitch etc. which are lost to un-simulated electrodes. To quantify this assumption, the input speech signal is processed using a CI Advanced Combined Encoder (ACE) signal processing strategy to construct the CI auditory electrodogram. The proposed study uses 50 speakers from each of three different databases for training the system using two different classifiers under quiet, and tested under both quiet and noisy conditions. The objective result shows that, the CI users can effectively identify a limited number of speakers. However, their performance decreases when more speakers are added in the system, as well as when noisy conditions are introduced. This information could therefore be used for improving CI-user signal processing techniques to improve human SID.
Image Captioning with Unseen Objects
Venue: arXiv cs.CV 2019
Year: 2019

Image caption generation is a long standing and challenging problem at the intersection of computer vision and natural language processing. A number of recently proposed approaches utilize a fully supervised object recognition model within the captioning approach. Such models, however, tend to generate sentences which only consist of objects predicted by the recognition models, excluding instances of the classes without labelled training examples. In this paper, we propose a new challenging scenario that targets the image captioning problem in a fully zero-shot learning setting, where the goal is to be able to generate captions of test images containing objects that are not seen during training. The proposed approach jointly uses a novel zero-shot object detection model and a template-based sentence generator. Our experiments show promising results on the COCO dataset.
Software-Enhanced Teaching and Visualization Capabilities of an Ultra-High-Resolution Video Wall
Venue: arXiv cs.GR 2019
Year: 2019

This paper presents a modular approach to enhance the capabilities and features of a visualization and teaching room using software. This approach was applied to a room with a large, high resolution (7680$\times$4320 pixels), tiled screen of 13 $\times$ 7.5 feet as its main display, and with a variety of audio and video inputs, connected over a network. Many of the techniques described are possible because of a software-enhanced setup, utilizing existing hardware and a collection of mostly open-source tools, allowing to perform collaborative, high-resolution visualizations as well as broadcasting and recording workshops and lectures. The software approach is flexible and allows one to add functionality without changing the hardware.
Adversarial Robustness Curves
Venue: arXiv cs.LG 2019
Year: 2019

The existence of adversarial examples has led to considerable uncertainty regarding the trust one can justifiably put in predictions produced by automated systems. This uncertainty has, in turn, lead to considerable research effort in understanding adversarial robustness. In this work, we take first steps towards separating robustness analysis from the choice of robustness threshold and norm. We propose robustness curves as a more general view of the robustness behavior of a model and investigate under which circumstances they can qualitatively depend on the chosen norm.
FairSight: Visual Analytics for Fairness in Decision Making
Venue: arXiv cs.HC 2019
Year: 2019

Data-driven decision making related to individuals has become increasingly pervasive, but the issue concerning the potential discrimination has been raised by recent studies. In response, researchers have made efforts to propose and implement fairness measures and algorithms, but those efforts have not been translated to the real-world practice of data-driven decision making. As such, there is still an urgent need to create a viable tool to facilitate fair decision making. We propose FairSight, a visual analytic system to address this need; it is designed to achieve different notions of fairness in ranking decisions through identifying the required actions -- understanding, measuring, diagnosing and mitigating biases -- that together lead to fairer decision making. Through a case study and user study, we demonstrate that the proposed visual analytic and diagnostic modules in the system are effective in understanding the fairness-aware decision pipeline and obtaining more fair outcomes.
Solving a Flowshop Scheduling Problem with Answer Set Programming: Exploiting the Problem to Reduce the Number of Combinations
Venue: arXiv cs.LO 2019
Year: 2019

Planning and scheduling have been a central theme of research in computer science. In particular, the simplicity of the theoretical approach of a no-wait flowshop scheduling problem does not allow to perceive the problem complexity at first sight. In this paper the applicability of the Answer Set Programming language is explored for the solution of the Automated Wet-etching scheduling problem in Semiconductor Manufacturing Systems. A method based in ranges is proposed in order to reduce the huge number of combinations.
On the Theoretical Gap of Channel Hopping Sequences with Maximum Rendezvous Diversity in the Multichannel Rendezvous Problem
Venue: arXiv cs.DC 2019
Year: 2019

In the literature, there are several well-known periodic channel hopping (CH) sequences that can achieve maximum rendezvous diversity in a cognitive radio network (CRN). For a CRN with $N$ channels, it is known that the period of such a CH sequence is at least $N^2$. The asymptotic approximation ratio, defined as the ratio of the period of a CH sequence to the lower bound $N^2$ when $N \to \infty$, is still 2.5 for the best known CH sequence in the literature. An open question in the multichannel rendezvous problem is whether it is possible to construct a periodic CH sequence that has the asymptotic approximation ratio 1. In this paper, we tighten the theoretical gap by proposing CH sequences, called IDEAL-CH, that have the asymptotic approximation ratio 2. For a weaker requirement that only needs the two users to rendezvous on one commonly available channel in a period, we propose channel hopping sequences, called ORTHO-CH, with period $(2p +1)p$, where $p$ is the smallest prime not less than $N$.
Automatic pre-grasps generation for unknown 3D objects
Venue: arXiv cs.RO 2019
Year: 2019

In this paper, the problem of automating the pre-grasps generation for novel 3d objects has been discussed. The objects represented as cloud of 3D points are split into parts and organized in a tree structure, where parts are approximated by simple box primitives. Applying grasping only on the individual object parts may miss a good grasp which involves a combination of parts. The problem has been addressed by traversing the decomposition tree and checking each node of the tree for possible pre-grasps against a set of conditions. Further, a face mask has been introduced to encode the free and blocked faces of the box primitives. Pre-grasps are generated only for the free faces. Finally, the proposed method implemented on a set twenty-four household objects and toys, where a grasp planner based on object slicing method has been used to compute the contact-level grasp plan.
An Improved Approximation Algorithm for TSP in the Half Integral Case
Venue: arXiv cs.DS 2019
Year: 2019

We design a $1.49993$-approximation algorithm for the metric traveling salesperson problem (TSP) for instances in which an optimal solution to the subtour linear programming relaxation is half-integral. These instances received significant attention over the last decade due to a conjecture of Schalekamp, Williamson and van Zuylen stating that half-integral LP solutions have the largest integrality gap over all fractional solutions. So, if the conjecture of Schalekamp et al. holds true, our result shows that the integrality gap of the subtour polytope is bounded away from $3/2$.
Pseudo-Labeling Curriculum for Unsupervised Domain Adaptation
Venue: arXiv cs.CV 2019
Year: 2019

To learn target discriminative representations, using pseudo-labels is a simple yet effective approach for unsupervised domain adaptation. However, the existence of false pseudo-labels, which may have a detrimental influence on learning target representations, remains a major challenge. To overcome this issue, we propose a pseudo-labeling curriculum based on a density-based clustering algorithm. Since samples with high density values are more likely to have correct pseudo-labels, we leverage these subsets to train our target network at the early stage, and utilize data subsets with low density values at the later stage. We can progressively improve the capability of our network to generate pseudo-labels, and thus these target samples with pseudo-labels are effective for training our model. Moreover, we present a clustering constraint to enhance the discriminative power of the learned target features. Our approach achieves state-of-the-art performance on three benchmarks: Office-31, imageCLEF-DA, and Office-Home.
Pyramid Real Image Denoising Network
Venue: arXiv cs.CV 2019
Year: 2019

While deep Convolutional Neural Networks (CNNs) have shown extraordinary capability of modelling specific noise and denoising, they still perform poorly on real-world noisy images. The main reason is that the real-world noise is more sophisticated and diverse. To tackle the issue of blind denoising, in this paper, we propose a novel pyramid real image denoising network (PRIDNet), which contains three stages. First, the noise estimation stage uses channel attention mechanism to recalibrate the channel importance of input noise. Second, at the multi-scale denoising stage, pyramid pooling is utilized to extract multi-scale features. Third, the stage of feature fusion adopts a kernel selecting operation to adaptively fuse multi-scale features. Experiments on two datasets of real noisy photographs demonstrate that our approach can achieve competitive performance in comparison with state-of-the-art denoisers in terms of both quantitative measure and visual perception quality. Code is available at https://github.com/491506870/PRIDNet.
JUCBNMT at WMT2018 News Translation Task: Character Based Neural Machine Translation of Finnish to English
Venue: arXiv cs.CL 2019
Year: 2019

In the current work, we present a description of the system submitted to WMT 2018 News Translation Shared task. The system was created to translate news text from Finnish to English. The system used a Character Based Neural Machine Translation model to accomplish the given task. The current paper documents the preprocessing steps, the description of the submitted system and the results produced using the same. Our system garnered a BLEU score of 12.9.
Physical Cue based Depth-Sensing by Color Coding with Deaberration Network
Venue: arXiv cs.CV 2019
Year: 2019

Color-coded aperture (CCA) methods can physically measure the depth of a scene given by physical cues from a single-shot image of a monocular camera. However, they are vulnerable to actual lens aberrations in real scenes because they assume an ideal lens for simplifying algorithms. In this paper, we propose physical cue-based deep learning for CCA photography. To address actual lens aberrations, we developed a deep deaberration network (DDN) that is additionally equipped with a self-attention mechanism of position and color channels to efficiently learn the lens aberration. Furthermore, a new Bayes L1 loss function based on Bayesian deep learning enables to handle the uncertainty of depth estimation more accurately. Quantitative and qualitative comparisons demonstrate that our method is superior to conventional methods including real outdoor scenes. Furthermore, compared to a long-baseline stereo camera, the proposed method provides an error-free depth map at close range, as there is no blind spot between the left and right cameras.
Sparse Regression via Range Counting
Venue: arXiv cs.DS 2019
Year: 2019

The sparse regression problem, also known as best subset selection problem, can be cast as follows: Given a set $S$ of $n$ points in $\mathbb{R}^d$, a point $y\in \mathbb{R}^d$, and an integer $2 \leq k \leq d$, find an affine combination of at most $k$ points of $S$ that is nearest to $y$. We describe a $O(n^{k-1} \log^{d-k+2} n)$-time randomized $(1+\varepsilon)$-approximation algorithm for this problem with \(d\) and \(\varepsilon\) constant. This is the first algorithm for this problem running in time $o(n^k)$. Its running time is similar to the query time of a data structure recently proposed by Har-Peled, Indyk, and Mahabadi (ICALP'18), while not requiring any preprocessing. Up to polylogarithmic factors, it matches a conditional lower bound relying on a conjecture about affine degeneracy testing. In the special case where $k = d = O(1)$, we also provide a simple $O_δ(n^{d-1+δ})$-time deterministic exact algorithm, for any \(δ> 0\). Finally, we show how to adapt the approximation algorithm for the sparse linear regression and sparse convex regression problems with the same running time, up to polylogarithmic factors.
Continual Learning via Online Leverage Score Sampling
Venue: arXiv cs.LG 2019
Year: 2019

In order to mimic the human ability of continual acquisition and transfer of knowledge across various tasks, a learning system needs the capability for continual learning, effectively utilizing the previously acquired skills. As such, the key challenge is to transfer and generalize the knowledge learned from one task to other tasks, avoiding forgetting and interference of previous knowledge and improving the overall performance. In this paper, within the continual learning paradigm, we introduce a method that effectively forgets the less useful data samples continuously and allows beneficial information to be kept for training of the subsequent tasks, in an online manner. The method uses statistical leverage score information to measure the importance of the data samples in every task and adopts frequent directions approach to enable a continual or life-long learning property. This effectively maintains a constant training size across all tasks. We first provide mathematical intuition for the method and then demonstrate its effectiveness in avoiding catastrophic forgetting and computational efficiency on continual learning of classification tasks when compared with the existing state-of-the-art techniques.
Evaluating Perceptual Bias During Geometric Scaling of Scatterplots
Venue: arXiv cs.HC 2019
Year: 2019

Scatterplots are frequently scaled to fit display areas in multi-view and multi-device data analysis environments. A common method used for scaling is to enlarge or shrink the entire scatterplot together with the inside points synchronously and proportionally. This process is called geometric scaling. However, geometric scaling of scatterplots may cause a perceptual bias, that is, the perceived and physical values of visual features may be dissociated with respect to geometric scaling. For example, if a scatterplot is projected from a laptop to a large projector screen, then observers may feel that the scatterplot shown on the projector has fewer points than that viewed on the laptop. This paper presents an evaluation study on the perceptual bias of visual features in scatterplots caused by geometric scaling. The study focuses on three fundamental visual features (i.e., numerosity, correlation, and cluster separation) and three hypotheses that are formulated on the basis of our experience. We carefully design three controlled experiments by using well-prepared synthetic data and recruit participants to complete the experiments on the basis of their subjective experience. With a detailed analysis of the experimental results, we obtain a set of instructive findings. First, geometric scaling causes a bias that has a linear relationship with the scale ratio. Second, no significant difference exists between the biases measured from normally and uniformly distributed scatterplots. Third, changing the point radius can correct the bias to a certain extent. These findings can be used to inspire the design decisions of scatterplots in various scenarios.
MIN: Co-Governing Multi-Identifier Network Architecture and its Prototype on Operator's Network
Venue: arXiv cs.NI 2019
Year: 2019

IP protocol is the core of TCP/IP network layer. However, since IP address and its Domain Name are allocated and managed by a single agency, there are risks of centralization. The semantic overload of IP address also reduces its scalability and mobility, which further hinders the security. This paper proposes a co-governing Multi-Identifier Network (MIN) architecture that constructs a network layer with parallel coexistence of multiple identifiers, including identity, content, geographic information, and IP address. On the management plane, we develop an efficient management system using consortium blockchain with voting consensus, so the network can simultaneously manage and support by hundreds or thousands of nodes with high throughput. On the data plane, we propose an algorithm merging hash table and prefix tree (HTP) for FIB, which avoids the false-negative error and can inter-translate different identifiers with tens of billions of entries. Further, we propose a scheme to transport IP packets using CCN as a tunnel for supporting progressive deployment. We deployed the prototype of MIN to the largest operators' network in Mainland China, Hongkong and Macao, and demonstrated that the network can register identifier under co-governing consensus algorithm, support VoD service very well.
Estimation of Tire-Road Friction for Road Vehicles: a Time Delay Neural Network Approach
Venue: arXiv cs.NE 2019
Year: 2019

The performance of vehicle active safety systems is dependent on the friction force arising from the contact of tires and the road surface. Therefore, an adequate knowledge of the tire-road friction coefficient is of great importance to achieve a good performance of different vehicle control systems. This paper deals with the tire-road friction coefficient estimation problem through the knowledge of lateral tire force. A time delay neural network (TDNN) is adopted for the proposed estimation design. The TDNN aims at detecting road friction coefficient under lateral force excitations avoiding the use of standard mathematical tire models, which may provide a more efficient method with robust results. Moreover, the approach is able to estimate the road friction at each wheel independently, instead of using lumped axle models simplifications. Simulations based on a realistic vehicle model are carried out on different road surfaces and driving maneuvers to verify the effectiveness of the proposed estimation method. The results are compared with a classical approach, a model-based method modeled as a nonlinear regression.
DIODE: A Dense Indoor and Outdoor DEpth Dataset
Venue: arXiv cs.CV 2019
Year: 2019

We introduce DIODE, a dataset that contains thousands of diverse high resolution color images with accurate, dense, long-range depth measurements. DIODE (Dense Indoor/Outdoor DEpth) is the first public dataset to include RGBD images of indoor and outdoor scenes obtained with one sensor suite. This is in contrast to existing datasets that focus on just one domain/scene type and employ different sensors, making generalization across domains difficult. The dataset is available for download at http://diode-dataset.org
Semantic Concept Spaces: Guided Topic Model Refinement using Word-Embedding Projections
Venue: arXiv cs.HC 2019
Year: 2019

We present a framework that allows users to incorporate the semantics of their domain knowledge for topic model refinement while remaining model-agnostic. Our approach enables users to (1) understand the semantic space of the model, (2) identify regions of potential conflicts and problems, and (3) readjust the semantic relation of concepts based on their understanding, directly influencing the topic modeling. These tasks are supported by an interactive visual analytics workspace that uses word-embedding projections to define concept regions which can then be refined. The user-refined concepts are independent of a particular document collection and can be transferred to related corpora. All user interactions within the concept space directly affect the semantic relations of the underlying vector space model, which, in turn, change the topic modeling. In addition to direct manipulation, our system guides the users' decision-making process through recommended interactions that point out potential improvements. This targeted refinement aims at minimizing the feedback required for an efficient human-in-the-loop process. We confirm the improvements achieved through our approach in two user studies that show topic model quality improvements through our visual knowledge externalization and learning process.