Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Deep Correspondence through Prior and Posterior Feature Constancy

Stereo matching algorithms usually consist of four steps, including matching cost calculation, matching cost aggregation, disparity calculation, and disparity refinement. Existing CNN-based methods only adopt CNN to solve parts of the four steps, or use different networks to deal with different steps, making them difficult to obtain the overall optimal solution. In this paper, we propose a network architecture to incorporate all steps of stereo matching. The network consists of three parts. The first part calculates the multi-scale shared features. The second part performs matching cost calculation, matching cost aggregation and disparity calculation to estimate the initial disparity using shared features. The initial disparity and the shared features are used to calculate the prior and posterior feature constancy. The initial disparity, the prior and posterior feature constancy are then fed to a sub-network to refine the initial disparity through a Bayesian inference process. The proposed method has been evaluated on the Scene Flow and KITTI datasets. It achieves the state-of-the-art performance on the KITTI 2012 and KITTI 2015 benchmarks while maintaining a very fast running time.
i-HUMO: An Interactive Human and Machine Cooperation Framework for Entity Resolution with Quality Guarantees

Even though many approaches have been proposed for entity resolution (ER), it remains very challenging to find one with quality guarantees. To this end, we propose an interactive HUman and Machine cOoperation framework for ER, denoted by i-HUMO. Similar to the existing HUMO framework, i-HUMO enforces both precision and recall levels by dividing an ER workload between the human and the machine. It essentially makes the machine label easy instances while assigning more challenging instances to the human. However, i-HUMO is a major improvement over HUMO in that it is interactive: its process of human workload selection is optimized based on real-time risk analysis on human-labeled results as well as pre-specified machine metrics. In this paper, we first introduce the i-HUMO framework and then present the risk analysis technique to prioritize the instances for manual labeling. Finally, we empirically evaluate i-HUMO's performance on real data. Our extensive experiments show that i-HUMO is effective in enforcing quality guarantees, and compared with the state-of-the-art alternatives, it can achieve better quality control with reduced human cost.
An automatic knowledge graph construction system for K-12 education

Motivated by the pressing need of educational applications with knowledge graph, we develop a system, called K12EduKG, to automatically construct knowledge graphs for K-12 educational subjects. Leveraging on heterogeneous domain-specific educational data, K12EduKG extracts educational concepts and identifies implicit relations with high educational significance. More specifically, it adopts named entity recognition (NER) techniques on educational data like curriculum standards to extract educational concepts, and employs data mining techniques to identify the cognitive prerequisite relations between educational concepts. In this paper, we present details of K12EduKG and demonstrate it with a knowledge graph constructed for the subject of mathematics.
Learning Shape-from-Shading for Deformable Surfaces

Recent years have seen the development of mature solutions for reconstructing deformable surfaces from a single image, provided that they are relatively well-textured. By contrast, recovering the 3D shape of texture-less surfaces remains an open problem, and essentially relates to Shape-from-Shading. In this paper, we introduce a data-driven approach to this problem. We introduce a general framework that can predict diverse 3D representations, such as meshes, normals, and depth maps. Our experiments show that, in contrast with the well-textured scenario, meshes a ill-suited to handle texture-less 3D reconstruction. Furthermore, we demonstrate that our approach generalizes well to unseen objects, and that it yields higher-quality reconstructions than a state-of-the-art SfS technique, particularly in terms of normal estimates. Our reconstructions accurately model the fine details of the surfaces, such as the creases of a T-Shirt worn by a person.
Vehicle Three-Dimensional Pose and Shape Estimation from Multiple Monocular Vision

This paper proposes an accurate approach to estimate vehicles' 3D pose and shape from multi-view monocular images with a small overlap. This approach utilizes a state-of-the-art convolutional neural network (CNN) to detect vehicles' semantic keypoint in images and then introduces a Cross Projection Optimization (CPO) method to estimate the 3D pose accurately. During the iterative CPO process, it implements a new vehicle shape adjustment method named Hierarchical Wireframe Constraint (HWC). The approach is tested under both simulated and real-world scenes for performance verification. It's shown that this approach outperforms other existing monocular and stereo visual methods for vehicles' 3D pose and shape estimation. This approach provides new and robust solutions for accurate visual vehicle localization and it can be applied to the massive surveillance camera networks for intelligent transportation applications such as automatic driving assistance.
H-CNNs: Convolutional Neural Networks for Riemannian Homogeneous Spaces

Convolutional neural networks are ubiquitous in Machine Learning applications for solving a variety of problems. They however can not be used as is when data naturally reside on commonly encountered manifolds such as the sphere, the special orthogonal group, the Grassmanian, the manifold of symmetric positive definite matrices and others. Most recently, generalization of CNNs to data residing on a sphere has been reported by some research groups, which go by several names but we will refer to them as spherical CNNs (SCNNs). The key property of SCNNs distinct from the standard CNNs is that they exhibit the rotational equivariance property. In this paper, we theoretically generalize the SCNNs to Riemannian homogeneous manifolds, that include many commonly encountered manifolds including the aforementioned example manifolds. Proof of concept experiments involving synthetic data generated on the manifold of $(3 \times 3)$ symmetric positive definite matrices and the product manifold of $\mathbf{R}^+ \times \mathbf{S}^2$ respectively, are presented. These manifolds are commonly encountered in diffusion magnetic resonance imaging, a non-invasive medical imaging modality.
Direct manipulation of blendshapes using a sketch-based interface

We introduce a method that localizes the direct manipulation of blendshape models for facial animation with a customized sketch-based interface. Direct manipulation methods address the problem of cumbersome weight editing process using the traditional tools with a practical "pin-and-drag" operation directly on the 3D facial models. However, most of the direct manipulation methods have a global deformation impact, which lead to unintuitive and unexpected results. To this end, we propose a new way to localize the theory of direct manipulation method, using geodesic circles for confining the edits to the local geometry. Inspired by artists' brush painting on canvas, we additionally introduce a sketch-based interface as an application that provides direct manipulation and produces expressive facial poses efficiently and intuitively. Our method allows the artists to simply sketch directly onto the 3D facial model and automatically produces the expeditious manipulation until the desired facial pose is obtained. We show that localized blendshape direct manipulation has the potential to reduce the time-consuming blendshape editing process to an easy freehand stroke drawing.
Enhancing Evolutionary Optimization in Uncertain Environments by Allocating Evaluations via Multi-armed Bandit Algorithms

Optimization problems with uncertain fitness functions are common in the real world, and present unique challenges for evolutionary optimization approaches. Existing issues include excessively expensive evaluation, lack of solution reliability, and incapability in maintaining high overall fitness during optimization. Using conversion rate optimization as an example, this paper proposes a series of new techniques for addressing these issues. The main innovation is to augment evolutionary algorithms by allocating evaluation budget through multi-armed bandit algorithms. Experimental results demonstrate that multi-armed bandit algorithms can be used to allocate evaluations efficiently, select the winning solution reliably and increase overall fitness during exploration. The proposed methods can be generalized to any optimization problems with noisy fitness functions.
Fault and Intrusion Tolerance on the Basis of Virtual Machines

Fault and intrusion tolerance is an important paradigm for building distributed systems that work in spite of accidental and malicious faults. This paper discusses how to harness virtualization technology for building such dependable systems. We show that virtualization promotes a hybrid fault model that allows tolerating malicious intrusions in application domains with little overhead. The proposed architecture features mechanisms for supporting heterogeneity of the replicas. A hypervisor-based replication controller achieves perpetual operation through periodic proactive recovery of the replicas. Re-mapping of state storage between virtual machines speeds up the state transfer of a stateful replicated service. Our VM-FIT prototype implements the core functionality of such a virtualization-based replication architecture. We present some performance measurements and close with a discussion of future research directions.
Fast Semantic Segmentation on Video Using Motion Vector-Based Feature Interpolation

Models optimized for accuracy on challenging, dense prediction tasks such as semantic segmentation entail significant inference costs, and are prohibitively slow to run on each frame in a video. Since nearby video frames are spatially similar, however, there is substantial opportunity to reuse computation. Existing work has explored basic feature reuse and feature warping based on optical flow, but has encountered limits to the speedup attainable with these techniques. In this paper, we present a new, two part approach to accelerating inference on video. Firstly, we propose a fast feature propagation scheme that utilizes the block motion vector maps present in compressed video to cheaply propagate features from frame to frame. Secondly, we develop a novel feature estimation scheme, termed feature interpolation, that fuses features propagated from enclosing keyframes to render accurate feature estimates, even at sparse keyframe frequencies. We evaluate our system on the Cityscapes dataset, comparing to both a frame-by-frame baseline and related work. We find that we are able to substantially accelerate segmentation on video, achieving almost twice the average inference speed as prior work at any target accuracy level.
Dynamic annotations on an interactive web-based 360° video player

The use of 360° videos has been increasing steadily in the 2010s, as content creators and users search for more immersive experiences. The freedom to choose where to look at during the video may hinder the overall experience instead of enhancing it, as there is no guarantee that the user will focus on relevant sections of the scene. Visual annotations superimposed on the video, such as text boxes or arrow icons, can help guide the user through the narrative of the video while maintaining freedom of movement. This paper presents a web-based immersive visualizer for 360° videos that contain dynamic media annotations, rendered in real-time. A set of annotations was created with the purpose of providing information or guiding the user to points of interest. The visualizer can be used with a computer, using a keyboard and mouse or HTC Vive, and in mobile devices with Cardboard VR headsets, to experience the video in virtual reality, which is made possible with the WebVR API. The visualizer was evaluated through usability tests, to analyze the impact of different annotation techniques on the users' experience. The obtained results demonstrate that annotations can assist in guiding the user during the video, and a careful design is imperative so that they are not intrusive and distracting for the viewers.
Decision functions from supervised machine learning algorithms as collective variables for accelerating molecular simulations

Selection of appropriate collective variables for enhancing molecular simulations remains an unsolved problem in computational biophysics. In particular, picking initial collective variables (CVs) is particularly challenging in higher dimensions. Which atomic coordinates or transforms there of from a list of thousands should one pick for enhanced sampling runs? How does a modeler even begin to pick starting coordinates for investigation? This remains true even in the case of simple two state systems and only increases in difficulty for multi-state systems. In this work, we attempt to solve the initial CV problem using a data-driven approach inspired by supervised machine learning literature. In particular, we show how the decision functions in supervised machine learning (SML) algorithms can be used as initial CVs for accelerated sampling. Using solvated alanine dipeptide and Chignolin mini-protein as our test cases, we illustrate how the distance to the Support Vector Machines decision hyperplane, the output probability estimates from Logistic Regression, and other classifiers may be used to reversibly sample slow structural transitions. We discuss the utility of other SML algorithms that might be useful for identifying CVs for accelerating molecular simulations.
Verification of Binarized Neural Networks

We study the problem of formal verification of Binarized Neural Networks (BNN), which have recently been proposed as a power-efficient alternative to more traditional learning networks. More precisely, given a trained BNN and a relation between possible inputs and outputs of this BNN, we develop verification procedures for establishing that the BNN indeed meets this specification for all possible inputs. For solving the verification problem of BNNs we build on well-known methods for hardware verification.The BNN verification problem is first encoded as a combinational miter. In a second step this miter is then transformed into a corresponding propositional satisfiability (SAT) problem. The main contributions of this paper are a number of essential optimizations for making this approach to BNN verification scalable. First, we provide a transformation on fully conntected BNNs for reducing the order of the number of bitwise operations in each layer of the BNN from quadratic to linear. Second, we are identifying redundant computations in a BNN based on {\em optimal factoring} techniques, and we provide transformations on BNNs for avoiding these multiple computations. We prove that the problem of optimal factoring is NP-hard, and we design efficient search procedures for generating approximate solutions of the optimal factoring problem. Third, we design a compositional verification procedure for analyzing each layer of a BNN separately, and for iteratively combining and refining local verification results. We experimentally demonstrate the scalability of our verification techniques to moderately-sized BNNs for embedded applications with thousands of neurons and inputs.
Towards Identifying Paid Open Source Developers - A Case Study with Mozilla Developers

Open source development contains contributions from both hired and volunteer software developers. Identification of this status is important when we consider the transferability of research results to the closed source software industry, as they include no volunteer developers. While many studies have taken the employment status of developers into account, this information is often gathered manually due to the lack of accurate automatic methods. In this paper, we present an initial step towards predicting paid and unpaid open source development using machine learning and compare our results with automatic techniques used in prior work. By relying on code source repository meta-data from Mozilla, and manually collected employment status, we built a dataset of the most active developers, both volunteer and hired by Mozilla. We define a set of metrics based on developers' usual commit time pattern and use different classification methods (logistic regression, classification tree, and random forest). The results show that our proposed method identify paid and unpaid commits with an AUC of 0.75 using random forest, which is higher than the AUC of 0.64 obtained with the best of the previously used automatic methods.
A Simply Study to Steganography on Social Networks

Steganography aims to conceal the very fact that the communication takes place, by embedding a message into a digit object such as image without introducing noticeable artifacts. A number of steganographic systems have been developed in past years, most of which, however, are confined to the laboratory conditions where the real-world use of steganography are rarely concerned. In this paper, we introduce an alternative perspective to steganography. A graph-theoretic model to steganography on social networks is presented to analyze real-world steganographic scenarios. In the graph, steganographic participants are corresponding to the vertices with meaningless unique identifiers. Each edge allows the two vertices to communicate with each other by any steganographic algorithm. Meanwhile, the edges are associated with weights to quantize the corresponding communication risk (or say cost). The optimization task is to minimize the overall risk, which is modeled as additive over the social network. We analyze different scenarios on a social network, and provide the suited solutions to the corresponding optimization tasks. We prove that a multiplicative probabilistic graph is equivalent to an additive weighted graph. From the viewpoint of an attacker, he may hope to detect suspicious communication channels, the data encoder(s) and the data decoder(s). We present limited detection analysis to steganographic communication on a network.
Automatic Document Image Binarization

Document image binarization is often a challenging task due to various forms of degradation. Although there exist several binarization techniques in literature, the binarized image is typically sensitive to control parameter settings of the employed technique. This paper presents an automatic document image binarization algorithm to segment the text from heavily degraded document images. The proposed technique uses a two band-pass filtering approach for background noise removal, and Bayesian optimization for automatic hyperparameter selection for optimal results. The effectiveness of the proposed binarization technique is empirically demonstrated on the Document Image Binarization Competition (DIBCO) and the Handwritten Document Image Binarization Competition (H-DIBCO) datasets.
LSTM stack-based Neural Multi-sequence Alignment TeCHnique (NeuMATCH)

The alignment of heterogeneous sequential data (video to text) is an important and challenging problem. Standard techniques for such alignment, including Dynamic Time Warping (DTW) and Conditional Random Fields (CRFs), suffer from inherent drawbacks. Mainly, the Markov assumption implies that, given the immediate past, future alignment decisions are independent of further history. The separation between similarity computation and alignment decision also prevents end-to-end training. In this paper, we propose an end-to-end neural architecture where alignment actions are implemented as moving data between stacks of Long Short-term Memory (LSTM) blocks. This flexible architecture supports a large variety of alignment tasks, including one-to-one, one-to-many, skipping unmatched elements, and (with extensions) non-monotonic alignment. Extensive experiments on synthetic and real datasets show that our algorithm outperforms state-of-the-art baselines.
SEAR: A Polynomial-Time Expected Constant-Factor Optimal Algorithmic Framework for Multi-Robot Path Planning

This work studies the labeled multi-robot path and motion planning problem in continuous domains, in the absence of static obstacles. Given $n$ robots which may be arbitrarily close to each other and assuming random start and goal configurations for the robots, we derived an $O(n^3)$, complete algorithm that produces solutions with constant-factor optimality guarantees on both makespan and distance optimality, in expectation. Furthermore, our algorithm only requires a small constant factor expansion of the initial and goal configuration footprints for solving the problem. In addition to strong theoretical guarantees, we present a thorough computational evaluation of the proposed solution. Beyond the baseline solution, adapting an effective (but non-polynomial time) robot routing subroutine, we also provide a highly efficient implementation that quickly computes near-optimal solutions. Hardware experiments on microMVP platform composed of non-holonomic robots confirms the practical applicability of our algorithmic pipeline.
Efficient Õ(n/∊) Spectral Sketches for the Laplacian and its Pseudoinverse

In this paper we consider the problem of efficiently computing ϵ-sketches for the Laplacian and its pseudoinverse. Given a Laplacian and an error tolerance ϵ, we seek to construct a function f such that for any vector x (chosen obliviously from f), with high probability (1−ϵ)x┬ Ax≤f(x)≤(1+ϵ)x┬ Ax where A is either the Laplacian or its pseudoinverse. Our goal is to construct such a sketch f efficiently and to store it in the least space possible. We provide nearly-linear time algorithms that, when given a Laplacian matrix [EQUATION] ∈ Rn×n and an error tolerance ϵ, produce O(n/ϵ)-size sketches of both [EQUATION] and its pseudoinverse. Our algorithms improve upon the previous best sketch size of O(n/ϵ1.6) for sketching the Laplacian form by [1] and O(n/ϵ2) for sketching the Laplacian pseudoinverse by [2]. Furthermore we show how to compute all-pairs effective resistances from our O(n/ϵ) size sketch in O(n2/ϵ) time. This improves upon the previous best running time of O(n2/ϵ2) by [3].
Full-Duplex Heterogeneous Networks with Decoupled User Association: Throughput Region and Scheduling

Full-duplex (FD) transmission in a point-to-point (P2P) link, wherein bidirectional traffic flows simultaneously share the same spectrum, has the capability of doubling the link throughput by completely removing self-interferences. However, the throughput gain of FD transmission in an interference-limited network is not as clear as a P2P link due to the interferences induced by complex FD and half-duplex (HD) transmission behaviors in the network. To thoroughly investigate the link throughputs of users and base stations (BSs) in a heterogeneous network (HetNet) with decoupled user association, a general approach is proposed to model a HetNet in which all BSs and users can perform HD or FD transmission depending on their traffic patterns. We first characterize the decoupled rate-optimal user association scheme and use it to define the downlink and uplink throughputs in the HetNet. The tight lower bounds on the link throughputs of the FD users and BSs are found in a neat form that characterizes general channel fading, user and BS intensities and imperfect self-interference cancellation. These bounds outline the throughput regions of the FD users that inspire proposing the opportunistic FD scheduling algorithms that maximize the sum throughput of each bidirectional traffic and stabilize each of the queues in the HetNet.