OpenRaiser/Intern-Atlas huggingface dataset:
Panoptic Image Annotation with a Collaborative Assistant
Venue: arXiv cs.CV 2020
Year: 2020

This paper aims to reduce the time to annotate images for panoptic segmentation, which requires annotating segmentation masks and class labels for all object instances and stuff regions. We formulate our approach as a collaborative process between an annotator and an automated assistant who take turns to jointly annotate an image using a predefined pool of segments. Actions performed by the annotator serve as a strong contextual signal. The assistant intelligently reacts to this signal by annotating other parts of the image on its own, which reduces the amount of work required by the annotator. We perform thorough experiments on the COCO panoptic dataset, both in simulation and with human annotators. These demonstrate that our approach is significantly faster than the recent machine-assisted interface of [4], and 2.4x to 5x faster than manual polygon drawing. Finally, we show on ADE20k that our method can be used to efficiently annotate new datasets, bootstrapping from a very small amount of annotated data.
Three-Dimensional Fourier Scattering Transform and Classification of Hyperspectral Images
Venue: arXiv cs.CV 2020
Year: 2020

Recent developments in machine learning and signal processing have resulted in many new techniques that are able to effectively capture the intrinsic yet complex properties of hyperspectral imagery. Tasks ranging from anomaly detection to classification can now be solved by taking advantage of very efficient algorithms which have their roots in representation theory and in computational approximation. Time-frequency methods are one example of such techniques. They provide means to analyze and extract the spectral content from data. On the other hand, hierarchical methods such as neural networks incorporate spatial information across scales and model multiple levels of dependencies between spectral features. Both of these approaches have recently been proven to provide significant advances in the spectral-spatial classification of hyperspectral imagery. The 3D Fourier scattering transform, which is introduced in this paper, is an amalgamation of time-frequency representations with neural network architectures. It leverages the benefits provided by the Short-Time Fourier Transform with the numerical efficiency of deep learning network structures. We test the proposed method on several standard hyperspectral datasets, and we present results that indicate that the 3D Fourier scattering transform is highly effective at representing spectral content when compared with other state-of-the-art spectral-spatial classification methods.
Exact and Consistent Interpretation of Piecewise Linear Models Hidden behind APIs: A Closed Form Solution
Venue: arXiv cs.LG 2020
Year: 2020

More and more AI services are provided through APIs on cloud where predictive models are hidden behind APIs. To build trust with users and reduce potential application risk, it is important to interpret how such predictive models hidden behind APIs make their decisions. The biggest challenge of interpreting such predictions is that no access to model parameters or training data is available. Existing works interpret the predictions of a model hidden behind an API by heuristically probing the response of the API with perturbed input instances. However, these methods do not provide any guarantee on the exactness and consistency of their interpretations. In this paper, we propose an elegant closed form solution named OpenAPI to compute exact and consistent interpretations for the family of Piecewise Linear Models (PLM), which includes many popular classification models. The major idea is to first construct a set of overdetermined linear equation systems with a small set of perturbed instances and the predictions made by the model on those instances. Then, we solve the equation systems to identify the decision features that are responsible for the prediction on an input instance. Our extensive experiments clearly demonstrate the exactness and consistency of our method.
Public Ledger for Sensitive Data
Venue: arXiv cs.CR 2020
Year: 2020

Satoshi Nakamoto's Blockchain allows to build publicly verifiable and almost immutable ledgers, but sometimes privacy has to be factored in. In this work an original protocol is presented that allows sensitive data to be stored on a ledger where its integrity may be publicly verified, but its privacy is preserved and owners can tightly manage the sharing of their information with efficient revocation.
Fine-grained complexity of the graph homomorphism problem for bounded-treewidth graphs
Venue: arXiv cs.CC 2020
Year: 2020

For graphs $G$ and $H$, a \emph{homomorphism} from $G$ to $H$ is an edge-preserving mapping from the vertex set of $G$ to the vertex set of $H$. For a fixed graph $H$, by \textsc{Hom($H$)} we denote the computational problem which asks whether a given graph $G$ admits a homomorphism to $H$. If $H$ is a complete graph with $k$ vertices, then \textsc{Hom($H$)} is equivalent to the $k$-\textsc{Coloring} problem, so graph homomorphisms can be seen as generalizations of colorings. It is known that \textsc{Hom($H$)} is polynomial-time solvable if $H$ is bipartite or has a vertex with a loop, and NP-complete otherwise [Hell and Nešetřil, JCTB 1990]. In this paper we are interested in the complexity of the problem, parameterized by the treewidth of the input graph $G$. If $G$ has $n$ vertices and is given along with its tree decomposition of width $\mathrm{tw}(G)$, then the problem can be solved in time $|V(H)|^{\mathrm{tw}(G)} \cdot n^{\mathcal{O}(1)}$, using a straightforward dynamic programming. We explore whether this bound can be improved. We show that if $H$ is a \emph{projective core}, then the existence of such a faster algorithm is unlikely: assuming the Strong Exponential Time Hypothesis (SETH), the \textsc{Hom($H$)} problem cannot be solved in time $(|V(H)|-ε)^{\mathrm{tw}(G)} \cdot n^{\mathcal{O}(1)}$, for any $ε> 0$. This result provides a full complexity characterization for a large class of graphs $H$, as almost all graphs are projective cores. We also notice that the naive algorithm can be improved for some graphs $H$, and show a complexity classification for all graphs $H$, assuming two conjectures from algebraic graph theory. In particular, there are no known graphs $H$ which are not covered by our result. In order to prove our results, we bring together some tools and techniques from algebra and from fine-grained complexity.
A robust direct modeling method for quadric B-rep models based on geometry-topology inconsistency tracking
Venue: arXiv cs.CG 2020
Year: 2020

Boundary representation (B-rep) model editing plays an essential role in computer-aided design and motivates the very recent direct modeling paradigm, which features intuitive push-pull manipulation of the model geometry. In mechanical design, a substantial part of B-rep models being used are quadric models (composed of linear and quadric surfaces). However, push-pulling such models is not trivial due to the possible smooth face-face connections in the models. The major issue is that, during push-pull moves, it is often desirable to preserve these connections for functional, manufacturing, or aesthetic reasons, but this could cause complex inconsistencies between the geometry and topology in the model and lead to robustness issues in updating the model. The challenge lies in effectiveness towards detecting the instants when geometry-topology inconsistencies occur during push-pull moves. This paper proposes a novel reverse detection method to solve the challenge and then, based on it, presents a robust method for push-pull direct modeling while preserving smooth connections. Case studies and comparisons have been conducted to demonstrate the effectiveness of the method.
ANAE: Learning Node Context Representation for Attributed Network Embedding
Venue: arXiv cs.SI 2020
Year: 2020

Attributed network embedding aims to learn low-dimensional node representations from both network structure and node attributes. Existing methods can be categorized into two groups: (1) the first group learns two separated node representations from network structure and node attribute respectively and concatenates them together; (2) the other group obtains node representations by translating node attributes into network structure or vice versa. However, both groups have their drawbacks. The first group neglects the correlation between network structure and node attributes, while the second group assumes strong dependence between these two types of information. In this paper, we address attributed network embedding from a novel perspective, i.e., learning node context representation for each node via modeling its attributed local subgraph. To achieve this goal, we propose a novel attributed network auto-encoder framework, namely ANAE. For a target node, ANAE first aggregates the attribute information from its attributed local subgraph, obtaining its low-dimensional representation. Next, ANAE diffuses the representation of the target node to nodes in its local subgraph to reconstruct their attributes. Such an encoder-decoder framework allows the learned representations to better preserve the context information manifested in both network structure and node attributes, thus having high capacity to learn good node representations for attributed network. Extensive experimental results on real-world datasets demonstrate that the proposed framework outperforms the state-of-the-art approaches at the tasks of link prediction and node classification.
A Cyclically-Trained Adversarial Network for Invariant Representation Learning
Venue: arXiv cs.CV 2020
Year: 2020

Recent studies show that deep neural networks are vulnerable to adversarial examples which can be generated via certain types of transformations. Being robust to a desired family of adversarial attacks is then equivalent to being invariant to a family of transformations. Learning invariant representations then naturally emerges as an important goal to achieve which we explore in this paper within specific application contexts. Specifically, we propose a cyclically-trained adversarial network to learn a mapping from image space to latent representation space and back such that the latent representation is invariant to a specified factor of variation (e.g., identity). The learned mapping assures that the synthesized image is not only realistic, but has the same values for unspecified factors (e.g., pose and illumination) as the original image and a desired value of the specified factor. Unlike disentangled representation learning, which requires two latent spaces, one for specified and another for unspecified factors, invariant representation learning needs only one such space. We encourage invariance to a specified factor by applying adversarial training using a variational autoencoder in the image space as opposed to the latent space. We strengthen this invariance by introducing a cyclic training process (forward and backward cycle). We also propose a new method to evaluate conditional generative networks. It compares how well different factors of variation can be predicted from the synthesized, as opposed to real, images. In quantitative terms, our approach attains state-of-the-art performance in experiments spanning three datasets with factors such as identity, pose, illumination or style. Our method produces sharp, high-quality synthetic images with little visible artefacts compared to previous approaches.
Predicting kills in Game of Thrones using network properties
Venue: arXiv cs.SI 2020
Year: 2020

TV series such as HBO's Game of Thrones have seen a high number of dedicated followers, mostly due to the dramatic murders of the most important characters. In our work, we try to predict killer and victim pairs using data about previous kills and additional metadata. We construct a network where two character nodes are linked if one killed the other and use a link prediction framework to evaluate different techniques for kill predictions. Lastly, we compute various network properties on a social network of characters and use them as features in conjunction with classic data mining techniques. Due to the small size of the dataset and the somewhat random kill distribution, we cannot predict much with standard indices alone, although using them in conjunction with additional rules based on degrees works surprisingly well. The features we compute on the social network help the classic machine learning approaches, but do not yield very accurate predictions. The best results overall are achieved using indices that use simple degree information, the best of which gives us the Area Under the ROC Curve of 0.875.
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
Venue: arXiv cs.GR 2020
Year: 2020

Immersive computer graphics systems strive to generate perceptually realistic user experiences. Current-generation virtual reality (VR) displays are successful in accurately rendering many perceptually important effects, including perspective, disparity, motion parallax, and other depth cues. In this article, we introduce ocular parallax rendering, a technology that accurately renders small amounts of gaze-contingent parallax capable of improving depth perception and realism in VR. Ocular parallax describes the small amounts of depth-dependent image shifts on the retina that are created as the eye rotates. The effect occurs because the centers of rotation and projection of the eye are not the same. We study the perceptual implications of ocular parallax rendering by designing and conducting a series of user experiments. Specifically, we estimate perceptual detection and discrimination thresholds for this effect and demonstrate that it is clearly visible in most VR applications. Additionally, we show that ocular parallax rendering provides an effective ordinal depth cue and it improves the impression of realistic depth in VR.
Interactive Optimization of Generative Image Modeling using Sequential Subspace Search and Content-based Guidance
Venue: arXiv cs.GR 2020
Year: 2020

Generative image modeling techniques such as GAN demonstrate highly convincing image generation result. However, user interaction is often necessary to obtain the desired results. Existing attempts add interactivity but require either tailored architectures or extra data. We present a human-in-the-optimization method that allows users to directly explore and search the latent vector space of generative image modeling. Our system provides multiple candidates by sampling the latent vector space, and the user selects the best blending weights within the subspace using multiple sliders. In addition, the user can express their intention through image editing tools. The system samples latent vectors based on inputs and presents new candidates to the user iteratively. An advantage of our formulation is that one can apply our method to arbitrary pre-trained model without developing specialized architecture or data. We demonstrate our method with various generative image modeling applications, and show superior performance in a comparative user study with prior art iGAN.
EKFPnP: Extended Kalman Filter for Camera Pose Estimation in a Sequence of Images
Venue: arXiv cs.CV 2020
Year: 2020

In real-world applications the Perspective-n-Point (PnP) problem should generally be applied in a sequence of images which a set of drift-prone features are tracked over time. In this paper, we consider both the temporal dependency of camera poses and the uncertainty of features for the sequential camera pose estimation. Using the Extended Kalman Filter (EKF), a priori estimate of the camera pose is calculated from the camera motion model and then corrected by minimizing the reprojection error of the reference points. Experimental results, using both simulated and real data, demonstrate that the proposed method improves the robustness of the camera pose estimation, in the presence of noise, compared to the state-of-the-art.
Emotion Cognizance Improves Health Fake News Identification
Venue: arXiv cs.SI 2020
Year: 2020

Identifying misinformation is increasingly being recognized as an important computational task with high potential social impact. Misinformation and fake contents are injected into almost every domain of news including politics, health, science, business, etc., among which, the fakeness in health domain pose serious adverse effects to scare or harm the society. Misinformation contains scientific claims or content from social media exaggerated with strong emotion content to attract eyeballs. In this paper, we consider the utility of the affective character of news articles for fake news identification in the health domain and present evidence that emotion cognizant representations are significantly more suited for the task. We outline a technique to leverage emotion intensity lexicons to develop emotionized text representations, and evaluate the utility of such a representation for identifying fake news relating to health in various supervised and unsupervised scenarios. The consistent and significant empirical gains that we observe over a range of technique types and parameter settings establish the utility of the emotional information in news articles, an often overlooked aspect, for the task of misinformation identification in the health domain.
Parallel Performance of Algebraic Multigrid Domain Decomposition (AMG-DD)
Venue: arXiv cs.MS 2020
Year: 2020

Algebraic multigrid (AMG) is a widely used scalable solver and preconditioner for large-scale linear systems resulting from the discretization of a wide class of elliptic PDEs. While AMG has optimal computational complexity, the cost of communication has become a significant bottleneck that limits its scalability as processor counts continue to grow on modern machines. This paper examines the design, implementation, and parallel performance of a novel algorithm, Algebraic Multigrid Domain Decomposition (AMG-DD), designed specifically to limit communication. The goal of AMG-DD is to provide a low-communication alternative to standard AMG V-cycles by trading some additional computational overhead for a significant reduction in communication cost. Numerical results show that AMG-DD achieves superior accuracy per communication cost compared to AMG, and speedup over AMG is demonstrated on a large GPU cluster.
A High-Performance Implementation of a Robust Preconditioner for Heterogeneous Problems
Venue: arXiv cs.MS 2020
Year: 2020

We present an efficient implementation of the highly robust and scalable GenEO preconditioner in the high-performance PDE framework DUNE. The GenEO coarse space is constructed by combining low energy solutions of a local generalised eigenproblem using a partition of unity. In this paper we demonstrate both weak and strong scaling for the GenEO solver on over 15,000 cores by solving an industrially motivated problem with over 200 million degrees of freedom. Further, we show that for highly complex parameter distributions arising in certain real-world applications, established methods become intractable while GenEO remains fully effective. The purpose of this paper is two-fold: to demonstrate the robustness and high parallel efficiency of the solver and to document the technical details that are crucial to the efficiency of the code.
XOX Fabric: A hybrid approach to blockchain transaction execution
Venue: arXiv cs.DC 2020
Year: 2020

Performance and scalability are major concerns for blockchains: permissionless systems are typically limited by slow proof of X consensus algorithms and sequential post-order transaction execution on every node of the network. By introducing a small amount of trust in their participants, permissioned blockchain systems such as Hyperledger Fabric can benefit from more efficient consensus algorithms and make use of parallel pre-order execution on a subset of network nodes. Fabric, in particular, has been shown to handle tens of thousands of transactions per second. However, this performance is only achievable for contention-free transaction workloads. If many transactions compete for a small set of hot keys in the world state, the effective throughput drops drastically. We therefore propose XOX: a novel two-pronged transaction execution approach that both minimizes invalid transactions in the Fabric blockchain and maximizes concurrent execution. Our approach additionally prevents unintentional denial of service attacks by clients re-submitting conflicting transactions. Even under fully contentious workloads, XOX can handle more than 3000 transactions per second, all of which would be discarded by regular Fabric.
A Story of Two Streams: Reinforcement Learning Models from Human Behavior and Neuropsychiatry
Venue: arXiv cs.LG 2020
Year: 2020

Drawing an inspiration from behavioral studies of human decision making, we propose here a more general and flexible parametric framework for reinforcement learning that extends standard Q-learning to a two-stream model for processing positive and negative rewards, and allows to incorporate a wide range of reward-processing biases -- an important component of human decision making which can help us better understand a wide spectrum of multi-agent interactions in complex real-world socioeconomic systems, as well as various neuropsychiatric conditions associated with disruptions in normal reward processing. From the computational perspective, we observe that the proposed Split-QL model and its clinically inspired variants consistently outperform standard Q-Learning and SARSA methods, as well as recently proposed Double Q-Learning approaches, on simulated tasks with particular reward distributions, a real-world dataset capturing human decision-making in gambling tasks, and the Pac-Man game in a lifelong learning setting across different reward stationarities.
Cascade-BGNN: Toward Efficient Self-supervised Representation Learning on Large-scale Bipartite Graphs
Venue: arXiv cs.SI 2020
Year: 2020

Bipartite graphs have been used to represent data relationships in many data-mining applications such as in E-commerce recommendation systems. Since learning in graph space is more complicated than in Euclidian space, recent studies have extensively utilized neural nets to effectively and efficiently embed a graph's nodes into a multidimensional space. However, this embedding method has not yet been applied to large-scale bipartite graphs. Existing techniques either cannot be scaled to large-scale bipartite graphs that have limited labels or cannot exploit the unique structure of bipartite graphs, which have distinct node features in two domains. Thus, we propose Cascade Bipartite Graph Neural Networks, Cascade-BGNN, a novel node representation learning for bipartite graphs that is domain-consistent, self-supervised, and efficient. To efficiently aggregate information both across and within the two partitions of a bipartite graph, BGNN utilizes a customized Inter-domain Message Passing (IDMP) and Intra-domain Alignment (IDA), which is our adaptation of adversarial learning, for message aggregation across and within partitions, respectively. BGNN is trained in a self-supervised manner. Moreover, we formulate a multi-layer BGNN in a cascaded training manner to enable multi-hop relationship modeling while improving training efficiency. Extensive experiments on several datasets of varying scales verify the effectiveness and efficiency of BGNN over baselines. Our design is further affirmed through theoretical analysis for domain alignment. The scalability of BGNN is additionally verified through its demonstrated rapid training speed and low memory cost over a large-scale real-world bipartite graph.