OpenRaiser/Intern-Atlas huggingface dataset:
Scene Categorization from Contours: Medial Axis Based Salience Measures
Venue: arXiv cs.CV 2018
Year: 2018

The computer vision community has witnessed recent advances in scene categorization from images, with the state-of-the art systems now achieving impressive recognition rates on challenging benchmarks such as the Places365 dataset. Such systems have been trained on photographs which include color, texture and shading cues. The geometry of shapes and surfaces, as conveyed by scene contours, is not explicitly considered for this task. Remarkably, humans can accurately recognize natural scenes from line drawings, which consist solely of contour-based shape cues. Here we report the first computer vision study on scene categorization of line drawings derived from popular databases including an artist scene database, MIT67, and Places365. Specifically, we use off-the-shelf pre-trained CNNs to perform scene classification given only contour information as input and find performance levels well above chance. We also show that medial-axis based contour salience methods can be used to select more informative subsets of contour pixels and that the variation in CNN classification performance on various choices for these subsets is qualitatively similar to that observed in human performance. Moreover, when the salience measures are used to weight the contours, as opposed to pruning them, we find that these weights boost our CNN performance above that for unweighted contour input. That is, the medial axis based salience weights appear to add useful information that is not available when CNNs are trained to use contours alone.
Scalable graph-based individual named entity identification
Venue: arXiv cs.IR 2018
Year: 2018

Named entity discovery (NED) is an important information retrieval problem that can be decomposed into two sub-problems. The first sub-problem, named entity recognition (NER), aims to tag pre-defined sets of words in a vocabulary (called "named entities": names, places, locations, ...) when they appear in natural language. The second subproblem, named entity linking/identification (NEL), considers these entity mentions as queries to be identified in a pre-existing database. In this paper, we consider the NEL problem, and assume a set of queries (or mentions) that have to be identified within a knowledge base. This knowledge base is represented by a text database paired with a semantic graph. We present state-of-the-art methods in NEL, and propose a 2-step method for individual identification of named entities. Our approach is well-motivated by the limitations brought by recent deep learning approaches that lack interpratability, and require lots of parameter tuning along with large volume of annotated data. First of all, we propose a filtering algorithm designed with information retrieval and text mining techniques, aiming to maximize precision at K (typically for 5 <= K <=20). Then, we introduce two graph-based methods for named entity identification to maximize precision at 1 by re-ranking the remaining top entity candidates. The first identification method is using parametrized graph mining, and the second similarity with graph kernels. Our approach capitalizes on a fine-grained classification of entities from annotated web data. We present our algorithms in details, and show experimentally on standard datasets (NIST TAC-KBP, CONLL/AIDA) their performance in terms of precision are better than any graph-based method reported, and competitive with state-of-the-art systems. Finally, we conclude on the advantages of our graph-based approach compared to recent deep learning methods.
GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
Venue: arXiv cs.CV 2018
Year: 2018

Generative Adversarial Networks (GANs) have recently achieved impressive results for many real-world applications, and many GAN variants have emerged with improvements in sample quality and training stability. However, they have not been well visualized or understood. How does a GAN represent our visual world internally? What causes the artifacts in GAN results? How do architectural choices affect GAN learning? Answering such questions could enable us to develop new insights and better models. In this work, we present an analytic framework to visualize and understand GANs at the unit-, object-, and scene-level. We first identify a group of interpretable units that are closely related to object concepts using a segmentation-based network dissection method. Then, we quantify the causal effect of interpretable units by measuring the ability of interventions to control objects in the output. We examine the contextual relationship between these units and their surroundings by inserting the discovered object concepts into new images. We show several practical applications enabled by our framework, from comparing internal representations across different layers, models, and datasets, to improving GANs by locating and removing artifact-causing units, to interactively manipulating objects in a scene. We provide open source interpretation tools to help researchers and practitioners better understand their GAN models.
Noisy Computations during Inference: Harmful or Helpful?
Venue: arXiv cs.LG 2018
Year: 2018

We study two aspects of noisy computations during inference. The first aspect is how to mitigate their side effects for naturally trained deep learning systems. One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of neural networks through the use of analog neuromorphic circuits. Traditional GPU/CPU-centered deep learning architectures exhibit bottlenecks in power-restricted applications (e.g., embedded systems). The use of specialized neuromorphic circuits, where analog signals passed through memory-cell arrays are sensed to accomplish matrix-vector multiplications, promises large power savings and speed gains but brings with it the problems of limited precision of computations and unavoidable analog noise. We manage to improve inference accuracy from 21.1% to 99.5% for MNIST images, from 29.9% to 89.1% for CIFAR10, and from 15.5% to 89.6% for MNIST stroke sequences with the presence of strong noise (with signal-to-noise power ratio being 0 dB) by noise-injected training and a voting method. This observation promises neural networks that are insensitive to inference noise, which reduces the quality requirements on neuromorphic circuits and is crucial for their practical usage. The second aspect is how to utilize the noisy inference as a defensive architecture against black-box adversarial attacks. During inference, by injecting proper noise to signals in the neural networks, the robustness of adversarially-trained neural networks against black-box attacks has been further enhanced by 0.5% and 1.13% for two adversarially trained models for MNIST and CIFAR10, respectively.
Stepping Stones to Inductive Synthesis of Low-Level Looping Programs
Venue: arXiv cs.AI 2018
Year: 2018

Inductive program synthesis, from input/output examples, can provide an opportunity to automatically create programs from scratch without presupposing the algorithmic form of the solution. For induction of general programs with loops (as opposed to loop-free programs, or synthesis for domain-specific languages), the state of the art is at the level of introductory programming assignments. Most problems that require algorithmic subtlety, such as fast sorting, have remained out of reach without the benefit of significant problem-specific background knowledge. A key challenge is to identify cues that are available to guide search towards correct looping programs. We present MAKESPEARE, a simple delayed-acceptance hillclimbing method that synthesizes low-level looping programs from input/output examples. During search, delayed acceptance bypasses small gains to identify significantly-improved stepping stone programs that tend to generalize and enable further progress. The method performs well on a set of established benchmarks, and succeeds on the previously unsolved "Collatz Numbers" program synthesis problem. Additional benchmarks include the problem of rapidly sorting integer arrays, in which we observe the emergence of comb sort (a Shell sort variant that is empirically fast). MAKESPEARE has also synthesized a record-setting program on one of the puzzles from the TIS-100 assembly language programming game.
Time-Aware and View-Aware Video Rendering for Unsupervised Representation Learning
Venue: arXiv cs.CV 2018
Year: 2018

The recent success in deep learning has lead to various effective representation learning methods for videos. However, the current approaches for video representation require large amount of human labeled datasets for effective learning. We present an unsupervised representation learning framework to encode scene dynamics in videos captured from multiple viewpoints. The proposed framework has two main components: Representation Learning Network (RL-NET), which learns a representation with the help of Blending Network (BL-NET), and Video Rendering Network (VR-NET), which is used for video synthesis. The framework takes as input video clips from different viewpoints and time, learns an internal representation and uses this representation to render a video clip from an arbitrary given viewpoint and time. The ability of the proposed network to render video frames from arbitrary viewpoints and time enable it to learn a meaningful and robust representation of the scene dynamics. We demonstrate the effectiveness of the proposed method in rendering view-aware as well as time-aware video clips on two different real-world datasets including UCF-101 and NTU-RGB+D. To further validate the effectiveness of the learned representation, we use it for the task of view-invariant activity classification where we observe a significant improvement (~26%) in the performance on NTU-RGB+D dataset compared to the existing state-of-the art methods.
Modular decomposition of graphs and hierarchical modeling
Venue: arXiv cs.DM 2018
Year: 2018

We consider Gallai's graph Modular Decomposition theory for network analytics. On the one hand, by arguing that this is a choice tool for understanding structural and functional similarities among nodes in a network. On the other, by proposing a model for random graphs based on this decomposition. Our approach establishes a well defined context for hierarchical modeling and provides a solid theoretical framework for probabilistic and statistical methods. Theoretical and simulation results show the model acknowledges scale free networks, high clustering coefficients and small diameters all of which are observed features in many natural and social networks.
Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System
Venue: arXiv cs.AI 2018
Year: 2018

Argumentation-based dialogue systems, which can handle and exchange arguments through dialogue, have been widely researched. It is required that these systems have sufficient supporting information to argue their claims rationally; however, the systems often do not have enough of such information in realistic situations. One way to fill in the gap is acquiring such missing information from dialogue partners (information-seeking dialogue). Existing information-seeking dialogue systems are based on handcrafted dialogue strategies that exhaustively examine missing information. However, the proposed strategies are not specialized in collecting information for constructing rational arguments. Moreover, the number of system's inquiry candidates grows in accordance with the size of the argument set that the system deal with. In this paper, we formalize the process of information-seeking dialogue as Markov decision processes (MDPs) and apply deep reinforcement learning (DRL) for automatically optimizing a dialogue strategy. By utilizing DRL, our dialogue strategy can successfully minimize objective functions, the number of turns it takes for our system to collect necessary information in a dialogue. We conducted dialogue experiments using two datasets from different domains of argumentative dialogue. Experimental results show that the proposed formalization based on MDP works well, and the policy optimized by DRL outperformed existing heuristic dialogue strategies.
Generating Attention from Classifier Activations for Fine-grained Recognition
Venue: arXiv cs.CV 2018
Year: 2018

Recent advances in fine-grained recognition utilize attention maps to localize objects of interest. Although there are many ways to generate attention maps, most of them rely on sophisticated loss functions or complex training processes. In this work, we propose a simple and straightforward attention generation model based on the output activations of classifiers. The advantage of our model is that it can be easily trained with image level labels and softmax loss functions. More specifically, multiple linear local classifiers are firstly adopted to perform fine-grained classification at each location of high level CNN feature maps. The attention map is generated by aggregating and max-pooling the output activations. Then the attention map serves as a surrogate target object mask to train those local classifiers, similar to training models for semantic segmentation. Our model achieves state-of-the-art results on three heavily benchmarked datasets, i.e. 87.9% on CUB-200-2011 dataset, 94.1% on Stanford Cars dataset and 92.1% on FGVC-Aircraft dataset, demonstrating its effectiveness on fine-grained recognition tasks.
Verb Argument Structure Alternations in Word and Sentence Embeddings
Venue: arXiv cs.CL 2018
Year: 2018

Verbs occur in different syntactic environments, or frames. We investigate whether artificial neural networks encode grammatical distinctions necessary for inferring the idiosyncratic frame-selectional properties of verbs. We introduce five datasets, collectively called FAVA, containing in aggregate nearly 10k sentences labeled for grammatical acceptability, illustrating different verbal argument structure alternations. We then test whether models can distinguish acceptable English verb-frame combinations from unacceptable ones using a sentence embedding alone. For converging evidence, we further construct LaVA, a corresponding word-level dataset, and investigate whether the same syntactic features can be extracted from word embeddings. Our models perform reliable classifications for some verbal alternations but not others, suggesting that while these representations do encode fine-grained lexical information, it is incomplete or can be hard to extract. Further, differences between the word- and sentence-level models show that some information present in word embeddings is not passed on to the down-stream sentence embeddings.
Flexible Attributed Network Embedding
Venue: arXiv cs.SI 2018
Year: 2018

Network embedding aims to find a way to encode network by learning an embedding vector for each node in the network. The network often has property information which is highly informative with respect to the node's position and role in the network. Most network embedding methods fail to utilize this information during network representation learning. In this paper, we propose a novel framework, FANE, to integrate structure and property information in the network embedding process. In FANE, we design a network to unify heterogeneity of the two information sources, and define a new random walking strategy to leverage property information and make the two information compensate. FANE is conceptually simple and empirically powerful. It improves over the state-of-the-art methods on Cora dataset classification task by over 5%, more than 10% on WebKB dataset classification task. Experiments also show that the results improve more than the state-of-the-art methods as increasing training size. Moreover, qualitative visualization show that our framework is helpful in network property information exploration. In all, we present a new way for efficiently learning state-of-the-art task-independent representations in complex attributed networks. The source code and datasets of this paper can be obtained from https://github.com/GraphWorld/FANE.
Lightweight Interactive Proving inside an Automatic Program Verifier
Venue: arXiv cs.SE 2018
Year: 2018

Among formal methods, the deductive verification approach allows establishing the strongest possible formal guarantees on critical software. The downside is the cost in terms of human effort required to design adequate formal specifications and to successfully discharge the required proof obligations. To popularize deductive verification in an industrial software development environment, it is essential to provide means to progressively transition from simple and automated approaches to deductive verification. The SPARK environment, for development of critical software written in Ada, goes towards this goal by providing automated tools for formally proving that some code fulfills the requirements expressed in Ada contracts. In a program verifier that makes use of automatic provers to discharge the proof obligations, a need for some additional user interaction with proof tasks shows up: either to help analyzing the reason of a proof failure or, ultimately, to discharge the verification conditions that are out-of-reach of state-of-the-art automatic provers. Adding interactive proof features in SPARK appears to be complicated by the fact that the proof toolchain makes use of the independent, intermediate verification tool Why3, which is generic enough to accept multiple front-ends for different input languages. This paper reports on our approach to extend Why3 with interactive proof features and also with a generic client-server infrastructure allowing integration of proof interaction into an external, front-end graphical user interface such as the one of SPARK.
Integrating User Design and Formal Models within PVSio-Web
Venue: arXiv cs.SE 2018
Year: 2018

Creating formal models of interactive systems has wide reaching benefits, not only for verifying low-level correctness, but also as a tool for ensuring user interfaces behave logically and consistently. Despite this, tools for designing user experiences and tools for creating and working with formal models are typically distinctly separate systems. This work aims to bridge this divide by allowing the generation of state machine diagrams and formal models via a simple, interactive prototyping tool that mirrors the basic functionality of many modern digital prototyping applications.
A Schur Complement Cheeger Inequality
Venue: arXiv cs.DM 2018
Year: 2018

Cheeger's inequality shows that any undirected graph $G$ with minimum nonzero normalized Laplacian eigenvalue $λ_G$ has a cut with conductance at most $O(\sqrt{λ_G})$. Qualitatively, Cheeger's inequality says that if the relaxation time of a graph is high, there is a cut that certifies this. However, there is a gap in this relationship, as cuts can have conductance as low as $Θ(λ_G)$. To better approximate the relaxation time of a graph, we consider a more general object. Instead of bounding the mixing time with cuts, we bound it with cuts in graphs obtained by Schur complementing out vertices from the graph $G$. Combinatorially, these Schur complements describe random walks in $G$ restricted to a subset of its vertices. As a result, all Schur complement cuts have conductance at least $Ω(λ_G)$. We show that unlike with cuts, this inequality is tight up to a constant factor. Specifically, there is a Schur complement cut with conductance at most $O(λ_G)$.
A Frequency Scaling based Performance Indicator Framework for Big Data Systems
Venue: arXiv cs.DB 2018
Year: 2018

It is important for big data systems to identify their performance bottleneck. However, the popular indicators such as resource utilizations, are often misleading and incomparable with each other. In this paper, a novel indicator framework which can directly compare the impact of different indicators with each other is proposed to identify and analyze the performance bottleneck efficiently. A methodology which can construct the indicator from the performance change with the CPU frequency scaling is described. Spark is used as an example of a big data system and two typical SQL benchmarks are used as the workloads to evaluate the proposed method. Experimental results show that the proposed method is accurate compared with the resource utilization method and easy to implement compared with some white-box method. Meanwhile, the analysis with our indicators lead to some interesting findings and valuable performance optimization suggestions for big data systems.
Algae Detection Using Computer Vision and Deep Learning
Venue: arXiv cs.CV 2018
Year: 2018

A disconcerting ramification of water pollution caused by burgeoning populations, rapid industrialization and modernization of agriculture, has been the exponential increase in the incidence of algal growth across the globe. Harmful algal blooms (HABs) have devastated fisheries, contaminated drinking water and killed livestock, resulting in economic losses to the tune of millions of dollars. Therefore, it is important to constantly monitor water bodies and identify any algae build-up so that prompt action against its accumulation can be taken and the harmful consequences can be avoided. In this paper, we propose a computer vision system based on deep learning for algae monitoring. The proposed system is fast, accurate and cheap, and it can be installed on any robotic platforms such as USVs and UAVs for autonomous algae monitoring. The experimental results demonstrate that the proposed system can detect algae in distinct environments regardless of the underlying hardware with high accuracy and in real time.
Research on CRO's Dilemma In Sapiens Chain: A Game Theory Method
Venue: arXiv cs.GT 2018
Year: 2018

In recent years, blockchain-based techniques have been widely used in cybersecurity, owing to the decentralization, anonymity, credibility and not be tampered properties of the blockchain. As one of the decentralized framework, Sapiens Chain was proposed to protect cybersecurity by scheduling the computational resources dynamically, which were owned by Computational Resources Owners (CROs). However, when CROs in the same pool attack each other, all CROs will earn less. In this paper, we tackle the problem of prisoner's dilemma from the perspective of CROs. We first define a game that a CRO infiltrates another pool and perform an attack. In such game, the honest CRO can control the payoffs and increase its revenue. By simulating this game, we propose to apply Zero Determinant (ZD) strategy on strategy decision, which can be categorized into cooperation and defecting. Our experimental results demonstrate the effectiveness of the proposed strategy decision method.