OpenRaiser/Intern-Atlas huggingface dataset:
Neural Baby Talk
Venue: arXiv cs.CV 2018
Year: 2018

We introduce a novel framework for image captioning that can produce natural language explicitly grounded in entities that object detectors find in the image. Our approach reconciles classical slot filling approaches (that are generally better grounded in images) with modern neural captioning approaches (that are generally more natural sounding and accurate). Our approach first generates a sentence `template' with slot locations explicitly tied to specific image regions. These slots are then filled in by visual concepts identified in the regions by object detectors. The entire architecture (sentence template generation and slot filling with object detectors) is end-to-end differentiable. We verify the effectiveness of our proposed model on different image captioning tasks. On standard image captioning and novel object captioning, our model reaches state-of-the-art on both COCO and Flickr30k datasets. We also demonstrate that our model has unique advantages when the train and test distributions of scene compositions -- and hence language priors of associated captions -- are different. Code has been made available at: https://github.com/jiasenlu/NeuralBabyTalk
MLE-induced Likelihood for Markov Random Fields
Venue: arXiv cs.LG 2018
Year: 2018

Due to the intractable partition function, the exact likelihood function for a Markov random field (MRF), in many situations, can only be approximated. Major approximation approaches include pseudolikelihood and Laplace approximation. In this paper, we propose a novel way of approximating the likelihood function through first approximating the marginal likelihood functions of individual parameters and then reconstructing the joint likelihood function from these marginal likelihood functions. For approximating the marginal likelihood functions, we derive a particular likelihood function from a modified scenario of coin tossing which is useful for capturing how one parameter interacts with the remaining parameters in the likelihood function. For reconstructing the joint likelihood function, we use an appropriate copula to link up these marginal likelihood functions. Numerical investigation suggests the superior performance of our approach. Especially as the size of the MRF increases, both the numerical performance and the computational cost of our approach remain consistently satisfactory, whereas Laplace approximation deteriorates and pseudolikelihood becomes computationally unbearable.
Cross-validation in high-dimensional spaces: a lifeline for least-squares models and multi-class LDA
Venue: arXiv cs.LG 2018
Year: 2018

Least-squares models such as linear regression and Linear Discriminant Analysis (LDA) are amongst the most popular statistical learning techniques. However, since their computation time increases cubically with the number of features, they are inefficient in high-dimensional neuroimaging datasets. Fortunately, for k-fold cross-validation, an analytical approach has been developed that yields the exact cross-validated predictions in least-squares models without explicitly training the model. Its computation time grows with the number of test samples. Here, this approach is systematically investigated in the context of cross-validation and permutation testing. LDA is used exemplarily but results hold for all other least-squares methods. Furthermore, a non-trivial extension to multi-class LDA is formally derived. The analytical approach is evaluated using complexity calculations, simulations, and permutation testing of an EEG/MEG dataset. Depending on the ratio between features and samples, the analytical approach is up to 10,000x faster than the standard approach (retraining the model on each training set). This allows for a fast cross-validation of least-squares models and multi-class LDA in high-dimensional data, with obvious applications in multi-dimensional datasets, Representational Similarity Analysis, and permutation testing.
Recent Developments from Attribute Profiles for Remote Sensing Image Classification
Venue: arXiv cs.CV 2018
Year: 2018

Morphological attribute profiles (APs) are among the most effective methods to model the spatial and contextual information for the analysis of remote sensing images, especially for classification task. Since their first introduction to this field in early 2010's, many research studies have been contributed not only to exploit and adapt their use to different applications, but also to extend and improve their performance for better dealing with more complex data. In this paper, we revisit and discuss different developments and extensions from APs which have drawn significant attention from researchers in the past few years. These studies are analyzed and gathered based on the concept of multi-stage AP construction. In our experiments, a comparative study on classification results of two remote sensing data is provided in order to show their significant improvements compared to the originally proposed APs.
Scalable closed-form trajectories for periodic and non-periodic human-like walking
Venue: arXiv cs.RO 2018
Year: 2018

We present a new framework to generate human-like lower-limb trajectories in periodic and non-periodic walking conditions. In our method, walking dynamics is encoded in 3LP, a linear simplified model composed of three pendulums to model falling, swing and torso balancing dynamics. To stabilize the motion, we use an optimal time-projecting controller which suggests new footstep locations. On top of gait generation and stabilization in the simplified space, we introduce a kinematic conversion method that synthesizes more human-like trajectories by combining geometric variables of the 3LP model adaptively. Without any tuning, numerical optimization or off-line data, our walking gaits are scalable with respect to body properties and gait parameters. We can change various parameters such as body mass and height, walking direction, speed, frequency, double support time, torso style, ground clearance and terrain inclination. We can also simulate the effect of constant external dragging forces or momentary perturbations. The proposed framework offers closed-form solutions in all the three stages which enable simulation speeds orders of magnitude faster than real time. This can be used for video games and animations on portable electronic devices with a limited power. It also gives insights for generation of more human-like walking gaits with humanoid robots.
Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning
Venue: arXiv cs.RO 2018
Year: 2018

This paper introduces a method, based on deep reinforcement learning, for automatically generating a general purpose decision making function. A Deep Q-Network agent was trained in a simulated environment to handle speed and lane change decisions for a truck-trailer combination. In a highway driving case, it is shown that the method produced an agent that matched or surpassed the performance of a commonly used reference model. To demonstrate the generality of the method, the exact same algorithm was also tested by training it for an overtaking case on a road with oncoming traffic. Furthermore, a novel way of applying a convolutional neural network to high level input that represents interchangeable objects is also introduced.
Tensor graph convolutional neural network
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we propose a novel tensor graph convolutional neural network (TGCNN) to conduct convolution on factorizable graphs, for which here two types of problems are focused, one is sequential dynamic graphs and the other is cross-attribute graphs. Especially, we propose a graph preserving layer to memorize salient nodes of those factorized subgraphs, i.e. cross graph convolution and graph pooling. For cross graph convolution, a parameterized Kronecker sum operation is proposed to generate a conjunctive adjacency matrix characterizing the relationship between every pair of nodes across two subgraphs. Taking this operation, then general graph convolution may be efficiently performed followed by the composition of small matrices, which thus reduces high memory and computational burden. Encapsuling sequence graphs into a recursive learning, the dynamics of graphs can be efficiently encoded as well as the spatial layout of graphs. To validate the proposed TGCNN, experiments are conducted on skeleton action datasets as well as matrix completion dataset. The experiment results demonstrate that our method can achieve more competitive performance with the state-of-the-art methods.
A New Target-specific Object Proposal Generation Method for Visual Tracking
Venue: arXiv cs.CV 2018
Year: 2018

Object proposal generation methods have been widely applied to many computer vision tasks. However, existing object proposal generation methods often suffer from the problems of motion blur, low contrast, deformation, etc., when they are applied to video related tasks. In this paper, we propose an effective and highly accurate target-specific object proposal generation (TOPG) method, which takes full advantage of the context information of a video to alleviate these problems. Specifically, we propose to generate target-specific object proposals by integrating the information of two important objectness cues: colors and edges, which are complementary to each other for different challenging environments in the process of generating object proposals. As a result, the recall of the proposed TOPG method is significantly increased. Furthermore, we propose an object proposal ranking strategy to increase the rank accuracy of the generated object proposals. The proposed TOPG method has yielded significant recall gain (about 20%-60% higher) compared with several state-of-the-art object proposal methods on several challenging visual tracking datasets. Then, we apply the proposed TOPG method to the task of visual tracking and propose a TOPG-based tracker (called as TOPGT), where TOPG is used as a sample selection strategy to select a small number of high-quality target candidates from the generated object proposals. Since the object proposals generated by the proposed TOPG cover many hard negative samples and positive samples, these object proposals can not only be used for training an effective classifier, but also be used as target candidates for visual tracking. Experimental results show the superior performance of TOPGT for visual tracking compared with several other state-of-the-art visual trackers (about 3%-11% higher than the winner of the VOT2015 challenge in term of distance precision).
Sampling the News Producers: A Large News and Feature Data Set for the Study of the Complex Media Landscape
Venue: arXiv cs.CY 2018
Year: 2018

The complexity and diversity of today's media landscape provides many challenges for researchers studying news producers. These producers use many different strategies to get their message believed by readers through the writing styles they employ, by repetition across different media sources with or without attribution, as well as other mechanisms that are yet to be studied deeply. To better facilitate systematic studies in this area, we present a large political news data set, containing over 136K news articles, from 92 news sources, collected over 7 months of 2017. These news sources are carefully chosen to include well-established and mainstream sources, maliciously fake sources, satire sources, and hyper-partisan political blogs. In addition to each article we compute 130 content-based and social media engagement features drawn from a wide range of literature on political bias, persuasion, and misinformation. With the release of the data set, we also provide the source code for feature computation. In this paper, we discuss the first release of the data set and demonstrate 4 use cases of the data and features: news characterization, engagement characterization, news attribution and content copying, and discovering news narratives.
The algebra of predicting agents
Venue: arXiv cs.GT 2018
Year: 2018

The category of open games, which provides a strongly compositional foundation of economic game theory, is intermediate between symmetric monoidal and compact closed. More precisely it has counits with no corresponding units, and a partially defined duality. There exist open games with the same types as unit maps, given by agents with the strategic goal of predicting a future value. Such agents appear in earlier work on selection functions. We explore the algebraic properties of these agents via the symmetric monoidal bicategory whose 2-cells are morphisms between open games, and show how the resulting structure approximates a compact closed category with a family of lax commutative bialgebras.
Public Git Archive: a Big Code dataset for all
Venue: arXiv cs.SE 2018
Year: 2018

The number of open source software projects has been growing exponentially. The major online software repository host, GitHub, has accumulated tens of millions of publicly available Git version-controlled repositories. Although the research potential enabled by the available open source code is clearly substantial, no significant large-scale open source code datasets exist. In this paper, we present the Public Git Archive -- dataset of 182,014 top-bookmarked Git repositories from GitHub. We describe the novel data retrieval pipeline to reproduce it. We also elaborate on the strategy for performing dataset updates and legal issues. The Public Git Archive occupies 3.0 TB on disk and is an order of magnitude larger than the current source code datasets. The dataset is made available through HTTP and provides the source code of the projects, the related metadata, and development history. The data retrieval pipeline employs an optimized worker queue model and an optimized archive format to efficiently store forked Git repositories, reducing the amount of data to download and persist. Public Git Archive aims to open a myriad of new opportunities for ``Big Code`` research.
A New Aggregation based Scheduling method for rapidly changing IEEE 802.11ac Wireless channels
Venue: arXiv cs.NI 2018
Year: 2018

In this paper we suggest a novel idea to improve the Throughput of a rapidly chang- ing WiFi channel by exploiting the standard aggregation schemes in IEEE 802.11ac networks, and by transmitting several copies of the same MPDU(s) in a single trans- mission attempt. We test this idea in scenarios where Link Adaptation is not used and show a significant improvement, in the order of tens of percents, in the achieved Throughput. Keywords
Live Multi-language Development and Runtime Environments
Venue: arXiv cs.PL 2018
Year: 2018

Context: Software development tools should work and behave consistently across different programming languages, so that developers do not have to familiarize themselves with new tooling for new languages. Also, being able to combine multiple programming languages in a program increases reusability, as developers do not have to recreate software frameworks and libraries in the language they develop in and can reuse existing software instead. Inquiry: However, developers often have a broad choice of tools, some of which are designed for only one specific programming language. Various Integrated Development Environments have support for multiple languages, but are usually unable to provide a consistent programming experience due to different language-specific runtime features. With regard to language integrations, common mechanisms usually use abstraction layers, such as the operating system or a network connection, which are often boundaries for tools and hence negatively affect the programming experience. Approach: In this paper, we present a novel approach for tool reuse that aims to improve the experience with regard to working with multiple high-level dynamic, object-oriented programming languages. As part of this, we build a multi-language virtual execution environment and reuse Smalltalk's live programming tools for other languages. Knowledge: An important part of our approach is to retrofit and align runtime capabilities for different languages as it is a requirement for providing consistent tools. Furthermore, it provides convenient means to reuse and even mix software libraries and frameworks written in different languages without breaking tool support. Grounding: The prototype system Squimera is an implementation of our approach and demonstrates that it is possible to reuse both development tools from a live programming system to improve the development experience as well as software artifacts from different languages to increase productivity. Importance: In the domain of polyglot programming systems, most research has focused on the integration of different languages and corresponding performance optimizations. Our work, on the other hand, focuses on tooling and the overall programming experience.
Fast, Flexible, Polyglot Instrumentation Support for Debuggers and other Tools
Venue: arXiv cs.PL 2018
Year: 2018

Context: Software development tools that interact with running programs such as debuggers, profilers, and dynamic analysis frameworks are presumed to demand difficult tradeoffs among implementation complexity (cost), functionality, usability, and performance. Among the many consequences, tools are often delivered late (if ever), have limited functionality, require non-standard configurations, and impose serious performance costs on running programs. Inquiry: Can flexible tool support become a practical, first class, intrinsic requirement for a modern highperformance programming language implementation framework? Approach: We extended the Truffle Language Implementation Framework, which together with the GraalVM execution environment makes possible very high performance language implementations. Truffle's new Instrumentation Framework is language-agnostic and designed to derive high performance from the same technologies as do language implementations. Truffle Instrumentation includes: (1) low overhead capture of execution events by dynamically adding "wrapper" nodes to executing ASTs; (2) extensions to the Language Implementation Framework that allow per-language specialization, primarily for visual display of values and names, among others; and (3) versatile APIs and support services for implementing many kinds of tools without VM modification. Knowledge: It is now possible for a client in a production environment to insert (dynamically, with thread safety) an instrumentation probe that captures and reports abstractly specified execution events. A probe in fully optimized code imposes very low overhead until actually used to access (or modify) execution state. Event capture has enabled construction of numerous GraalVM services and tools that work for all implemented languages, either singly or in combination. Instrumentation has also proved valuable for implementing some traditionally tricky language features, as well as some GraalVM services such as placing bounds on resources consumed by running programs. Grounding: Tools for debugging (via multiple clients), profiling, statement counting, dynamic analysis, and others are now part of GraalVM or are in active development. Third parties have also used Truffle Instrumentation for innovative tool implementations. Importance: Experience with Truffle Instrumentation validates the notion that addressing developer tools support as a forethought can change expectations about the availability of practical, efficient tools for high-performance languages. Tool development becomes a natural part of language implementation, requiring little additional effort and offering the advantage of early and continuous availability.
A Formal TLS Handshake Model in LNT
Venue: arXiv cs.CR 2018
Year: 2018

Testing of network services represents one of the biggest challenges in cyber security. Because new vulnerabilities are detected on a regular basis, more research is needed. These faults have their roots in the software development cycle or because of intrinsic leaks in the system specification. Conformance testing checks whether a system behaves according to its specification. Here model-based testing provides several methods for automated detection of shortcomings. The formal specification of a system behavior represents the starting point of the testing process. In this paper, a widely used cryptographic protocol is specified and tested for conformance with a test execution framework. The first empirical results are presented and discussed.
Comparative Study of Eight Formal Specifications of the Message Authenticator Algorithm
Venue: arXiv cs.PL 2018
Year: 2018

The Message Authenticator Algorithm (MAA) is one of the first cryptographic functions for computing a Message Authentication Code. Between 1987 and 2001, the MAA was adopted in international standards (ISO 8730 and ISO 8731-2) to ensure the authenticity and integrity of banking transactions. In 1990 and 1991, three formal, yet non-executable, specifications of the MAA (in VDM, Z, and LOTOS) were developed at NPL. Since then, five formal executable specifications of the MAA (in LOTOS, LNT, and term rewrite systems) have been designed at INRIA Grenoble. This article provides an overview of the MAA and compares its formal specifications with respect to common-sense criteria, such as conciseness, readability, and efficiency of code generation.
Modeling a Cache Coherence Protocol with the Guarded Action Language
Venue: arXiv cs.LO 2018
Year: 2018

We present a formal model built for verification of the hardware Tera-Scale ARchitecture (TSAR), focusing on its Distributed Hybrid Cache Coherence Protocol (DHCCP). This protocol is by nature asynchronous, concurrent and distributed, which makes classical validation of the design (e.g. through testing) difficult. We therefore applied formal methods to prove essential properties of the protocol, such as absence of deadlocks, eventual consensus, and fairness.
Graph Convolutions on Spectral Embeddings: Learning of Cortical Surface Data
Venue: arXiv cs.CV 2018
Year: 2018

Neuronal cell bodies mostly reside in the cerebral cortex. The study of this thin and highly convoluted surface is essential for understanding how the brain works. The analysis of surface data is, however, challenging due to the high variability of the cortical geometry. This paper presents a novel approach for learning and exploiting surface data directly across surface domains. Current approaches rely on geometrical simplifications, such as spherical inflations, a popular but costly process. For instance, the widely used FreeSurfer takes about 3 hours to parcellate brain surfaces on a standard machine. Direct learning of surface data via graph convolutions would provide a new family of fast algorithms for processing brain surfaces. However, the current limitation of existing state-of-the-art approaches is their inability to compare surface data across different surface domains. Surface bases are indeed incompatible between brain geometries. This paper leverages recent advances in spectral graph matching to transfer surface data across aligned spectral domains. This novel approach enables a direct learning of surface data across compatible surface bases. It exploits spectral filters over intrinsic representations of surface neighborhoods. We illustrate the benefits of this approach with an application to brain parcellation. We validate the algorithm over 101 manually labeled brain surfaces. The results show a significant improvement in labeling accuracy over recent Euclidean approaches, while gaining a drastic speed improvement over conventional methods.