OpenRaiser/Intern-Atlas huggingface dataset:
Hardware-efficient on-line learning through pipelined truncated-error backpropagation in binary-state networks
Venue: arXiv cs.NE 2017
Year: 2017

Artificial neural networks (ANNs) trained using backpropagation are powerful learning architectures that have achieved state-of-the-art performance in various benchmarks. Significant effort has been devoted to developing custom silicon devices to accelerate inference in ANNs. Accelerating the training phase, however, has attracted relatively little attention. In this paper, we describe a hardware-efficient on-line learning technique for feedforward multi-layer ANNs that is based on pipelined backpropagation. Learning is performed in parallel with inference in the forward pass, removing the need for an explicit backward pass and requiring no extra weight lookup. By using binary state variables in the feedforward network and ternary errors in truncated-error backpropagation, the need for any multiplications in the forward and backward passes is removed, and memory requirements for the pipelining are drastically reduced. Further reduction in addition operations owing to the sparsity in the forward neural and backpropagating error signal paths contributes to highly efficient hardware implementation. For proof-of-concept validation, we demonstrate on-line learning of MNIST handwritten digit classification on a Spartan 6 FPGA interfacing with an external 1Gb DDR2 DRAM, that shows small degradation in test error performance compared to an equivalently sized binary ANN trained off-line using standard back-propagation and exact errors. Our results highlight an attractive synergy between pipelined backpropagation and binary-state networks in substantially reducing computation and memory requirements, making pipelined on-line learning practical in deep networks.
Gray-box optimization and factorized distribution algorithms: where two worlds collide
Venue: arXiv cs.NE 2017
Year: 2017

The concept of gray-box optimization, in juxtaposition to black-box optimization, revolves about the idea of exploiting the problem structure to implement more efficient evolutionary algorithms (EAs). Work on factorized distribution algorithms (FDAs), whose factorizations are directly derived from the problem structure, has also contributed to show how exploiting the problem structure produces important gains in the efficiency of EAs. In this paper we analyze the general question of using problem structure in EAs focusing on confronting work done in gray-box optimization with related research accomplished in FDAs. This contrasted analysis helps us to identify, in current studies on the use problem structure in EAs, two distinct analytical characterizations of how these algorithms work. Moreover, we claim that these two characterizations collide and compete at the time of providing a coherent framework to investigate this type of algorithms. To illustrate this claim, we present a contrasted analysis of formalisms, questions, and results produced in FDAs and gray-box optimization. Common underlying principles in the two approaches, which are usually overlooked, are identified and discussed. Besides, an extensive review of previous research related to different uses of the problem structure in EAs is presented. The paper also elaborates on some of the questions that arise when extending the use of problem structure in EAs, such as the question of evolvability, high cardinality of the variables and large definition sets, constrained and multi-objective problems, etc. Finally, emergent approaches that exploit neural models to capture the problem structure are covered.
An Optimal Bayesian Network Based Solution Scheme for the Constrained Stochastic On-line Equi-Partitioning Problem
Venue: arXiv cs.AI 2017
Year: 2017

A number of intriguing decision scenarios revolve around partitioning a collection of objects to optimize some application specific objective function. This problem is generally referred to as the Object Partitioning Problem (OPP) and is known to be NP-hard. We here consider a particularly challenging version of OPP, namely, the Stochastic On-line Equi-Partitioning Problem (SO-EPP). In SO-EPP, the target partitioning is unknown and has to be inferred purely from observing an on-line sequence of object pairs. The paired objects belong to the same partition with probability $p$ and to different partitions with probability $1-p$, with $p$ also being unknown. As an additional complication, the partitions are required to be of equal cardinality. Previously, only sub-optimal solution strategies have been proposed for SO- EPP. In this paper, we propose the first optimal solution strategy. In brief, the scheme that we propose, BN-EPP, is founded on a Bayesian network representation of SO-EPP problems. Based on probabilistic reasoning, we are not only able to infer the underlying object partitioning with optimal accuracy. We are also able to simultaneously infer $p$, allowing us to accelerate learning as object pairs arrive. Furthermore, our scheme is the first to support arbitrary constraints on the partitioning (Constrained SO-EPP). Being optimal, BN-EPP provides superior performance compared to existing solution schemes. We additionally introduce Walk-BN-EPP, a novel WalkSAT inspired algorithm for solving large scale BN-EPP problems. Finally, we provide a BN-EPP based solution to the problem of order picking, a representative real-life application of BN-EPP.
Adversarial Generation of Training Examples: Applications to Moving Vehicle License Plate Recognition
Venue: arXiv cs.CV 2017
Year: 2017

Generative Adversarial Networks (GAN) have attracted much research attention recently, leading to impressive results for natural image generation. However, to date little success was observed in using GAN generated images for improving classification tasks. Here we attempt to explore, in the context of car license plate recognition, whether it is possible to generate synthetic training data using GAN to improve recognition accuracy. With a carefully-designed pipeline, we show that the answer is affirmative. First, a large-scale image set is generated using the generator of GAN, without manual annotation. Then, these images are fed to a deep convolutional neural network (DCNN) followed by a bidirectional recurrent neural network (BRNN) with long short-term memory (LSTM), which performs the feature learning and sequence labelling. Finally, the pre-trained model is fine-tuned on real images. Our experimental results on a few data sets demonstrate the effectiveness of using GAN images: an improvement of 7.5% over a strong baseline with moderate-sized real data being available. We show that the proposed framework achieves competitive recognition accuracy on challenging test datasets. We also leverage the depthwise separate convolution to construct a lightweight convolutional RNN, which is about half size and 2x faster on CPU. Combining this framework and the proposed pipeline, we make progress in performing accurate recognition on mobile and embedded devices.
Multi-antenna Enabled Cluster-based Cooperation in Wireless Powered Communication Networks
Venue: arXiv cs.NI 2017
Year: 2017

In this paper, we consider a wireless powered communication network (WPCN) consisting of a multi-antenna hybrid access point (HAP) that transfers wireless energy to and receives sensing data from a cluster of low-power wireless devices (WDs). To enhance the throughput performance of some far-away WDs, we allow one of the WDs to act as the cluster head (CH) that helps forward the messages of the other cluster members (CMs). However, the performance of the proposed cluster-based cooperation is fundamentally limited by the high energy consumption of the CH, who needs to transmit all the WDs' messages including its own. To tackle this issue, we exploit the capability of multi-antenna energy beamforming (EB) at the HAP, which can focus more transferred power to the CH to balance its energy consumption in assisting the other WDs. Specifically, we first derive the throughput performance of each individual WD under the proposed scheme. Then, we jointly optimize the EB design, the transmit time allocation among the HAP and the WDs, and the transmit power allocation of the CH to maximize the minimum data rate achievable among all the WDs (the max-min throughput) for improved throughput fairness among the WDs. An efficient optimal algorithm is proposed to solve the joint optimization problem. Moreover, we simulate under practical network setups and show that the proposed multi-antenna enabled cluster-based cooperation can effectively improve the throughput fairness of WPCN.
CoAP over ICN
Venue: arXiv cs.NI 2017
Year: 2017

The Constrained Application Protocol (CoAP) is a specialized Web transfer protocol for resource-oriented applications intended to run on constrained devices, typically part of the Internet of Things. In this paper we leverage Information-Centric Networking (ICN), deployed within the domain of a network provider that interconnects, in addition to other terminals, CoAP endpoints in order to provide enhanced CoAP services. We present various CoAP-specific communication scenarios and discuss how ICN can provide benefits to both network providers and CoAP applications, even though the latter are not aware of the existence of ICN. In particular, the use of ICN results in smaller state management complexity at CoAP endpoints, simpler implementation at CoAP endpoints, and less communication overhead in the network.
The Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks Simultaneously
Venue: arXiv cs.AI 2017
Year: 2017

This paper introduces the Intentional Unintentional (IU) agent. This agent endows the deep deterministic policy gradients (DDPG) agent for continuous control with the ability to solve several tasks simultaneously. Learning to solve many tasks simultaneously has been a long-standing, core goal of artificial intelligence, inspired by infant development and motivated by the desire to build flexible robot manipulators capable of many diverse behaviours. We show that the IU agent not only learns to solve many tasks simultaneously but it also learns faster than agents that target a single task at-a-time. In some cases, where the single task DDPG method completely fails, the IU agent successfully solves the task. To demonstrate this, we build a playroom environment using the MuJoCo physics engine, and introduce a grounded formal language to automatically generate tasks.
Modification of Symmetric Cryptography with Combining Affine Chiper and Caesar Chiper which Dynamic Nature in Matrix of Chiper Transposition by Applying Flow Pattern in the Planting Rice
Venue: arXiv cs.CR 2017
Year: 2017

Classical cryptography is a way of disguising the news done by the people when there was no computer. The goal is to protect information by way of encoding. This paper describesa modification of classical algorithms to make cryptanalis difficult to steal undisclosed messages. There are three types of classical algorithms that are combined affine chiper, Caesar chiper and chiper transposition. Where for chiperteks affine chiper and Caesar chiper can be looped as much as the initial key, because the result can be varied as much as key value, then affine chiper and Caesar chiper in this case is dynamic. Then the results of the affine and Caesar will be combined in the transposition chiper matrix by applying the pattern of rice cultivation path and for chipertext retrieval by finally applying the pattern of rice planting path. And the final digit of the digit shown in the form of binary digits so that 5 characters can be changed to 80 digit bits are scrambled. Thus the cryptanalyst will be more difficult and takes a very long time to hack information that has been kept secret.
Automated Game Design Learning
Venue: arXiv cs.AI 2017
Year: 2017

While general game playing is an active field of research, the learning of game design has tended to be either a secondary goal of such research or it has been solely the domain of humans. We propose a field of research, Automated Game Design Learning (AGDL), with the direct purpose of learning game designs directly through interaction with games in the mode that most people experience games: via play. We detail existing work that touches the edges of this field, describe current successful projects in AGDL and the theoretical foundations that enable them, point to promising applications enabled by AGDL, and discuss next steps for this exciting area of study. The key moves of AGDL are to use game programs as the ultimate source of truth about their own design, and to make these design properties available to other systems and avenues of inquiry.
Fast Amortized Inference and Learning in Log-linear Models with Randomly Perturbed Nearest Neighbor Search
Venue: arXiv cs.LG 2017
Year: 2017

Inference in log-linear models scales linearly in the size of output space in the worst-case. This is often a bottleneck in natural language processing and computer vision tasks when the output space is feasibly enumerable but very large. We propose a method to perform inference in log-linear models with sublinear amortized cost. Our idea hinges on using Gumbel random variable perturbations and a pre-computed Maximum Inner Product Search data structure to access the most-likely elements in sublinear amortized time. Our method yields provable runtime and accuracy guarantees. Further, we present empirical experiments on ImageNet and Word Embeddings showing significant speedups for sampling, inference, and learning in log-linear models.
Detecting Policy Preferences and Dynamics in the UN General Debate with Neural Word Embeddings
Venue: arXiv cs.CL 2017
Year: 2017

Foreign policy analysis has been struggling to find ways to measure policy preferences and paradigm shifts in international political systems. This paper presents a novel, potential solution to this challenge, through the application of a neural word embedding (Word2vec) model on a dataset featuring speeches by heads of state or government in the United Nations General Debate. The paper provides three key contributions based on the output of the Word2vec model. First, it presents a set of policy attention indices, synthesizing the semantic proximity of political speeches to specific policy themes. Second, it introduces country-specific semantic centrality indices, based on topological analyses of countries' semantic positions with respect to each other. Third, it tests the hypothesis that there exists a statistical relation between the semantic content of political speeches and UN voting behavior, falsifying it and suggesting that political speeches contain information of different nature then the one behind voting outcomes. The paper concludes with a discussion of the practical use of its results and consequences for foreign policy analysis, public accountability, and transparency.
Fault-Induced Dynamics of Oblivious Robots on a Line
Venue: arXiv cs.DC 2017
Year: 2017

The study of computing in presence of faulty robots in the Look-Compute-Move model has been the object of extensive investigation, typically with the goal of designing algorithms tolerant to as many faults as possible. In this paper, we initiate a new line of investigation on the presence of faults, focusing on a rather different issue. We are interested in understanding the dynamics of a group of robots when they execute an algorithm designed for a fault-free environment, in presence of some undetectable crashed robots. We start this investigation focusing on the classic point-convergence algorithm by Ando et al. for robots with limited visibility, in a simple setting (which already presents serious challenges): the robots operate fully synchronously on a line, and at most two of them are faulty. Interestingly, and perhaps surprisingly, the presence of faults induces the robots to perform some form of scattering, rather than point-convergence. In fact, we discover that they arrange themselves inside the segment delimited by the two faults in interleaved sequences of equidistant robots. The structure that they form has a hierarchical nature: robots organize themselves in groups where a group of some level converges to an equidistant distribution only after all groups of lower levels have converged. This is the first study on the unintended dynamics of oblivious robots induced by the presence of faults.
Deep Learning for Sensor-based Activity Recognition: A Survey
Venue: arXiv cs.CV 2017
Year: 2017

Sensor-based activity recognition seeks the profound high-level knowledge about human activities from multitudes of low-level sensor readings. Conventional pattern recognition approaches have made tremendous progress in the past years. However, those methods often heavily rely on heuristic hand-crafted feature extraction, which could hinder their generalization performance. Additionally, existing methods are undermined for unsupervised and incremental learning tasks. Recently, the recent advancement of deep learning makes it possible to perform automatic high-level feature extraction thus achieves promising performance in many areas. Since then, deep learning based methods have been widely adopted for the sensor-based activity recognition tasks. This paper surveys the recent advance of deep learning based sensor-based activity recognition. We summarize existing literature from three aspects: sensor modality, deep model, and application. We also present detailed insights on existing work and propose grand challenges for future research.
VMEXT: A Visualization Tool for Mathematical Expression Trees
Venue: arXiv cs.HC 2017
Year: 2017

Mathematical expressions can be represented as a tree consisting of terminal symbols, such as identifiers or numbers (leaf nodes), and functions or operators (non-leaf nodes). Expression trees are an important mechanism for storing and processing mathematical expressions as well as the most frequently used visualization of the structure of mathematical expressions. Typically, researchers and practitioners manually visualize expression trees using general-purpose tools. This approach is laborious, redundant, and error-prone. Manual visualizations represent a user's notion of what the markup of an expression should be, but not necessarily what the actual markup is. This paper presents VMEXT - a free and open source tool to directly visualize expression trees from parallel MathML. VMEXT simultaneously visualizes the presentation elements and the semantic structure of mathematical expressions to enable users to quickly spot deficiencies in the Content MathML markup that does not affect the presentation of the expression. Identifying such discrepancies previously required reading the verbose and complex MathML markup. VMEXT also allows one to visualize similar and identical elements of two expressions. Visualizing expression similarity can support support developers in designing retrieval approaches and enable improved interaction concepts for users of mathematical information retrieval systems. We demonstrate VMEXT's visualizations in two web-based applications. The first application presents the visualizations alone. The second application shows a possible integration of the visualizations in systems for mathematical knowledge management and mathematical information retrieval. The application converts LaTeX input to parallel MathML, computes basic similarity measures for mathematical expressions, and visualizes the results using VMEXT.
Verifying Array Manipulating Programs by Tiling
Venue: arXiv cs.SE 2017
Year: 2017

Formally verifying properties of programs that manipulate arrays in loops is computationally challenging. In this paper, we focus on a useful class of such programs, and present a novel property-driven verification method that first infers array access patterns in loops using simple heuristics, and then uses this information to compositionally prove universally quantified assertions about arrays. Specifically, we identify tiles of array accesses patterns in a loop, and use the tiling information to reduce the problem of checking a quantified assertion at the end of a loop to an inductive argument that checks only a slice of the assertion for a single iteration of the loop body. We show that this method can be extended to programs with sequentially composed loops and nested loops as well. We have implemented our method in a tool called Tiler. Initial experiments show that Tiler outperforms several state-of-the-art tools on a suite of interesting benchmarks.
Cooperative Game Theory Approaches for Network Partitioning
Venue: arXiv cs.SI 2017
Year: 2017

The paper is devoted to game-theoretic methods for community detection in networks. The traditional methods for detecting community structure are based on selecting denser subgraphs inside the network. Here we propose to use the methods of cooperative game theory that highlight not only the link density but also the mechanisms of cluster formation. Specifically, we suggest two approaches from cooperative game theory: the first approach is based on the Myerson value, whereas the second approach is based on hedonic games. Both approaches allow to detect clusters with various resolution. However, the tuning of the resolution parameter in the hedonic games approach is particularly intuitive. Furthermore, the modularity based approach and its generalizations can be viewed as particular cases of the hedonic games.
A Bibliometric Model for Identifying Emerging Research Topics
Venue: arXiv cs.DL 2017
Year: 2017

Detecting emerging research topics is essential, not only for research agencies but also for individual researchers. Previous studies have created various bibliographic indicators for the identification of emerging research topics. However, as indicated by Rotolo et al. (2015), the most serious problems are the lack of an acknowledged definition of emergence and incomplete elaboration of the linkages between the definitions that are used and the indicators that are created. With these issues in mind, this study first adjusts the definition of an emerging technology that Rotolo et al. (2015) have proposed in order to accommodate the analysis. Next, a set of criteria for the identification of emerging topics is proposed according to the adjusted definition and attributes of emergence. By the use of two sets of parameter values, several emerging research topics are identified. Finally, evaluation tests are conducted by demonstration of the proposed approach and comparison with previous studies. The strength of the present methodology lies in the fact that it is fully transparent, straightforward, and flexible.
OMICtools: a community-driven search engine for biological data analysis
Venue: arXiv cs.IR 2017
Year: 2017

With high-throughput biotechnologies generating unprecedented quantities of data, researchers are faced with the challenge of locating and comparing an exponentially growing number of programs and websites dedicated to computational biology, in order to maximize the potential of their data. OMICtools is designed to meet this need with its open-access search engine offering an easy means of locating the right tools corresponding to each researcher and their specific biological data analyses. The OMICtools website (https://OMICtools.com) centralizes more than 18,500 software tools and databases, manually classified, by a team of biocurators including many scientific experts. Key information, a direct link, and access to discussions and evaluations by the biomedical community are provided for each tool. Anyone can join the OMICtools community and create a profile page, to share their expertise and comment on tools. In addition, developers can directly upload their code versions which are registered for identification and citation in scientific publications, improving research traceability. The OMICtools community links thousands of life scientists and developers worldwide, who use this bioinformatics platform to accelerate their projects and biological data analyses.