OpenRaiser/Intern-Atlas huggingface dataset:
BISMO: A Scalable Bit-Serial Matrix Multiplication Overlay for Reconfigurable Computing
Venue: arXiv cs.AR 2018
Year: 2018

Matrix-matrix multiplication is a key computational kernel for numerous applications in science and engineering, with ample parallelism and data locality that lends itself well to high-performance implementations. Many matrix multiplication-dependent applications can use reduced-precision integer or fixed-point representations to increase their performance and energy efficiency while still offering adequate quality of results. However, precision requirements may vary between different application phases or depend on input data, rendering constant-precision solutions ineffective. We present BISMO, a vectorized bit-serial matrix multiplication overlay for reconfigurable computing. BISMO utilizes the excellent binary-operation performance of FPGAs to offer a matrix multiplication performance that scales with required precision and parallelism. We characterize the resource usage and performance of BISMO across a range of parameters to build a hardware cost model, and demonstrate a peak performance of 6.5 TOPS on the Xilinx PYNQ-Z1 board.
Robust Navigation In GNSS Degraded Environment Using Graph Optimization
Venue: arXiv cs.RO 2018
Year: 2018

Robust navigation in urban environments has received a considerable amount of both academic and commercial interest over recent years. This is primarily due to large commercial organizations such as Google and Uber stepping into the autonomous navigation market. Most of this research has shied away from Global Navigation Satellite System (GNSS) based navigation. The aversion to utilizing GNSS data is due to the degraded nature of the data in urban environment (e.g., multipath, poor satellite visibility). The degradation of the GNSS data in urban environments makes it such that traditional (GNSS) positioning methods (e.g., extended Kalman filter, particle filters) perform poorly. However, recent advances in robust graph theoretic based sensor fusion methods, primarily applied to Simultaneous Localization and Mapping (SLAM) based robotic applications, can also be applied to GNSS data processing. This paper will utilize one such method known as the factor graph in conjunction several robust optimization techniques to evaluate their applicability to robust GNSS data processing. The goals of this study are two-fold. First, for GNSS applications, we will experimentally evaluate the effectiveness of robust optimization techniques within a graph-theoretic estimation framework. Second, by releasing the software developed and data sets used for this study, we will introduce a new open-source front-end to the Georgia Tech Smoothing and Mapping (GTSAM) library for the purpose of integrating GNSS pseudorange observations.
Defending Malware Classification Networks Against Adversarial Perturbations with Non-Negative Weight Restrictions
Venue: arXiv cs.LG 2018
Year: 2018

There is a growing body of literature showing that deep neural networks are vulnerable to adversarial input modification. Recently this work has been extended from image classification to malware classification over boolean features. In this paper we present several new methods for training restricted networks in this specific domain that are highly effective at preventing adversarial perturbations. We start with a fully adversarially resistant neural network that has hard non-negative weight restrictions and is equivalent to learning a monotonic boolean function and then attempt to relax the constraints to improve classifier accuracy.
In-situ Stochastic Training of MTJ Crossbar based Neural Networks
Venue: arXiv cs.NE 2018
Year: 2018

Owing to high device density, scalability and non-volatility, Magnetic Tunnel Junction-based crossbars have garnered significant interest for implementing the weights of an artificial neural network. The existence of only two stable states in MTJs implies a high overhead of obtaining optimal binary weights in software. We illustrate that the inherent parallelism in the crossbar structure makes it highly appropriate for in-situ training, wherein the network is taught directly on the hardware. It leads to significantly smaller training overhead as the training time is independent of the size of the network, while also circumventing the effects of alternate current paths in the crossbar and accounting for manufacturing variations in the device. We show how the stochastic switching characteristics of MTJs can be leveraged to perform probabilistic weight updates using the gradient descent algorithm. We describe how the update operations can be performed on crossbars both with and without access transistors and perform simulations on them to demonstrate the effectiveness of our techniques. The results reveal that stochastically trained MTJ-crossbar NNs achieve a classification accuracy nearly same as that of real-valued-weight networks trained in software and exhibit immunity to device variations.
Disentangled VAE Representations for Multi-Aspect and Missing Data
Venue: arXiv cs.LG 2018
Year: 2018

Many problems in machine learning and related application areas are fundamentally variants of conditional modeling and sampling across multi-aspect data, either multi-view, multi-modal, or simply multi-group. For example, sampling from the distribution of English sentences conditioned on a given French sentence or sampling audio waveforms conditioned on a given piece of text. Central to many of these problems is the issue of missing data: we can observe many English, French, or German sentences individually but only occasionally do we have data for a sentence pair. Motivated by these applications and inspired by recent progress in variational autoencoders for grouped data, we develop factVAE, a deep generative model capable of handling multi-aspect data, robust to missing observations, and with a prior that encourages disentanglement between the groups and the latent dimensions. The effectiveness of factVAE is demonstrated on a variety of rich real-world datasets, including motion capture poses and pictures of faces captured from varying poses and perspectives.
Distributed Edge Caching in Ultra-dense Fog Radio Access Networks: A Mean Field Approach
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, the edge caching problem in ultra-dense fog radio access networks (F-RAN) is investigated. Taking into account time-variant user requests and ultra-dense deployment of fog access points (F-APs), we propose a dynamic distributed edge caching scheme to jointly minimize the request service delay and fronthaul traffic load. Considering the interactive relationship among F-APs, we model the caching optimization problem as a stochastic differential game (SDG) which captures the temporal dynamics of F-AP states and incorporates user requests status. The SDG is further approximated as a mean field game (MFG) by exploiting the ultra-dense property of F-RAN. In the MFG, each F-AP can optimize its caching policy independently through iteratively solving the corresponding partial differential equations without any information exchange with other F-APs. The simulation results show that the proposed edge caching scheme outperforms the baseline schemes under both static and time-variant user requests.
CNN-based Action Recognition and Supervised Domain Adaptation on 3D Body Skeletons via Kernel Feature Maps
Venue: arXiv cs.CV 2018
Year: 2018

Deep learning is ubiquitous across many areas areas of computer vision. It often requires large scale datasets for training before being fine-tuned on small-to-medium scale problems. Activity, or, in other words, action recognition, is one of many application areas of deep learning. While there exist many Convolutional Neural Network architectures that work with the RGB and optical flow frames, training on the time sequences of 3D body skeleton joints is often performed via recurrent networks such as LSTM. In this paper, we propose a new representation which encodes sequences of 3D body skeleton joints in texture-like representations derived from mathematically rigorous kernel methods. Such a representation becomes the first layer in a standard CNN network e.g., ResNet-50, which is then used in the supervised domain adaptation pipeline to transfer information from the source to target dataset. This lets us leverage the available Kinect-based data beyond training on a single dataset and outperform simple fine-tuning on any two datasets combined in a naive manner. More specifically, in this paper we utilize the overlapping classes between datasets. We associate datapoints of the same class via so-called commonality, known from the supervised domain adaptation. We demonstrate state-of-the-art results on three publicly available benchmarks.
Ethical Implications of Social Internet of Vehicles Systems
Venue: arXiv cs.CY 2018
Year: 2018

The core concept of IoT is to equip real world objects with computing, processing and communicating capabilities to enable socializing between them. Internet of Vehicles (IoV) is an adherent of IoT that has realized significant advancements using communication technologies. Vehicles connected through Internet are capable of sharing information that can substantially enhance the quality of traffic on roads. Social Internet of Things (SIoT) is an instance of IoT that deals specifically in socialization of connected objects. SIoT enables the notion of Social Internet of Vehicles (SIoV) where vehicles are the key entities for sharing information between themselves and the infrastructure (commonly known as Road Side Units (RSUs)). Vehicles in SIoV socialize by exchanging data such as traffic congestions, weather conditions, infotainment, vacant parking slots, alternate routes and discount coupons for restaurants etc. In SIoV, vehicles can communicate with other vehicles and infrastructure through traditional communication technologies like Wi-Fi, Cellular networks or through Dedicated Short Range Communication (DSRC) etc. SIoV will be confronted with ethical dilemmas and expected to function in an ethically responsible manner. This paper highlights the ethical implications of SIoV systems. Vehicle to Vehicle (V2V) and Vehicle to Infrastructure (V2I) involves autonomous decision making that requires setting ethical and moral rules before taking verdict. The article discusses the lack of ethical guidelines in designing and deploying of SIoV systems that are of utmost importance. Finally, an addition to SIoV architecture is proposed to incorporate the ethical and moral principles for scheming the SIoV systems.
Artwork Identification from Wearable Camera Images for Enhancing Experience of Museum Audiences
Venue: arXiv cs.CV 2018
Year: 2018

Recommendation systems based on image recognition could prove a vital tool in enhancing the experience of museum audiences. However, for practical systems utilizing wearable cameras, a number of challenges exist which affect the quality of image recognition. In this pilot study, we focus on recognition of museum collections by using a wearable camera in three different museum spaces. We discuss the application of wearable cameras, and the practical and technical challenges in devising a robust system that can recognize artworks viewed by the visitors to create a detailed record of their visit. Specifically, to illustrate the impact of different kinds of museum spaces on image recognition, we collect three training datasets of museum exhibits containing variety of paintings, clocks, and sculptures. Subsequently, we equip selected visitors with wearable cameras to capture artworks viewed by them as they stroll along exhibitions. We use Convolutional Neural Networks (CNN) which are pre-trained on the ImageNet dataset and fine-tuned on each of the training sets for the purpose of artwork identification. In the testing stage, we use CNNs to identify artworks captured by the visitors with a wearable camera. We analyze the accuracy of their recognition and provide an insight into the applicability of such a system to further engage audiences with museum exhibitions.
Graph-based Cooperative Caching in Fog-RAN
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, the cooperative caching problem in fog radio access networks (F-RAN) is investigated. To maximize the incremental offloaded traffic, we formulate the clustering optimization problem with the consideration of cooperative caching and local content popularity, which falls into the scope of combinatorial programming. % and is NP-hard. We then propose an effective graph-based approach to solve this challenging problem. Firstly, a node graph is constructed with its vertex set representing the considered fog access points (F-APs) and its edge set reflecting the potential cooperations among the F-APs. %whether the F-APs the distance and load difference among the F-APs. Then, by exploiting the adjacency table of each vertex of the node graph, we propose to get the complete subgraphs through indirect searching for the maximal complete subgraphs for the sake of a reduced searching complexity. Furthermore, by using the complete subgraphs so obtained, a weighted graph is constructed. By setting the weights of the vertices of the weighted graph to be the incremental offloaded traffics of their corresponding complete subgraphs, the original clustering optimization problem can be transformed into an equivalent 0-1 integer programming problem. The max-weight independent subset of the vertex set of the weighted graph, which is equivalent to the objective cluster sets, can then be readily obtained by solving the above optimization problem through the greedy algorithm that we propose. Our proposed graph-based approach has an apparently low complexity in comparison with the brute force approach which has an exponential complexity. Simulation results show the remarkable improvements in terms of offloading gain by using our proposed approach.
Blockchain Technologies for the Internet of Things: Research Issues and Challenges
Venue: arXiv cs.CR 2018
Year: 2018

This paper presents a comprehensive survey of the existing blockchain protocols for the Internet of Things (IoT) networks. We start by describing the blockchains and summarizing the existing surveys that deal with blockchain technologies. Then, we provide an overview of the application domains of blockchain technologies in IoT, e.g, Internet of Vehicles, Internet of Energy, Internet of Cloud, Fog computing, etc. Moreover, we provide a classification of threat models, which are considered by blockchain protocols in IoT networks, into five main categories, namely, identity-based attacks, manipulation-based attacks, cryptanalytic attacks, reputation-based attacks, and service-based attacks. In addition, we provide a taxonomy and a side-by-side comparison of the state-of-the-art methods towards secure and privacy-preserving blockchain technologies with respect to the blockchain model, specific security goals, performance, limitations, computation complexity, and communication overhead. Based on the current survey, we highlight open research challenges and discuss possible future research directions in the blockchain technologies for IoT.
FBI-Pose: Towards Bridging the Gap between 2D Images and 3D Human Poses using Forward-or-Backward Information
Venue: arXiv cs.CV 2018
Year: 2018

Although significant advances have been made in the area of human poses estimation from images using deep Convolutional Neural Network (ConvNet), it remains a big challenge to perform 3D pose inference in-the-wild. This is due to the difficulty to obtain 3D pose groundtruth for outdoor environments. In this paper, we propose a novel framework to tackle this problem by exploiting the information of each bone indicating if it is forward or backward with respect to the view of the camera(we term it Forwardor-Backward Information abbreviated as FBI). Our method firstly trains a ConvNet with two branches which maps an image of a human to both the 2D joint locations and the FBI of bones. These information is further fed into a deep regression network to predict the 3D positions of joints. To support the training, we also develop an annotation user interface and labeled such FBI for around 12K in-the-wild images which are randomly selected from MPII (a public dataset of 2D pose annotation). Our experimental results on the standard benchmarks demonstrate that our approach outperforms state-of-the-art methods both qualitatively and quantitatively.
Best Vision Technologies Submission to ActivityNet Challenge 2018-Task: Dense-Captioning Events in Videos
Venue: arXiv cs.CV 2018
Year: 2018

This note describes the details of our solution to the dense-captioning events in videos task of ActivityNet Challenge 2018. Specifically, we solve this problem with a two-stage way, i.e., first temporal event proposal and then sentence generation. For temporal event proposal, we directly leverage the three-stage workflow in [13, 16]. For sentence generation, we capitalize on LSTM-based captioning framework with temporal attention mechanism (dubbed as LSTM-T). Moreover, the input visual sequence to the LSTM-based video captioning model is comprised of RGB and optical flow images. At inference, we adopt a late fusion scheme to fuse the two LSTM-based captioning models for sentence generation.
RAM: A Region-Aware Deep Model for Vehicle Re-Identification
Venue: arXiv cs.CV 2018
Year: 2018

Previous works on vehicle Re-ID mainly focus on extracting global features and learning distance metrics. Because some vehicles commonly share same model and maker, it is hard to distinguish them based on their global appearances. Compared with the global appearance, local regions such as decorations and inspection stickers attached to the windshield, may be more distinctive for vehicle Re-ID. To embed the detailed visual cues in those local regions, we propose a Region-Aware deep Model (RAM). Specifically, in addition to extracting global features, RAM also extracts features from a series of local regions. As each local region conveys more distinctive visual cues, RAM encourages the deep model to learn discriminative features. We also introduce a novel learning algorithm to jointly use vehicle IDs, types/models, and colors to train the RAM. This strategy fuses more cues for training and results in more discriminative global and regional features. We evaluate our methods on two large-scale vehicle Re-ID datasets, i.e., VeRi and VehicleID. Experimental results show our methods achieve promising performance in comparison with recent works.
Improving Chemical Autoencoder Latent Space and Molecular De novo Generation Diversity with Heteroencoders
Venue: arXiv cs.LG 2018
Year: 2018

Chemical autoencoders are attractive models as they combine chemical space navigation with possibilities for de-novo molecule generation in areas of interest. This enables them to produce focused chemical libraries around a single lead compound for employment early in a drug discovery project. Here it is shown that the choice of chemical representation, such as SMILES strings, has a large influence on the properties of the latent space. It is further explored to what extent translating between different chemical representations influences the latent space similarity to the SMILES strings or circular fingerprints. By employing SMILES enumeration for either the encoder or decoder, it is found that the decoder has the largest influence on the properties of the latent space. Training a sequence to sequence heteroencoder based on recurrent neural networks(RNNs) with long short-term memory cells (LSTM) to predict different enumerated SMILES strings from the same canonical SMILES string gives the largest similarity between latent space distance and molecular similarity measured as circular fingerprints similarity. Using the output from the bottleneck in QSAR modelling of five molecular datasets shows that heteroencoder derived vectors markedly outperforms autoencoder derived vectors as well as models built using ECFP4 fingerprints, underlining the increased chemical relevance of the latent space. However, the use of enumeration during training of the decoder leads to a markedly increase in the rate of decoding to a different molecules than encoded, a tendency that can be counteracted with more complex network architectures.
Evaluation of Information Retrieval Systems Using Structural Equation Modelling
Venue: arXiv cs.IR 2018
Year: 2018

The interpretation of the experimental data collected by testing systems across input datasets and model parameters is of strategic importance for system design and implementation. In particular, finding relationships between variables and detecting the latent variables affecting retrieval performance can provide designers, engineers and experimenters with useful if not necessary information about how a system is performing. This paper discusses the use of Structural Equation Modelling (SEM) in providing an in-depth explanation of evaluation results and an explanation of failures and successes of a system; in particular, we focus on the case of Information Retrieval.
Sum-of-Squares meets Nash: Optimal Lower Bounds for Finding any Equilibrium
Venue: arXiv cs.CC 2018
Year: 2018

Several works have shown unconditional hardness (via integrality gaps) of computing equilibria using strong hierarchies of convex relaxations. Such results however only apply to the problem of computing equilibria that optimize a certain objective function and not to the (arguably more fundamental) task of finding \emph{any} equilibrium. We present an algorithmic model based on the sum-of-squares (SoS) hierarchy that allows escaping this inherent limitation of integrality gaps. In this model, algorithms access the input game only through a relaxed solution to the natural SoS relaxation for computing equilibria. They can then adaptively construct a list of candidate solutions and invoke a verification oracle to check if any candidate on the list is a solution. This model captures most well-studied approximation algorithms such as those for Max-Cut, Sparsest Cut, and Unique-Games. The state-of-the-art algorithms for computing exact and approximate equilibria in two-player, n-strategy games are captured in this model and require that at least one of i) size (~ running time) of the SoS relaxation or ii) the size of the list of candidates, be at least $2^{Ω(n)}$ and $n^{Ω(\log{(n)})}$ respectively. Our main result shows a lower bound that matches these upper bound up to constant factors in the exponent. This can be interpreted as an unconditional confirmation, in our restricted algorithmic framework, of Rubinstein's recent conditional hardness \cite{Rub} for computing approximate equilibria. Our proof strategy involves constructing a family of games that all share a common sum-of-squares solution but every (approximate) equilibrium of one game is far from every (approximate) equilibrium of any other game in the family.
Pushing the boundaries of parallel Deep Learning -- A practical approach
Venue: arXiv cs.DC 2018
Year: 2018

This work aims to assess the state of the art of data parallel deep neural network training, trying to identify potential research tracks to be exploited for performance improvement. Beside, it presents a design for a practical C++ library dedicated at implementing and unifying the current state of the art methodologies for parallel training in a performance-conscious framework, allowing the user to explore novel strategies without departing significantly from its usual work-flow.