OpenRaiser/Intern-Atlas huggingface dataset:
2nd Place Solution to the GQA Challenge 2019
Venue: arXiv cs.CV 2019
Year: 2019

We present a simple method that achieves unexpectedly superior performance for Complex Reasoning involved Visual Question Answering. Our solution collects statistical features from high-frequency words of all the questions asked about an image and use them as accurate knowledge for answering further questions of the same image. We are fully aware that this setting is not ubiquitously applicable, and in a more common setting one should assume the questions are asked separately and they cannot be gathered to obtain a knowledge base. Nonetheless, we use this method as an evidence to demonstrate our observation that the bottleneck effect is more severe on the feature extraction part than it is on the knowledge reasoning part. We show significant gaps when using the same reasoning model with 1) ground-truth features; 2) statistical features; 3) detected features from completely learned detectors, and analyze what these gaps mean to researches on visual reasoning topics. Our model with the statistical features achieves the 2nd place in the GQA Challenge 2019.
Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

Improving the accuracy and robustness of deep neural nets (DNNs) and adapting them to small training data are primary tasks in deep learning research. In this paper, we replace the output activation function of DNNs, typically the data-agnostic softmax function, with a graph Laplacian-based high dimensional interpolating function which, in the continuum limit, converges to the solution of a Laplace-Beltrami equation on a high dimensional manifold. Furthermore, we propose end-to-end training and testing algorithms for this new architecture. The proposed DNN with graph interpolating activation integrates the advantages of both deep learning and manifold learning. Compared to the conventional DNNs with the softmax function as output activation, the new framework demonstrates the following major advantages: First, it is better applicable to data-efficient learning in which we train high capacity DNNs without using a large number of training data. Second, it remarkably improves both natural accuracy on the clean images and robust accuracy on the adversarial images crafted by both white-box and black-box adversarial attacks. Third, it is a natural choice for semi-supervised learning. For reproducibility, the code is available at \url{https://github.com/BaoWangMath/DNN-DataDependentActivation}.
Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving
Venue: arXiv cs.CR 2019
Year: 2019

In Autonomous Vehicles (AVs), one fundamental pillar is perception, which leverages sensors like cameras and LiDARs (Light Detection and Ranging) to understand the driving environment. Due to its direct impact on road safety, multiple prior efforts have been made to study its the security of perception systems. In contrast to prior work that concentrates on camera-based perception, in this work we perform the first security study of LiDAR-based perception in AV settings, which is highly important but unexplored. We consider LiDAR spoofing attacks as the threat model and set the attack goal as spoofing obstacles close to the front of a victim AV. We find that blindly applying LiDAR spoofing is insufficient to achieve this goal due to the machine learning-based object detection process. Thus, we then explore the possibility of strategically controlling the spoofed attack to fool the machine learning model. We formulate this task as an optimization problem and design modeling methods for the input perturbation function and the objective function. We also identify the inherent limitations of directly solving the problem using optimization and design an algorithm that combines optimization and global sampling, which improves the attack success rates to around 75%. As a case study to understand the attack impact at the AV driving decision level, we construct and evaluate two attack scenarios that may damage road safety and mobility. We also discuss defense directions at the AV system, sensor, and machine learning model levels.
Deep Reinforcement Learning Based Robot Arm Manipulation with Efficient Training Data through Simulation
Venue: arXiv cs.RO 2019
Year: 2019

Deep reinforcement learning trains neural networks using experiences sampled from the replay buffer, which is commonly updated at each time step. In this paper, we propose a method to update the replay buffer adaptively and selectively to train a robot arm to accomplish a suction task in simulation. The response time of the agent is thoroughly taken into account. The state transitions that remain stuck at the boundary of constraint are not stored. The policy trained with our method works better than the one with the common replay buffer update method. The result is demonstrated both by simulation and by experiment with a real robot arm.
Single-bit-per-weight deep convolutional neural networks without batch-normalization layers for embedded systems
Venue: arXiv cs.LG 2019
Year: 2019

Batch-normalization (BN) layers are thought to be an integrally important layer type in today's state-of-the-art deep convolutional neural networks for computer vision tasks such as classification and detection. However, BN layers introduce complexity and computational overheads that are highly undesirable for training and/or inference on low-power custom hardware implementations of real-time embedded vision systems such as UAVs, robots and Internet of Things (IoT) devices. They are also problematic when batch sizes need to be very small during training, and innovations such as residual connections introduced more recently than BN layers could potentially have lessened their impact. In this paper we aim to quantify the benefits BN layers offer in image classification networks, in comparison with alternative choices. In particular, we study networks that use shifted-ReLU layers instead of BN layers. We found, following experiments with wide residual networks applied to the ImageNet, CIFAR 10 and CIFAR 100 image classification datasets, that BN layers do not consistently offer a significant advantage. We found that the accuracy margin offered by BN layers depends on the data set, the network size, and the bit-depth of weights. We conclude that in situations where BN layers are undesirable due to speed, memory or complexity costs, that using shifted-ReLU layers instead should be considered; we found they can offer advantages in all these areas, and often do not impose a significant accuracy cost.
The Bregman-Tweedie Classification Model
Venue: arXiv cs.LG 2019
Year: 2019

This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling parameter, and related high-level mathematical structures, such as the Bregman-Tweedie loss function and the Bregman-Tweedie divergence. The base function of this divergence is the convex function of Legendre type induced from the extended exponential function. The Bregman-Tweedie loss function of the proposed classification model is the regular Legendre transformation of the Bregman-Tweedie divergence. This loss function is a polynomial parameterized function between unhinge loss and the logistic loss function. Actually, we have two sub-models of the Bregman-Tweedie classification model; H-Bregman with hinge-like loss function and L-Bregman with logistic-like loss function. Although the proposed classification model is nonconvex and unbounded, empirically, we have observed that the H-Bregman and L-Bregman outperform, in terms of the Friedman ranking, logistic regression and SVM and show reasonable performance in terms of the classification accuracy in the category of the binary linear classification problem.
Assessing Refugees' Integration via Spatio-temporal Similarities of Mobility and Calling Behaviors
Venue: arXiv cs.CY 2019
Year: 2019

In Turkey the increasing tension, due to the presence of 3.4 million Syrian refugees, demands the formulation of effective integration policies. Moreover, their design requires tools aimed at understanding the integration of refugees despite the complexity of this phenomenon. In this work, we propose a set of metrics aimed at providing insights and assessing the integration of Syrians refugees, by analyzing a real-world Call Details Records (CDRs) dataset including calls from refugees and locals in Turkey throughout 2017. Specifically, we exploit the similarity between refugees' and locals' spatial and temporal behaviors, in terms of communication and mobility in order to assess integration dynamics. Together with the already known methods for data analysis, we use a novel computational approach to analyze spatiotemporal patterns: Computational Stigmergy, a bio-inspired scalar and temporal aggregation of samples. Computational Stigmergy associates each sample to a virtual pheromone deposit (mark). Marks in spatiotemporal proximity are aggregated into functional structures called trails, which summarize the spatiotemporal patterns in data and allows computing the similarity between different patterns. According to our results, collective mobility and behavioral similarity with locals have great potential as measures of integration, since they are: (i) correlated with the amount of interaction with locals; (ii) an effective proxy for refugee's economic capacity, thus refugee's potential employment; and (iii) able to capture events that may disrupt the integration phenomena, such as social tensions.
A Unified Deep Framework for Joint 3D Pose Estimation and Action Recognition from a Single RGB Camera
Venue: arXiv cs.CV 2019
Year: 2019

We present a deep learning-based multitask framework for joint 3D human pose estimation and action recognition from RGB video sequences. Our approach proceeds along two stages. In the first, we run a real-time 2D pose detector to determine the precise pixel location of important keypoints of the body. A two-stream neural network is then designed and trained to map detected 2D keypoints into 3D poses. In the second, we deploy the Efficient Neural Architecture Search (ENAS) algorithm to find an optimal network architecture that is used for modeling the spatio-temporal evolution of the estimated 3D poses via an image-based intermediate representation and performing action recognition. Experiments on Human3.6M, MSR Action3D and SBU Kinect Interaction datasets verify the effectiveness of the proposed method on the targeted tasks. Moreover, we show that our method requires a low computational budget for training and inference.
Speed estimation evaluation on the KITTI benchmark based on motion and monocular depth information
Venue: arXiv cs.CV 2019
Year: 2019

In this technical report we investigate speed estimation of the ego-vehicle on the KITTI benchmark using state-of-the-art deep neural network based optical flow and single-view depth prediction methods. Using a straightforward intuitive approach and approximating a single scale factor, we evaluate several application schemes of the deep networks and formulate meaningful conclusions such as: combining depth information with optical flow improves speed estimation accuracy as opposed to using optical flow alone; the quality of the deep neural network methods influences speed estimation performance; using the depth and optical flow results from smaller crops of wide images degrades performance. With these observations in mind, we achieve a RMSE of less than 1 m/s for vehicle speed estimation using monocular images as input from recordings of the KITTI benchmark. Limitations and possible future directions are discussed as well.
Data Selection for training Semantic Segmentation CNNs with cross-dataset weak supervision
Venue: arXiv cs.CV 2019
Year: 2019

Training convolutional networks for semantic segmentation with strong (per-pixel) and weak (per-bounding-box) supervision requires a large amount of weakly labeled data. We propose two methods for selecting the most relevant data with weak supervision. The first method is designed for finding visually similar images without the need of labels and is based on modeling image representations with a Gaussian Mixture Model (GMM). As a byproduct of GMM modeling, we present useful insights on characterizing the data generating distribution. The second method aims at finding images with high object diversity and requires only the bounding box labels. Both methods are developed in the context of automated driving and experimentation is conducted on Cityscapes and Open Images datasets. We demonstrate performance gains by reducing the amount of employed weakly labeled images up to 100 times for Open Images and up to 20 times for Cityscapes.
Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Training deep convolutional neural networks usually requires a large amount of labeled data. However, it is expensive and time-consuming to annotate data for medical image segmentation tasks. In this paper, we present a novel uncertainty-aware semi-supervised framework for left atrium segmentation from 3D MR images. Our framework can effectively leverage the unlabeled data by encouraging consistent predictions of the same input under different perturbations. Concretely, the framework consists of a student model and a teacher model, and the student model learns from the teacher model by minimizing a segmentation loss and a consistency loss with respect to the targets of the teacher model. We design a novel uncertainty-aware scheme to enable the student model to gradually learn from the meaningful and reliable targets by exploiting the uncertainty information. Experiments show that our method achieves high performance gains by incorporating the unlabeled data. Our method outperforms the state-of-the-art semi-supervised methods, demonstrating the potential of our framework for the challenging semi-supervised problems.
Blockchain Mutability: Challenges and Proposed Solutions
Venue: arXiv cs.CR 2019
Year: 2019

Blockchain's evolution during the past decade is astonishing: from bitcoin to over 2.000 altcoins, and from decentralised electronic payments to transactions programmable by smart contracts and complex tokens governed by decentralised organisations. While the new generation of blockchain applications is still evolving, blockchain's technical characteristics are also advancing. Yet, immutability, a hitherto indisputable property according to which blockchain data cannot be edited nor deleted, remains the cornerstone of blockchain's security. Nevertheless, blockchain's immutability is being called into question lately in the light of the new erasing requirements imposed by the GDPR's ``\textit{Right to be Forgotten (RtbF)}'' provision. As the RtbF obliges blockchain data to be editable in order restricted content redactions, modifications or deletions to be applied when requested, blockchains compliance with the regulation is indeed challenging, if not impracticable. Towards resolving this contradiction, various methods and techniques for mutable blockchains have been proposed in an effort to satisfy regulatory erasing requirements while preserving blockchains' security. To this end, this work aims to provide a comprehensive review on the state-of-the-art research approaches, technical workarounds and advanced cryptographic techniques that have been put forward to resolve this conflict and to discuss their potentials, constraints and limitations when applied in the wild to either permissioned or permissionless blockchains.
Topology Based Scalable Graph Kernels
Venue: arXiv cs.LG 2019
Year: 2019

We propose a new graph kernel for graph classification and comparison using Ollivier Ricci curvature. The Ricci curvature of an edge in a graph describes the connectivity in the local neighborhood. An edge in a densely connected neighborhood has positive curvature and an edge serving as a local bridge has negative curvature. We use the edge curvature distribution to form a graph kernel which is then used to compare and cluster graphs. The curvature kernel uses purely the graph topology and thereby works for settings when node attributes are not available.
On the Performance of Renewable Energy-Powered UAV-Assisted Wireless Communications
Venue: arXiv cs.NI 2019
Year: 2019

We develop novel statistical models of the harvested energy from renewable energy sources (such as solar and wind energy) considering harvest-store-consume (HSC) architecture. We consider three renewable energy harvesting scenarios, i.e. (i) harvesting from the solar power, (ii) harvesting from the wind power, and (iii) hybrid solar and wind power. In this context, we first derive the closed-form expressions for the probability density function (PDF) and cumulative density function (CDF) of the harvested power from the solar and wind energy sources. Based on the derived expressions, we calculate the probability of energy outage at UAVs and signal-to-noise ratio (SNR) outage at ground cellular users. We derive novel closed-form expressions for the moment generating function (MGF) of the harvested solar power and wind power. Then, we apply Gil-Pelaez inversion to evaluate the energy outage at the UAV and signal-to-noise-ratio (SNR) outage at the ground users. We formulate the SNR outage minimization problem and obtain closed-form solutions for the transmit power and flight time of the UAV. In addition, we derive novel closed-form expressions for the moments of the solar power and wind power and demonstrate their applications in computing novel performance metrics considering the stochastic nature of the amount of harvested energy as well as energy arrival time. These performance metrics include the probability of charging the UAV battery within the flight time, average UAV battery charging time, probability of energy outage at UAVs, and the probability of eventual energy outage (i.e. the probability of energy outage in a finite duration of time) at UAVs.
Mediation Challenges and Socio-Technical Gaps for Explainable Deep Learning Applications
Venue: arXiv cs.AI 2019
Year: 2019

The presumed data owners' right to explanations brought about by the General Data Protection Regulation in Europe has shed light on the social challenges of explainable artificial intelligence (XAI). In this paper, we present a case study with Deep Learning (DL) experts from a research and development laboratory focused on the delivery of industrial-strength AI technologies. Our aim was to investigate the social meaning (i.e. meaning to others) that DL experts assign to what they do, given a richly contextualized and familiar domain of application. Using qualitative research techniques to collect and analyze empirical data, our study has shown that participating DL experts did not spontaneously engage into considerations about the social meaning of machine learning models that they build. Moreover, when explicitly stimulated to do so, these experts expressed expectations that, with real-world DL application, there will be available mediators to bridge the gap between technical meanings that drive DL work, and social meanings that AI technology users assign to it. We concluded that current research incentives and values guiding the participants' scientific interests and conduct are at odds with those required to face some of the scientific challenges involved in advancing XAI, and thus responding to the alleged data owners' right to explanations or similar societal demands emerging from current debates. As a concrete contribution to mitigate what seems to be a more general problem, we propose three preliminary XAI Mediation Challenges with the potential to bring together technical and social meanings of DL applications, as well as to foster much needed interdisciplinary collaboration among AI and the Social Sciences researchers.
RayTracer.jl: A Differentiable Renderer that supports Parameter Optimization for Scene Reconstruction
Venue: arXiv cs.GR 2019
Year: 2019

In this paper, we present RayTracer.jl, a renderer in Julia that is fully differentiable using source-to-source Automatic Differentiation (AD). This means that RayTracer not only renders 2D images from 3D scene parameters, but it can be used to optimize for model parameters that generate a target image in a Differentiable Programming (DP) pipeline. We interface our renderer with the deep learning library Flux for use in combination with neural networks. We demonstrate the use of this differentiable renderer in rendering tasks and in solving inverse graphics problems.
Scene Motion Decomposition for Learnable Visual Odometry
Venue: arXiv cs.CV 2019
Year: 2019

Optical Flow (OF) and depth are commonly used for visual odometry since they provide sufficient information about camera ego-motion in a rigid scene. We reformulate the problem of ego-motion estimation as a problem of motion estimation of a 3D-scene with respect to a static camera. The entire scene motion can be represented as a combination of motions of its visible points. Using OF and depth we estimate a motion of each point in terms of 6DoF and represent results in the form of motion maps, each one addressing single degree of freedom. In this work we provide motion maps as inputs to a deep neural network that predicts 6DoF of scene motion. Through our evaluation on outdoor and indoor datasets we show that utilizing motion maps leads to accuracy improvement in comparison with naive stacking of depth and OF. Another contribution of our work is a novel network architecture that efficiently exploits motion maps and outperforms learnable RGB/RGB-D baselines.
Optimizing method for Neural Network based on Genetic Random Weight Change Learning Algorithm
Venue: arXiv cs.NE 2019
Year: 2019

Random weight change (RWC) algorithm is extremely component and robust for the hardware implementation of neural networks. RWC and Genetic algorithm (GA) are well known methodologies used for optimizing and learning the neural network (NN). Individually, each of these two algorithms has its strength and weakness along with separate objectives. However, recently, researchers combine these two algorithms for better learning and optimization of NN. In this paper, we proposed a methodology by combining the RWC and GA, namely Genetic Random Weight Change (GRWC), as well as demonstrate a seminal way to reduce the complexity of the neural network by removing weak weights of GRWC. In contrast to RWC and GA, GRWC contains an effective optimization procedure which is worthy at exploring a large and complex space in intellectual strategies influenced by the GA/RWC synergy. The learning behavior of the proposed algorithm was tested on MNIST dataset and it was able to prove its performance.
A New Distribution Version of Boneh-Goh-Nissim Cryptosystem: Security and performance analysis
Venue: arXiv cs.CR 2019
Year: 2019

The aim of this paper is to provide two distributed versions of the Boneh-Goh-Nissim Cryptosystem (BGNC). We give a proof of the semantic security for the first one. This guaranties that our algorithm is semantically secure in the contest of active non-adaptive adversaries. Furthermore, we prove that the second version of our distributed scheme is computationally more efficient than the ElGamal destributed elliptic curve cryptosystem (EDECC) and secure under the Subgroup Decision problem (SDP) assumption.