OpenRaiser/Intern-Atlas huggingface dataset:
Improving Image-Based Localization with Deep Learning: The Impact of the Loss Function
Venue: arXiv cs.CV 2019
Year: 2019

This work investigates the impact of the loss function on the performance of Neural Networks, in the context of a monocular, RGB-only, image localization task. A common technique used when regressing a camera's pose from an image is to formulate the loss as a linear combination of positional and rotational mean squared error (using tuned hyperparameters as coefficients). In this work we observe that changes to rotation and position mutually affect the captured image, and in order to improve performance, a pose regression network's loss function should include a term which combines the error of both of these coupled quantities. Based on task specific observations and experimental tuning, we present said loss term, and create a new model by appending this loss term to the loss function of the pre-existing pose regression network `PoseNet'. We achieve improvements in the localization accuracy of the network for indoor scenes; with decreases of up to 26.7% and 24.0% in the median positional and rotational error respectively, when compared to the default PoseNet.
OpenEDS: Open Eye Dataset
Venue: arXiv cs.CV 2019
Year: 2019

We present a large scale data set, OpenEDS: Open Eye Dataset, of eye-images captured using a virtual-reality (VR) head mounted display mounted with two synchronized eyefacing cameras at a frame rate of 200 Hz under controlled illumination. This dataset is compiled from video capture of the eye-region collected from 152 individual participants and is divided into four subsets: (i) 12,759 images with pixel-level annotations for key eye-regions: iris, pupil and sclera (ii) 252,690 unlabelled eye-images, (iii) 91,200 frames from randomly selected video sequence of 1.5 seconds in duration and (iv) 143 pairs of left and right point cloud data compiled from corneal topography of eye regions collected from a subset, 143 out of 152, participants in the study. A baseline experiment has been evaluated on OpenEDS for the task of semantic segmentation of pupil, iris, sclera and background, with the mean intersectionover-union (mIoU) of 98.3 %. We anticipate that OpenEDS will create opportunities to researchers in the eye tracking community and the broader machine learning and computer vision community to advance the state of eye-tracking for VR applications. The dataset is available for download upon request at https://research.fb.com/programs/openeds-challenge
Deep Learning for TDD and FDD Massive MIMO: Mapping Channels in Space and Frequency
Venue: arXiv cs.IT 2019
Year: 2019

Can we map the channels at one set of antennas and one frequency band to the channels at another set of antennas---possibly at a different location and a different frequency band? If this channel-to-channel mapping is possible, we can expect dramatic gains for massive MIMO systems. For example, in FDD massive MIMO, the uplink channels can be mapped to the downlink channels or the downlink channels at one subset of antennas can be mapped to the downlink channels at all the other antennas. This can significantly reduce (or even eliminate) the downlink training/feedback overhead. In the context of cell-free/distributed massive MIMO systems, this channel mapping can be leveraged to reduce the fronthaul signaling overhead as only the channels at a subset of the distributed terminals need to be fed to the central unit which can map them to the channels at all the other terminals. This mapping can also find interesting applications in mmWave beam prediction, MIMO radar, and massive MIMO based positioning. In this paper, we introduce the new concept of channel mapping in space and frequency, where the channels at one set of antennas and one frequency band are mapped to the channels at another set of antennas and frequency band. First, we prove that this channel-to-channel mapping function exists under the condition that the mapping from the candidate user positions to the channels at the first set of antennas is bijective; a condition that can be achieved with high probability in several practical MIMO communication scenarios. Then, we note that the channel-to-channel mapping function, even if it exists, is typically unknown and very hard to characterize analytically as it heavily depends on the various elements of the surrounding environment. With this motivation, we propose to leverage the powerful learning capabilities of deep neural networks ....
Ship classification from overhead imagery using synthetic data and domain adaptation
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we revisit the problem of classifying ships (maritime vessels) detected from overhead imagery. Despite the last decade of research on this very important and pertinent problem, it remains largely unsolved. One of the major issues with the detection and classification of ships and other objects in the maritime domain is the lack of substantial ground truth data needed to train state-of-the-art machine learning algorithms. We address this issue by building a large (200k) synthetic image dataset using the Unity gaming engine and 3D ship models. We demonstrate that with the use of synthetic data, classification performance increases dramatically, particularly when there are very few annotated images used in training.
Integrating Artificial Intelligence into Weapon Systems
Venue: arXiv cs.AI 2019
Year: 2019

The integration of Artificial Intelligence (AI) into weapon systems is one of the most consequential tactical and strategic decisions in the history of warfare. Current AI development is a remarkable combination of accelerating capability, hidden decision mechanisms, and decreasing costs. Implementation of these systems is in its infancy and exists on a spectrum from resilient and flexible to simplistic and brittle. Resilient systems should be able to effectively handle the complexities of a high-dimensional battlespace. Simplistic AI implementations could be manipulated by an adversarial AI that identifies and exploits their weaknesses. In this paper, we present a framework for understanding the development of dynamic AI/ML systems that interactively and continuously adapt to their user's needs. We explore the implications of increasingly capable AI in the kill chain and how this will lead inevitably to a fully automated, always on system, barring regulation by treaty. We examine the potential of total integration of cyber and physical security and how this likelihood must inform the development of AI-enabled systems with respect to the "fog of war", human morals, and ethics.
RetroFlow: Maintaining Control Resiliency and Flow Programmability for Software-Defined WANs
Venue: arXiv cs.NI 2019
Year: 2019

Providing resilient network control is a critical concern for deploying Software-Defined Networking (SDN) into Wide-Area Networks (WANs). For performance reasons, a Software-Defined WAN is divided into multiple domains controlled by multiple controllers with a logically centralized view. Under controller failures, we need to remap the control of offline switches from failed controllers to other active controllers. Existing solutions could either overload active controllers to interrupt their normal operations or degrade network performance because of increasing the controller-switch communication overhead. In this paper, we propose RetroFlow to achieve low communication overhead without interrupting the normal processing of active controllers during controller failures. By intelligently configuring a set of selected offline switches working under the legacy routing mode, RetroFlow relieves the active controllers from controlling the selected offline switches while maintaining the flow programmability (e.g., the ability to change paths of flows) of SDN. RetroFlow also smartly transfers the control of offline switches with the SDN routing mode to active controllers to minimize the communication overhead from these offline switches to the active controllers. Simulation results show that compared with the baseline algorithm, RetroFlow can reduce the communication overhead up to 52.6% during a moderate controller failure by recovering 100% flows from offline switches and can reduce the communication overhead up to 61.2% during a serious controller failure by setting to recover 90% of flows from offline switches.
Development and Experimental Evaluation of Grey-Box Models for Application in Model Predictive Control of a Microscale Polygeneration System
Venue: arXiv cs.SY 2019
Year: 2019

With the need for optimisation based supervisory controllers for complex energy systems, comes the need for reduced order system models representing not only the non-linear characteristics of the components, but also certain unknown process dynamics like their internal control logic. We present in this paper an extensive literature study of existing methods and a rational modelling procedure based on the grey-box methodology that satisfies the necessary characteristics for models to be applied in an economic-MPC of a real-world polygeneration system at the Offenburg University of Applied Sciences. The engineering application of the models and their fitting coefficients are shared in this paper. Finally, the models are evaluated against experimental data and the efficacy of the methodology is discussed based on quantitative and qualitative arguments.
From Functional to Logical Scenarios: Detailing a Keyword-Based Scenario Description for Execution in a Simulation Environment
Venue: arXiv cs.SE 2019
Year: 2019

Scenario-based development and test processes are a promising approach for verifying and validating automated driving functions. For this purpose, scenarios have to be generated during the development process in a traceable manner. In early development stages, the operating scenarios of the item to be developed are usually described in an abstract, linguistic way.Within the scope of a simulation-assisted test process, these linguistically described scenarios have to be transformed into a state space representation and converted into data formats which can be used with the respective simulation environment. Currently, this step of detailing scenarios takes a considerable manual effort. Furthermore, a standardized interpretation of the linguistically described scenarios and a consistent transformation into the data formats are not guaranteed due to multiple authors as well as many constraints between the scenario parameters. In this paper, the authors present an approach to automatically detail a keyword-based scenario description for execution in a simulation environment and provide a basis for test case generation. As a first step, the keyword-based description is transformed into a parameter space representation. At the same time, constraints regarding the selection and combination of parameter values are documented for the following process steps (e. g. evolutionary or stochastic test methods). As a second step, the parameter space representation is converted into data formats required by the simulation environment. As an example, the authors use scenarios on German freeways and convert them into the data formats OpenDRIVE (description of the road) and OpenSCENARIO (description of traffic participants and environmental conditions) for execution in the simulation environment Virtual Test Drive.
A fast online cascaded regression algorithm for face alignment
Venue: arXiv cs.CV 2019
Year: 2019

Traditional face alignment based on machine learning usually tracks the localizations of facial landmarks employing a static model trained offline where all of the training data is available in advance. When new training samples arrive, the static model must be retrained from scratch, which is excessively time-consuming and memory-consuming. In many real-time applications, the training data is obtained one by one or batch by batch. It results in that the static model limits its performance on sequential images with extensive variations. Therefore, the most critical and challenging aspect in this field is dynamically updating the tracker's models to enhance predictive and generalization capabilities continuously. In order to address this question, we develop a fast and accurate online learning algorithm for face alignment. Particularly, we incorporate on-line sequential extreme learning machine into a parallel cascaded regression framework, coined incremental cascade regression(ICR). To the best of our knowledge, this is the first incremental cascaded framework with the non-linear regressor. One main advantage of ICR is that the tracker model can be fast updated in an incremental way without the entire retraining process when a new input is incoming. Experimental results demonstrate that the proposed ICR is more accurate and efficient on still or sequential images compared with the recent state-of-the-art cascade approaches. Furthermore, the incremental learning proposed in this paper can update the trained model in real time.
On the Distribution of AoI for the GI/GI/1/1 and GI/GI/1/2* Systems: Exact Expressions and Bounds
Venue: arXiv cs.IT 2019
Year: 2019

Since Age of Information (AoI) has been proposed as a metric that quantifies the freshness of information updates in a communication system, there has been a constant effort in understanding and optimizing different statistics of the AoI process for classical queueing systems. In addition to classical queuing systems, more recently, systems with no queue or a unit capacity queue storing the latest packet have been gaining importance as storing and transmitting older packets do not reduce AoI at the receiver. Following this line of research, we study the distribution of AoI for the GI/GI/1/1 and GI/GI/1/2* systems, under non-preemptive scheduling. For any single-source-single-server queueing system, we derive, using sample path analysis, a fundamental result that characterizes the AoI violation probability, and use it to obtain closed-form expressions for D/GI/1/1, M/GI/1/1 as well as systems that use zero-wait policy. Further, when exact results are not tractable, we present a simple methodology for obtaining upper bounds for the violation probability for both GI/GI/1/1 and GI/GI/1/2* systems. An interesting feature of the proposed upper bounds is that, if the departure rate is given, they overestimate the violation probability by at most a value that decreases with the arrival rate. Thus, given the departure rate and for a fixed average service, the bounds are tighter at higher utilization.
Towards Egocentric Person Re-identification and Social Pattern Analysis
Venue: arXiv cs.CV 2019
Year: 2019

Wearable cameras capture a first-person view of the daily activities of the camera wearer, offering a visual diary of the user behaviour. Detection of the appearance of people the camera user interacts with for social interactions analysis is of high interest. Generally speaking, social events, lifestyle and health are highly correlated, but there is a lack of tools to monitor and analyse them. We consider that egocentric vision provides a tool to obtain information and understand users social interactions. We propose a model that enables us to evaluate and visualize social traits obtained by analysing social interactions appearance within egocentric photostreams. Given sets of egocentric images, we detect the appearance of faces within the days of the camera wearer, and rely on clustering algorithms to group their feature descriptors in order to re-identify persons. Recurrence of detected faces within photostreams allows us to shape an idea of the social pattern of behaviour of the user. We validated our model over several weeks recorded by different camera wearers. Our findings indicate that social profiles are potentially useful for social behaviour interpretation.
Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Occlusion and pose variations, which can change facial appearance significantly, are two major obstacles for automatic Facial Expression Recognition (FER). Though automatic FER has made substantial progresses in the past few decades, occlusion-robust and pose-invariant issues of FER have received relatively less attention, especially in real-world scenarios. This paper addresses the real-world pose and occlusion robust FER problem with three-fold contributions. First, to stimulate the research of FER under real-world occlusions and variant poses, we build several in-the-wild facial expression datasets with manual annotations for the community. Second, we propose a novel Region Attention Network (RAN), to adaptively capture the importance of facial regions for occlusion and pose variant FER. The RAN aggregates and embeds varied number of region features produced by a backbone convolutional neural network into a compact fixed-length representation. Last, inspired by the fact that facial expressions are mainly defined by facial action units, we propose a region biased loss to encourage high attention weights for the most important regions. We validate our RAN and region biased loss on both our built test datasets and four popular datasets: FERPlus, AffectNet, RAF-DB, and SFEW. Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose. Our method also achieves state-of-the-art results on FERPlus, AffectNet, RAF-DB, and SFEW. Code and the collected test data will be publicly available.
Emergent Escape-based Flocking Behavior using Multi-Agent Reinforcement Learning
Venue: arXiv cs.MA 2019
Year: 2019

In nature, flocking or swarm behavior is observed in many species as it has beneficial properties like reducing the probability of being caught by a predator. In this paper, we propose SELFish (Swarm Emergent Learning Fish), an approach with multiple autonomous agents which can freely move in a continuous space with the objective to avoid being caught by a present predator. The predator has the property that it might get distracted by multiple possible preys in its vicinity. We show that this property in interaction with self-interested agents which are trained with reinforcement learning to solely survive as long as possible leads to flocking behavior similar to Boids, a common simulation for flocking behavior. Furthermore we present interesting insights in the swarming behavior and in the process of agents being caught in our modeled environment.
Horn Clauses in Hybrid-Dynamic First-Order Logic
Venue: arXiv cs.LO 2019
Year: 2019

We propose a hybrid-dynamic first-order logic as a formal foundation for specifying and reasoning about reconfigurable systems. As the name suggests, the formalism we develop extends (many-sorted) first-order logic with features that are common to hybrid logics and to dynamic logics. This provides certain key advantages for dealing with reconfigurable systems, such as: (a) a signature of nominals, including operation and relation symbols, that allows references to specific possible worlds / system configurations -- as in the case of hybrid logics; (b) distinguished signatures of rigid and flexible symbols, where the rigid symbols are interpreted uniformly across possible worlds; this supports a rigid form of quantification, which ensures that variables have the same interpretation regardless of the possible world where they are evaluated; (c) hybrid terms, which increase the expressive power of the logic in the context of rigid symbols; and (d) modal operators over dynamic-logic actions, which are defined as regular expressions over binary nominal relations. We then study Horn clauses in this hybrid-dynamic logic, and develop a series of results that lead to an initial-semantics theorem for arbitrary sets of clauses. This shows that a significant fragment of hybrid-dynamic first-order logic has good computational properties, and can serve as a basis for defining executable languages for reconfigurable systems. Lastly, we set out the foundations of logic programming in this fragment by proving a hybrid-dynamic variant of Herbrand's theorem, which reduces the semantic entailment of a logic-programming query by a program to the search of a suitable answer substitution.
Markov perfect equilibria in non-stationary mean-field games
Venue: arXiv cs.GT 2019
Year: 2019

In this paper, we consider both finite and infinite horizon discounted dynamic mean-field games where there is a large population of homogeneous players sequentially making strategic decisions and each player is affected by other players through an aggregate population state. Each player has a private type that only she observes. Such games have been studied in the literature under simplifying assumption that population state dynamics are stationary. In this paper, we consider non-stationary population state dynamics and present a novel backward recursive algorithm to compute Markov perfect equilibrium (MPE) that depend on both, a player's private type, and current (dynamic) population state. Using this algorithm, we study a security problem in cyberphysical system where infected nodes put negative externality on the system, and each node makes a decision to get vaccinated. We numerically compute MPE of the game.
An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs
Venue: arXiv cs.RO 2019
Year: 2019

Nano-size unmanned aerial vehicles (UAVs), with few centimeters of diameter and sub-10 Watts of total power budget, have so far been considered incapable of running sophisticated visual-based autonomous navigation software without external aid from base-stations, ad-hoc local positioning infrastructure, and powerful external computation servers. In this work, we present what is, to the best of our knowledge, the first 27g nano-UAV system able to run aboard an end-to-end, closed-loop visual pipeline for autonomous navigation based on a state-of-the-art deep-learning algorithm, built upon the open-source CrazyFlie 2.0 nano-quadrotor. Our visual navigation engine is enabled by the combination of an ultra-low power computing device (the GAP8 system-on-chip) with a novel methodology for the deployment of deep convolutional neural networks (CNNs). We enable onboard real-time execution of a state-of-the-art deep CNN at up to 18Hz. Field experiments demonstrate that the system's high responsiveness prevents collisions with unexpected dynamic obstacles up to a flight speed of 1.5m/s. In addition, we also demonstrate the capability of our visual navigation engine of fully autonomous indoor navigation on a 113m previously unseen path. To share our key findings with the embedded and robotics communities and foster further developments in autonomous nano-UAVs, we publicly release all our code, datasets, and trained networks.
Do Autonomous Agents Benefit from Hearing?
Venue: arXiv cs.LG 2019
Year: 2019

Mapping states to actions in deep reinforcement learning is mainly based on visual information. The commonly used approach for dealing with visual information is to extract pixels from images and use them as state representation for reinforcement learning agent. But, any vision only agent is handicapped by not being able to sense audible cues. Using hearing, animals are able to sense targets that are outside of their visual range. In this work, we propose the use of audio as complementary information to visual only in state representation. We assess the impact of such multi-modal setup in reach-the-goal tasks in ViZDoom environment. Results show that the agent improves its behavior when visual information is accompanied with audio features.