OpenRaiser/Intern-Atlas huggingface dataset:
Synthetic Event Time Series Health Data Generation
Venue: arXiv cs.LG 2019
Year: 2019

Synthetic medical data which preserves privacy while maintaining utility can be used as an alternative to real medical data, which has privacy costs and resource constraints associated with it. At present, most models focus on generating cross-sectional health data which is not necessarily representative of real data. In reality, medical data is longitudinal in nature, with a single patient having multiple health events, non-uniformly distributed throughout their lifetime. These events are influenced by patient covariates such as comorbidities, age group, gender etc. as well as external temporal effects (e.g. flu season). While there exist seminal methods to model time series data, it becomes increasingly challenging to extend these methods to medical event time series data. Due to the complexity of the real data, in which each patient visit is an event, we transform the data by using summary statistics to characterize the events for a fixed set of time intervals, to facilitate analysis and interpretability. We then train a generative adversarial network to generate synthetic data. We demonstrate this approach by generating human sleep patterns, from a publicly available dataset. We empirically evaluate the generated data and show close univariate resemblance between synthetic and real data. However, we also demonstrate how stratification by covariates is required to gain a deeper understanding of synthetic data quality.
Reversible Hardware for Acoustic Communications
Venue: arXiv cs.ET 2019
Year: 2019

Reversible computation has been recognised as a potential solution to the technological bottleneck in the future of computing machinery. Rolf Landauer determined the lower limit for power dissipation in computation and noted that dissipation happens when information is lost, i.e., when a bit is erased. This meant that reversible computation, conserving information conserves energy as well, and as such can operate on arbitrarily small power. There were only a few applications and use cases of reversible computing hardware. Here we present a novel reversible computation architecture for time reversal of waves, with an application to sound wave communications. This energy efficient design is also a natural one, and it allows the use of the same hardware for transmission and reception at the time reversal mirror.
Automated Augmentation with Reinforcement Learning and GANs for Robust Identification of Traffic Signs using Front Camera Images
Venue: arXiv cs.CV 2019
Year: 2019

Traffic sign identification using camera images from vehicles plays a critical role in autonomous driving and path planning. However, the front camera images can be distorted due to blurriness, lighting variations and vandalism which can lead to degradation of detection performances. As a solution, machine learning models must be trained with data from multiple domains, and collecting and labeling more data in each new domain is time consuming and expensive. In this work, we present an end-to-end framework to augment traffic sign training data using optimal reinforcement learning policies and a variety of Generative Adversarial Network (GAN) models, that can then be used to train traffic sign detector modules. Our automated augmenter enables learning from transformed nightime, poor lighting, and varying degrees of occlusions using the LISA Traffic Sign and BDD-Nexar dataset. The proposed method enables mapping training data from one domain to another, thereby improving traffic sign detection precision/recall from 0.70/0.66 to 0.83/0.71 for nighttime images.
Scalable and Reliable Multi-Dimensional Aggregation of Sensor Data Streams
Venue: arXiv cs.DC 2019
Year: 2019

Ever-increasing amounts of data and requirements to process them in real time lead to more and more analytics platforms and software systems being designed according to the concept of stream processing. A common area of application is the processing of continuous data streams from sensors, for example, IoT devices or performance monitoring tools. In addition to analyzing pure sensor data, analyses of data for groups of sensors often need to be performed as well. Therefore, data streams of the individual sensors have to be continuously aggregated to a data stream for a group. Motivated by a real-world application scenario, we propose that such a stream aggregation approach has to allow for aggregating sensors in hierarchical groups, support multiple such hierarchies in parallel, provide reconfiguration at runtime, and preserve the scalability and reliability qualities induced by applying stream processing techniques. We propose a stream processing architecture fulfilling these requirements, which can be integrated into existing big data architectures. We present a pilot implementation of such an extended architecture and show how it is used in industry. Furthermore, in experimental evaluations we show that our solution scales linearly with the amount of sensors and provides adequate reliability in the case of faults.
Fine-grained Qualitative Spatial Reasoning about Point Positions
Venue: arXiv cs.AI 2019
Year: 2019

The ability to persist in the spacial environment is, not only in the robotic context, an essential feature. Positional knowledge is one of the most important aspects of space and a number of methods to represent these information have been developed in the in the research area of spatial cognition. The basic qualitative spatial representation and reasoning techniques are presented in this thesis and several calculi are briefly reviewed. Features and applications of qualitative calculi are summarized. A new calculus for representing and reasoning about qualitative spatial orientation and distances is being designed. It supports an arbitrary level of granularity over ternary relations of points. Ways of improving the complexity of the composition are shown and an implementation of the calculus demonstrates its capabilities. Existing qualitative spatial calculi of positional information are compared to the new approach and possibilities for future research are outlined.
Learning Models over Relational Data: A Brief Tutorial
Venue: arXiv cs.DB 2019
Year: 2019

This tutorial overviews the state of the art in learning models over relational databases and makes the case for a first-principles approach that exploits recent developments in database research. The input to learning classification and regression models is a training dataset defined by feature extraction queries over relational databases. The mainstream approach to learning over relational data is to materialize the training dataset, export it out of the database, and then learn over it using a statistical package. This approach can be expensive as it requires the materialization of the training dataset. An alternative approach is to cast the machine learning problem as a database problem by transforming the data-intensive component of the learning task into a batch of aggregates over the feature extraction query and by computing this batch directly over the input database. The tutorial highlights a variety of techniques developed by the database theory and systems communities to improve the performance of the learning task. They rely on structural properties of the relational data and of the feature extraction query, including algebraic (semi-ring), combinatorial (hypertree width), statistical (sampling), or geometric (distance) structure. They also rely on factorized computation, code specialization, query compilation, and parallelization.
Design of Internet of Things Based Controller for Direct Current Motors
Venue: arXiv cs.RO 2019
Year: 2019

It is known that internet have been widespread in many areas of life and enabled machines to communicate each other. The terms of Internet of Things (IoT) has emerged as a result of networks consist of machines and equipment that work independent of operator and communicate each other. Sensors, embedded systems, communication technologies and data storage systems like clouds create wide area networks that can communicate and share data. In this study, the application of the IoT on a low speed mechanical benchmark driven by a direct current motor has been designed. It is aimed to keep the motor speed fixed at desired value according to the changings occurs depending on varying the pressure acting on the actuators. In presented study, a speed control is performed by establishing a modeling and simulation mechanism that will give the closest results to the real system.
Bootstrapping NLU Models with Multi-task Learning
Venue: arXiv cs.CL 2019
Year: 2019

Bootstrapping natural language understanding (NLU) systems with minimal training data is a fundamental challenge of extending digital assistants like Alexa and Siri to a new language. A common approach that is adapted in digital assistants when responding to a user query is to process the input in a pipeline manner where the first task is to predict the domain, followed by the inference of intent and slots. However, this cascaded approach instigates error propagation and prevents information sharing among these tasks. Further, the use of words as the atomic units of meaning as done in many studies might lead to coverage problems for morphologically rich languages such as German and French when data is limited. We address these issues by introducing a character-level unified neural architecture for joint modeling of the domain, intent, and slot classification. We compose word-embeddings from characters and jointly optimize all classification tasks via multi-task learning. In our results, we show that the proposed architecture is an optimal choice for bootstrapping NLU systems in low-resource settings thus saving time, cost and human effort.
A Turing Test for Crowds
Venue: arXiv cs.MA 2019
Year: 2019

The realism and believability of crowd simulations underpins computational studies of human collective behaviour, with implications for urban design, policing, security and many other areas. Realism concerns the closeness of the fit between a simulation and observed data, and believability concerns the human perception of plausibility. In this paper, we ask two questions, via a so-called "Turing Test" for crowds: (1) Can human observers distinguish between real and simulated crowds, and (2) Can human observers identify real crowds versus simulated crowds? In a study with student volunteers (n=384), we find convincing evidence that non-specialist individuals are able to reliably distinguish between real and simulated crowds. A rather more surprising result is that such individuals are overwhelmingly unable to identify real crowds. That is, they can tell real from simulated crowds, but are unable to say which is which. Our main conclusion is that (to the lay-person, at least) realistic crowds are not believable (and vice versa).
Computationally Data-Independent Memory Hard Functions
Venue: arXiv cs.CR 2019
Year: 2019

Memory hard functions (MHFs) are an important cryptographic primitive that are used to design egalitarian proofs of work and in the construction of moderately expensive key-derivation functions resistant to brute-force attacks. Broadly speaking, MHFs can be divided into two categories: data-dependent memory hard functions (dMHFs) and data-independent memory hard functions (iMHFs). iMHFs are resistant to certain side-channel attacks as the memory access pattern induced by the honest evaluation algorithm is independent of the potentially sensitive input e.g., password. While dMHFs are potentially vulnerable to side-channel attacks (the induced memory access pattern might leak useful information to a brute-force attacker), they can achieve higher cumulative memory complexity (CMC) in comparison than an iMHF. In this paper, we introduce the notion of computationally data-independent memory hard functions (ciMHFs). Intuitively, we require that memory access pattern induced by the (randomized) ciMHF evaluation algorithm appears to be independent from the standpoint of a computationally bounded eavesdropping attacker --- even if the attacker selects the initial input. We then ask whether it is possible to circumvent known upper bound for iMHFs and build a ciMHF with CMC $Ω(N^2)$. Surprisingly, we answer the question in the affirmative when the ciMHF evaluation algorithm is executed on a two-tiered memory architecture (RAM/Cache). See paper for the full abstract.
Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of years. Although compelling, the idea of co-adapting morphology and behaviours in robots is often unfeasible because of the long manufacturing times, and the need to re-design an appropriate controller for each morphology. In this paper, we propose a novel approach to automatically and efficiently co-adapt a robot morphology and its controller. Our approach is based on recent advances in deep reinforcement learning, and specifically the soft actor critic algorithm. Key to our approach is the possibility of leveraging previously tested morphologies and behaviors to estimate the performance of new candidate morphologies. As such, we can make full use of the information available for making more informed decisions, with the ultimate goal of achieving a more data-efficient co-adaptation (i.e., reducing the number of morphologies and behaviors tested). Simulated experiments show that our approach requires drastically less design prototypes to find good morphology-behaviour combinations, making this method particularly suitable for future co-adaptation of robot designs in the real world.
Handwritten and Machine printed OCR for Geez Numbers Using Artificial Neural Network
Venue: arXiv cs.CV 2019
Year: 2019

Researches have been done on Ethiopic scripts. However studies excluded the Geez numbers from the studies because of different reasons. This paper presents offline handwritten and machine printed Geez number recognition using feed forward back propagation artificial neural network. On this study, different Geez image characters were collected from google image search and three persons are instructed to write the numbers using pencil. In total we have collected 560 numbers of characters. We have used 460 of the characters for training and 100 are used for testing. Accordingly we have achieved overall all classification ~89:88%
Multi-attention Networks for Temporal Localization of Video-level Labels
Venue: arXiv cs.CV 2019
Year: 2019

Temporal localization remains an important challenge in video understanding. In this work, we present our solution to the 3rd YouTube-8M Video Understanding Challenge organized by Google Research. Participants were required to build a segment-level classifier using a large-scale training data set with noisy video-level labels and a relatively small-scale validation data set with accurate segment-level labels. We formulated the problem as a multiple instance multi-label learning and developed an attention-based mechanism to selectively emphasize the important frames by attention weights. The model performance is further improved by constructing multiple sets of attention networks. We further fine-tuned the model using the segment-level data set. Our final model consists of an ensemble of attention/multi-attention networks, deep bag of frames models, recurrent neural networks and convolutional neural networks. It ranked 13th on the private leader board and stands out for its efficient usage of resources.
Explanatory Masks for Neural Network Interpretability
Venue: arXiv cs.LG 2019
Year: 2019

Neural network interpretability is a vital component for applications across a wide variety of domains. In such cases it is often useful to analyze a network which has already been trained for its specific purpose. In this work, we develop a method to produce explanation masks for pre-trained networks. The mask localizes the most important aspects of each input for prediction of the original network. Masks are created by a secondary network whose goal is to create as small an explanation as possible while still preserving the predictive accuracy of the original network. We demonstrate the applicability of our method for image classification with CNNs, sentiment analysis with RNNs, and chemical property prediction with mixed CNN/RNN architectures.
New Query Lower Bounds for Submodular Function MInimization
Venue: arXiv cs.DS 2019
Year: 2019

We consider submodular function minimization in the oracle model: given black-box access to a submodular set function $f:2^{[n]}\rightarrow \mathbb{R}$, find an element of $\arg\min_S \{f(S)\}$ using as few queries to $f(\cdot)$ as possible. State-of-the-art algorithms succeed with $\tilde{O}(n^2)$ queries [LeeSW15], yet the best-known lower bound has never been improved beyond $n$ [Harvey08]. We provide a query lower bound of $2n$ for submodular function minimization, a $3n/2-2$ query lower bound for the non-trivial minimizer of a symmetric submodular function, and a $\binom{n}{2}$ query lower bound for the non-trivial minimizer of an asymmetric submodular function. Our $3n/2-2$ lower bound results from a connection between SFM lower bounds and a novel concept we term the cut dimension of a graph. Interestingly, this yields a $3n/2-2$ cut-query lower bound for finding the global mincut in an undirected, weighted graph, but we also prove it cannot yield a lower bound better than $n+1$ for $s$-$t$ mincut, even in a directed, weighted graph.
On q-ary Bent and Plateaued Functions
Venue: arXiv cs.IT 2019
Year: 2019

We obtain the following results. For any prime $q$ the minimal Hamming distance between distinct regular $q$-ary bent functions of $2n$ variables is equal to $q^n$. The number of $q$-ary regular bent functions at the distance $q^n$ from the quadratic bent function $Q_n=x_1x_2+\dots+x_{2n-1}x_{2n}$ is equal to $q^n(q^{n-1}+1)\cdots(q+1)(q-1)$ for $q>2$. The Hamming distance between distinct binary $s$-plateaued functions of $n$ variables is not less than $2^{\frac{s+n-2}{2}}$ and the Hamming distance between distinctternary $s$-plateaued functions of $n$ variables is not less than $3^{\frac{s+n-1}{2}}$. These bounds are tight. For $q=3$ we prove an upper bound on nonlinearity of ternary functions in terms of their correlation immunity. Moreover, functions reaching this bound are plateaued. For $q=2$ analogous result are well known but for large $q$ it seems impossible. Constructions and some properties of $q$-ary plateaued functions are discussed.
VLUC: An Empirical Benchmark for Video-Like Urban Computing on Citywide Crowd and Traffic Prediction
Venue: arXiv cs.LG 2019
Year: 2019

Nowadays, massive urban human mobility data are being generated from mobile phones, car navigation systems, and traffic sensors. Predicting the density and flow of the crowd or traffic at a citywide level becomes possible by using the big data and cutting-edge AI technologies. It has been a very significant research topic with high social impact, which can be widely applied to emergency management, traffic regulation, and urban planning. In particular, by meshing a large urban area to a number of fine-grained mesh-grids, citywide crowd and traffic information in a continuous time period can be represented like a video, where each timestamp can be seen as one video frame. Based on this idea, a series of methods have been proposed to address video-like prediction for citywide crowd and traffic. In this study, we publish a new aggregated human mobility dataset generated from a real-world smartphone application and build a standard benchmark for such kind of video-like urban computing with this new dataset and the existing open datasets. We first comprehensively review the state-of-the-art works of literature and formulate the density and in-out flow prediction problem, then conduct a thorough performance assessment for those methods. With this benchmark, we hope researchers can easily follow up and quickly launch a new solution on this topic.
Faster AutoAugment: Learning Augmentation Strategies using Backpropagation
Venue: arXiv cs.CV 2019
Year: 2019

Data augmentation methods are indispensable heuristics to boost the performance of deep neural networks, especially in image recognition tasks. Recently, several studies have shown that augmentation strategies found by search algorithms outperform hand-made strategies. Such methods employ black-box search algorithms over image transformations with continuous or discrete parameters and require a long time to obtain better strategies. In this paper, we propose a differentiable policy search pipeline for data augmentation, which is much faster than previous methods. We introduce approximate gradients for several transformation operations with discrete parameters as well as the differentiable mechanism for selecting operations. As the objective of training, we minimize the distance between the distributions of augmented data and the original data, which can be differentiated. We show that our method, Faster AutoAugment, achieves significantly faster searching than prior work without a performance drop.
Maintaining Discrimination and Fairness in Class Incremental Learning
Venue: arXiv cs.CV 2019
Year: 2019

Deep neural networks (DNNs) have been applied in class incremental learning, which aims to solve common real-world problems of learning new classes continually. One drawback of standard DNNs is that they are prone to catastrophic forgetting. Knowledge distillation (KD) is a commonly used technique to alleviate this problem. In this paper, we demonstrate it can indeed help the model to output more discriminative results within old classes. However, it cannot alleviate the problem that the model tends to classify objects into new classes, causing the positive effect of KD to be hidden and limited. We observed that an important factor causing catastrophic forgetting is that the weights in the last fully connected (FC) layer are highly biased in class incremental learning. In this paper, we propose a simple and effective solution motivated by the aforementioned observations to address catastrophic forgetting. Firstly, we utilize KD to maintain the discrimination within old classes. Then, to further maintain the fairness between old classes and new classes, we propose Weight Aligning (WA) that corrects the biased weights in the FC layer after normal training process. Unlike previous work, WA does not require any extra parameters or a validation set in advance, as it utilizes the information provided by the biased weights themselves. The proposed method is evaluated on ImageNet-1000, ImageNet-100, and CIFAR-100 under various settings. Experimental results show that the proposed method can effectively alleviate catastrophic forgetting and significantly outperform state-of-the-art methods.