Reviewerly/exHarmony huggingface dataset:
A Neural Network-Based Driver Gaze Classification System with Vehicle Signals

Driver monitoring can play an essential part in avoiding accidents by warning the driver and shifting the driver's attention to the traffic scenery in time during critical situations. This may apply for the different levels of automated driving, for take-over requests as well as for driving in manual mode. A great proxy for this purpose has always been the driver's gazing direction. The aim of this work is to introduce a robust gaze detection system. In this regard, we make several contributions that are novel in the area of gaze detection systems. In particular, we propose a deep learning approach to predict gaze regions, which is based on informative features such as eye landmarks and head pose angles of the driver. Moreover, we introduce different post-processing techniques that improve the accuracy by exploiting temporal information from videos and the availability of other vehicle signals. Last but not least, we confirm our method with a leave-one-driver-out cross-validation. Unlike previous studies, we do not use gazes to predict maneuver changes, but we consider the human-computer-interaction aspect and use vehicle signals to improve the performance of the estimation. The proposed system is able to achieve an accuracy of 92.3% outperforming earlier landmark-based gaze estimators.
On card guessing game with one time riffle shuffle and complete feedback

This paper studies the game of guessing riffle-shuffled cards with complete feedback. A deck of n cards labeled 1 to n is riffle-shuffled once and placed on a table. A player tries to guess the cards from top and is given complete feedback after each guess. The goal is to find the guessing strategy with maximum reward (expected number of correct guesses). We give the optimal strategy for this game and prove that the maximum expected reward is n∕2+2∕π⋅n+O(1), partially solving an open problem of Bayer and Diaconis (Bayer and Diaconis, 1992).
On Simple Reactive Neural Networks for Behaviour-Based Reinforcement Learning

We present a behaviour-based reinforcement learning approach, inspired by Brook's subsumption architecture, in which simple fully connected networks are trained as reactive behaviours. Our working assumption is that a pick and place robotic task can be simplified by leveraging domain knowledge of a robotics developer to decompose and train reactive behaviours; namely, approach, grasp, and retract. Then the robot autonomously learns how to combine reactive behaviours via an Actor-Critic architecture. We use an Actor-Critic policy to determine the activation and inhibition mechanisms of the reactive behaviours in a particular temporal sequence. We validate our approach in a simulated robot environment where the task is about picking a block and taking it to a target position while orienting the gripper from a top grasp. The latter represents an extra degree-of-freedom of which current end-to-end reinforcement learning approaches fail to generalise. Our findings suggest that robotic learning can be more effective if each behaviour is learnt in isolation and then combined them to accomplish the task. That is, our approach learns the pick and place task in 8,000 episodes, which represents a drastic reduction in the number of training episodes required by an end-to-end approach ( 95,000 episodes) and existing state-of-the-art algorithms.
Deep Learning for Text Detection and Recognition in Complex Engineering Diagrams

Engineering drawings such as Piping and Instrumentation Diagrams contain a vast amount of text data which is essential to identify shapes, pipeline activities, tags, amongst others.These diagrams are often stored in undigitised format, such as paper copy, meaning the information contained within the diagrams is not readily accessible to inspect and use for further data analytics.In this paper, we make use of the benefits of recent deep learning advances by selecting models for both text detection and text recognition, and apply them to the digitisation of text from within real world complex engineering diagrams.Evaluation of such deep learning methods on a dataset of Piping and Instrumentation Diagrams from the Oil & Gas industry showed promising results on detecting and recognising text, without the need for pre-processing steps in complex engineering diagrams.
Autonomous cognition development with lifelong learning: A self-organizing and reflecting cognitive network

Lifelong learning is still a great challenge for cognitive robots since the continuous streaming data they encounter is usually enormous and no-stationary. Traditional cognitive methods suffer from large storage and computation consumption in this situation. Therefore, we propose a self-organizing and reflecting cognitive network (SORCN) to realize robotic lifelong cognitive development through incremental learning and regular reflecting. The network integrates a self-organizing incremental neural network (SOINN) with a modified CFS clustering algorithm. SOINN develops concise object concepts to alleviate storage consumption. Moreover, we modify SOINN by an efficient competitive method based on reflection results to reduce the learning computation. The modified CFS clustering algorithm is designed for reflecting knowledge learned by SOINN periodically. It improves the traditional CFS as a three-step clustering method including clustering, merging and splitting. Specifically, an autonomous center selection strategy is employed for CFS to cater to online learning. Moreover, a series of cluster merging and splitting strategies are proposed to enable CFS to cluster data incrementally and improve its clustering effect. Additionally, the reflection results are utilized to adjust the topological structure of SOINN and guide the future learning. Experimental results demonstrate that SORCN can achieve better learning effectiveness and efficiency over several state-of-art algorithms.
Big data and Spark: Comparison with Hadoop

Now a days, data is generating by different industries on a daily basis which amount growing from terabyte to petabyte. To handle this large amount of data, big data technology is great revolution and has impacted the trends of the applied science. For the analysis of the large amount of data, Hadoop MapReduce technology is used in parallel multiple nodes. The numerous amount of data and information is stored through HDFS, Map and Reduce are two main functions of the MapReduce. There are many loop holes in MapReduce so for fast queries and real time data stream, Spark is introduced. The base of the Spark is DAG and RDD approaches. The main purpose of this research is to make comparison between the basic features of Hadoop and Spark on which basis their performance will be evaluate.
Hardware-Aware Pruning of DNNs using LFSR-Generated Pseudo-Random Indices

Deep neural networks (DNNs) have been emerged as the state-of-the-art algorithms in broad range of applications. To reduce the memory foot-print of DNNs, in particular for embedded applications, sparsification techniques have been proposed. Unfortunately, these techniques come with a large hardware overhead. In this paper, we present a hardware-aware pruning method where the locations of non-zero weights are derived in real-time from a Linear Feedback Shift Registers (LFSRs). Using the proposed method, we demonstrate a total saving of energy and area up to 63.96% and 64.23% for VGG-16 network on down-sampled ImageNet, respectively for iso-compression-rate and iso-accuracy.
The effects of consultant avatar size and dynamics on customer trust in online consultations

This study investigates the impact of avatars on interactions between customers and consultants in remote, online consultations supported by Augmented Reality (AR). Based on past research, we were interested whether the appearance of an avatar and its dynamics affect important factors for online consultations such as social presence, trust in the consultant and perceived customer satisfaction. In particular, we chose avatar size and dynamics (movement/gaze) to compare different avatars in a 2x2 experiment, in which customers wear AR head mounted devices to consult a remotely located consultant in a mock furniture consultation session. Our results show no significant differences in trust and satisfaction, but significantly different levels of perceived social presence for life-sized, dynamic avatars as well as significantly higher co-presence for all life-sized avatars. Additional data from interviews with the participants revealed a clear preference for dynamic avatars over static ones. Based on an analysis of these findings, we make design recommendations and suggest directions for future research.
Parameter Tuning Using Adaptive Moment Estimation in Deep Learning Neural Networks

The twin issues of loss quality (accuracy) and training time are critical in choosing a stochastic optimizer for training deep neural networks. Optimization methods for machine learning include gradient descent, simulated annealing, genetic algorithm and second order techniques like Newton’s method. However, the popular method for optimizing neural networks is gradient descent. Overtime, researchers have made gradient descent more responsive to the requirements of improved quality loss (accuracy) and reduced training time by progressing from using simple learning rate to using adaptive moment estimation technique for parameter tuning. In this work, we investigate the performances of established stochastic gradient descent algorithms like Adam, RMSProp, Adagrad, and Adadelta in terms of training time and loss quality. We show practically, using series of stochastic experiments, that adaptive moment estimation has improved the gradient descent optimization method. Based on the empirical outcomes, we recommend further improvement of the method by using higher moments of gradient for parameter tuning (weight update). The output of our experiments also indicate that neural network is a stochastic algorithm.
Improving multi-atlas cardiac structure segmentation of computed tomography angiography: A performance evaluation based on a heterogeneous dataset

Multi-atlas based segmentation is an effective technique that transforms a representative set of atlas images and labels into a target image for structural segmentation. However, a significant limitation of this approach relates to the fact that the atlas and the target images need to be similar in volume orientation, coverage, or acquisition protocols in order to prevent image misregistration and avoid segmentation fault. In this study, we aim to evaluate the impact of using a heterogeneous Computed Tomography Angiography (CTA) dataset on the performance of a multi-atlas cardiac structure segmentation framework. We propose a generalized technique based upon using the Simple Linear Iterative Clustering (SLIC) supervoxel method to detect a bounding box region enclosing the heart before subsequent cardiac structure segmentation. This technique facilitates our framework to process CTA datasets acquired from distinct imaging protocols and to improve its segmentation accuracy and speed. In a four-way cross comparison based on 60 CTA studies from our institution and 60 CTA datasets from the Multi-Modality Whole Heart Segmentation MICCAI challenge, we show that the proposed framework performs well in segmenting seven different cardiac structures based upon interchangeable atlas and target datasets acquired from different imaging settings. For the overall results, our automated segmentation framework attains a median Dice, mean distance, and Hausdorff distance of 0.88, 1.5 mm, and 9.69 mm over the entire datasets. The average processing time was 1.55 min for both datasets. Furthermore, this study shows that it is feasible to exploit heterogenous datasets from different imaging protocols and institutions for accurate multi-atlas cardiac structure segmentation.
PROMES: An ontology‐based messaging service for semantically interoperable information exchange during disaster response

Abstract Disaster response requires the cooperation of multiple emergency responder organizations (EROs). However, after‐action reports relating to large‐scale disasters identity communication difficulties among EROs as a major hindrance to collaboration. On the one hand, the use of two‐radio communication, based on multiple orthogonal frequencies and uneven coverage, has been shown to degrade inter‐organization communication. On the other hand, because they reflect different areas of expertise, EROs use differing terminologies, which are difficult to reconcile. These issues lead to ambiguities, misunderstandings, and inefficient exchange of data and information among those involved, which can impede the response process and slow decision making. We, therefore, hypothesize that promoting semantic interoperability across ERO information systems might improve information exchange among stakeholders and thereby allow a more coherent response to the disaster. We propose an ontology‐based messaging service on the basis of the Emergency Data Exchange Language (EDXL) standards. The parties involved will continue to use the terminologies to which they are accustomed, but the system will resolve inconsistencies and thereby enhance mutual understanding among EROs by ensuring semantic translation of the exchanged information. The evaluation of the semantic translation demonstrated the effectiveness and accuracy of the proposed service.
Temporally Consistent Horizon Lines

The horizon line is an important geometric feature for many image processing and scene understanding tasks in computer vision. For instance, in navigation of autonomous vehicles or driver assistance, it can be used to improve 3D reconstruction as well as for semantic interpretation of dynamic environments. While both algorithms and datasets exist for single images, the problem of horizon line estimation from video sequences has not gained attention. In this paper, we show how convolutional neural networks are able to utilise the temporal consistency imposed by video sequences in order to increase the accuracy and reduce the variance of horizon line estimates. A novel CNN architecture with an improved residual convolutional LSTM is presented for temporally consistent horizon line estimation. We propose an adaptive loss function that ensures stable training as well as accurate results. Furthermore, we introduce an extension of the KITTI dataset which contains precise horizon line labels for 43699 images across 72 video sequences. A comprehensive evaluation shows that the proposed approach consistently achieves superior performance compared with existing methods.
Sample-Efficient Robot Motion Learning using Gaussian Process Latent Variable Models

Robotic manipulators are reaching a state where we could see them in household environments in the following decade. Nevertheless, such robots need to be easy to instruct by lay people. This is why kinesthetic teaching has become very popular in recent years, in which the robot is taught a motion that is encoded as a parametric function - usually a Movement Primitive (MP)-. This approach produces trajectories that are usually suboptimal, and the robot needs to be able to improve them through trial-and-error. Such optimization is often done with Policy Search (PS) reinforcement learning, using a given reward function. PS algorithms can be classified as model-free, where neither the environment nor the reward function are modelled, or model-based, which can use a surrogate model of the reward function and/or a model for the dynamics of the task. However, MPs can become very high-dimensional in terms of parameters, which constitute the search space, so their optimization often requires too many samples. In this paper, we assume we have a robot motion task characterized with an MP of which we cannot model the dynamics. We build a surrogate model for the reward function, that maps an MP parameter latent space (obtained through a Mutual-information-weighted Gaussian Process Latent Variable Model) into a reward. While we do not model the task dynamics, using mutual information to shrink the task space makes it more consistent with the reward and so the policy improvement is faster in terms of sample efficiency.
Boosting meta-learning with simulated data complexity measures

Meta-Learning has been largely used over the last years to support the recommendation of the most suitable machine learning algorithm(s) and hyperparameters for new datasets. Traditionally, a meta-base is created containing meta-features extracted from several datasets along with the performance of a pool of machine learning algorithms when applied to these datasets. The meta-features must describe essential aspects of the dataset and distinguish different problems and solutions. However, if one wants the use of Meta-Learning to be computationally efficient, the extraction of the meta-feature values should also show a low computational cost, considering a trade-off between the time spent to run all the algorithms and the time required to extract the meta-features. One class of measures with successful results in the characterization of classification datasets is concerned with estimating the underlying complexity of the classification problem. These data complexity measures take into account the overlap between classes imposed by the feature values, the separability of the classes and distribution of the instances within the classes. However, the extraction of these measures from datasets usually presents a high computational cost. In this paper, we propose an empirical approach designed to decrease the computational cost of computing the data complexity measures, while still keeping their descriptive ability. The proposal consists of a novel Meta-Learning system able to predict the values of the data complexity measures for a dataset by using simpler meta-features as input. In an extensive set of experiments, we show that the predictive performance achieved by Meta-Learning systems which use the predicted data complexity measures is similar to the performance obtained using the original data complexity measures, but the computational cost involved in their computation is significantly reduced.
SuperSlash: A Unified Design Space Exploration and Model Compression Methodology for Design of Deep Learning Accelerators With Reduced Off-Chip Memory Access Volume

Deploying deep learning (DL) models on resource-constrained embedded devices is a challenging task. The limited on-chip memory on such devices results in increased off-chip memory access volume, thus limiting the size of DL models that can be efficiently realized in such systems. Design space exploration (DSE) under memory constraint, or to achieve minimal off-chip memory access volume, has recently received much attention. Unfortunately, DSE alone cannot reduce the amount of off-chip memory accesses beyond a certain point due to the fixed model size. Model compression via pruning can be employed to reduce the size of the model and the associated off-chip memory accesses. However, in this article, we demonstrate that pruned models with even the same accuracy and model size may require a different number of off-chip memory accesses depending upon the pruning strategy adopted. Thus, mainstream pruning techniques may not be closely tied to the design goals, and thereby hard to be integrated with existing DSE techniques. To overcome this problem, we propose SuperSlash, a unified solution for DSE and model compression. SuperSlash estimates off-chip memory access volume overhead of each layer of a DL model by exploring multiple design candidates. In particular, it evaluates multiple data reuse strategies for each layer, along with the possibility of layer fusion. Layer fusion aims at reducing the off-chip memory access volume by avoiding the intermediate off-chip storage of a layer's output and directly using it for processing of the subsequent layer. SuperSlash then guides the pruning process via a ranking function, which ranks each layer according to its explored off-chip memory access cost. We demonstrate that SuperSlash not only offers an extensive design space coverage but also provides lower off-chip memory access volume (up to 57.71%, 25.83%, 47.73%, and 29.02% reduction for VGG16, ResNet56, ResNet110, and MobileNetV1, respectively) as compared to the state-of-art.
Single Image Super-Resolution Via Residual Neuron Attention Networks

Deep Convolutional Neural Networks (DCNNs) have achieved impressive performance in Single Image Super-Resolution (SISR). To further improve the performance, existing CNN-based methods generally focus on designing deeper architecture of the network. However, we argue blindly increasing network's depth is not the most sensible way. In this paper, we propose a novel end-to-end Residual Neuron Attention Networks (RNAN) for more efficient and effective SISR. Structurally, our RNAN is a sequential integration of the well-designed Global Context-enhanced Residual Groups (GCRGs), which extracts super-resolved features from coarse to fine. Our GCRG is designed with two novelties. Firstly, the Residual Neuron Attention (RNA) mechanism is proposed in each block of GCRG to reveal the relevance of neurons for better feature representation. Furthermore, the Global Context (GC) block is embedded into RNAN at the end of each GCRG for effectively modeling the global contextual information. Experiments results demonstrate that our RNAN achieves the comparable results with state-of-the-art methods in terms of both quantitative metrics and visual quality, however, with simplified network architecture.
Machine learned Green's functions that approximately satisfy the wave equation

Green’s functions are wavefield solutions for a particular point source. They form basis functions to build wavefields for modeling and inversion. However, calculating Green’s functions are both costly and memory intensive. We formulate frequency-domain machine-learned Green’s functions that are represented by neural networks (NN). This NN outputs a complex number (two values representing the real and imaginary part) for the scattered Green’s function at a location in space for a specific source location (both locations are input to the network). Considering a background homogeneous medium admitting an analytical Green’s function solution, the network is trained by fitting the output perturbed Green’s function and its derivatives to the wave equation expressed in terms of the perturbed Green’s function. The derivatives are calculated through the concept of automatic differentiation. In this case, the background Green’s function absorbs the point source singularity, which will allow us to train the network using random points over space and source location using a uniform distribution. Thus, feeding a reasonable number of random points from the model space will ultimately train a fully connected 8-layer deep neural network, to predict the scattered Green’s function. Initial tests on part of the simple layered model (extracted from the left side of the Marmousi model) with sources on the surface demonstrate the successful training of the NN for this application. Using the trained NN model for the Marmousi as an initial NN model for solving for the scattered Green’s function for a 2D slice from the Sigsbee model helped the NN converge faster to a reasonable solution. Presentation Date: Wednesday, October 14, 2020 Session Start Time: 1:50 PM Presentation Time: 2:15 PM Location: 360A Presentation Type: Oral