OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging the bfloat16 Artificial Intelligence Datatype For Higher-Precision Computations
Venue: arXiv cs.MS 2019
Year: 2019

In recent years fused-multiply-add (FMA) units with lower-precision multiplications and higher-precision accumulation have proven useful in machine learning/artificial intelligence applications, most notably in training deep neural networks due to their extreme computational intensity. Compared to classical IEEE-754 32 bit (FP32) and 64 bit (FP64) arithmetic, these reduced precision arithmetic can naturally be sped up disproportional to their shortened width. The common strategy of all major hardware vendors is to aggressively further enhance their performance disproportionately. One particular FMA operation that multiplies two BF16 numbers while accumulating in FP32 has been found useful in deep learning, where BF16 is the 16-bit floating point datatype with IEEE FP32 numerical range but 8 significant bits of precision. In this paper, we examine the use this FMA unit to implement higher-precision matrix routines in terms of potential performance gain and implications on accuracy. We demonstrate how a decomposition into multiple smaller datatypes can be used to assemble a high-precision result, leveraging the higher precision accumulation of the FMA unit. We first demonstrate that computations of vector inner products and by natural extension, matrix-matrix products can be achieved by decomposing FP32 numbers in several BF16 numbers followed by appropriate computations that can accommodate the dynamic range and preserve accuracy compared to standard FP32 computations, while projecting up to 5.2x speed-up. Furthermore, we examine solution of linear equations formulated in the residual form that allows for iterative refinement. We demonstrate that the solution obtained to be comparable to those offered by FP64 under a large range of linear system condition numbers.
Macrocanonical Models for Texture Synthesis
Venue: arXiv cs.CV 2019
Year: 2019

In this article we consider macrocanonical models for texture synthesis. In these models samples are generated given an input texture image and a set of features which should be matched in expectation. It is known that if the images are quantized, macrocanonical models are given by Gibbs measures, using the maximum entropy principle. We study conditions under which this result extends to real-valued images. If these conditions hold, finding a macrocanonical model amounts to minimizing a convex function and sampling from an associated Gibbs measure. We analyze an algorithm which alternates between sampling and minimizing. We present experiments with neural network features and study the drawbacks and advantages of using this sampling scheme.
Distributed Deep Learning Model for Intelligent Video Surveillance Systems with Edge Computing
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a Distributed Intelligent Video Surveillance (DIVS) system using Deep Learning (DL) algorithms and deploy it in an edge computing environment. We establish a multi-layer edge computing architecture and a distributed DL training model for the DIVS system. The DIVS system can migrate computing workloads from the network center to network edges to reduce huge network communication overhead and provide low-latency and accurate video analysis solutions. We implement the proposed DIVS system and address the problems of parallel training, model synchronization, and workload balancing. Task-level parallel and model-level parallel training methods are proposed to further accelerate the video analysis process. In addition, we propose a model parameter updating method to achieve model synchronization of the global DL model in a distributed EC environment. Moreover, a dynamic data migration approach is proposed to address the imbalance of workload and computational power of edge nodes. Experimental results showed that the EC architecture can provide elastic and scalable computing power, and the proposed DIVS system can efficiently handle video surveillance and analysis tasks.
dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs
Venue: arXiv cs.CV 2019
Year: 2019

Objective assessment of image quality is fundamentally important in many image processing tasks. In this work, we focus on learning blind image quality assessment (BIQA) models which predict the quality of a digital image with no access to its original pristine-quality counterpart as reference. One of the biggest challenges in learning BIQA models is the conflict between the gigantic image space (which is in the dimension of the number of image pixels) and the extremely limited reliable ground truth data for training. Such data are typically collected via subjective testing, which is cumbersome, slow, and expensive. Here we first show that a vast amount of reliable training data in the form of quality-discriminable image pairs (DIP) can be obtained automatically at low cost by exploiting large-scale databases with diverse image content. We then learn an opinion-unaware BIQA (OU-BIQA, meaning that no subjective opinions are used for training) model using RankNet, a pairwise learning-to-rank (L2R) algorithm, from millions of DIPs, each associated with a perceptual uncertainty level, leading to a DIP inferred quality (dipIQ) index. Extensive experiments on four benchmark IQA databases demonstrate that dipIQ outperforms state-of-the-art OU-BIQA models. The robustness of dipIQ is also significantly improved as confirmed by the group MAximum Differentiation (gMAD) competition method. Furthermore, we extend the proposed framework by learning models with ListNet (a listwise L2R algorithm) on quality-discriminable image lists (DIL). The resulting DIL Inferred Quality (dilIQ) index achieves an additional performance gain.
An Integrated Autoencoder-Based Filter for Sparse Big Data
Venue: arXiv cs.LG 2019
Year: 2019

We propose a novel filter for sparse big data, called an integrated autoencoder (IAE), which utilizes auxiliary information to mitigate data sparsity. The proposed model achieves an appropriate balance between prediction accuracy, convergence speed, and complexity. We implement experiments on a GPS trajectory dataset, and the results demonstrate that the IAE is more accurate and robust than some state-of-the-art methods.
Challenges in Integrating Technology into Education
Venue: arXiv cs.CY 2019
Year: 2019

Despite significant amount of investment, there seems to be obstacles to technology integration in education. In order to shed light on the nature of perceived obstacles to technology integration, opinions of 117 professionals, who were selected by Turkish Ministry of National Education as experts in their respective fields, about the obstacles to integration of technology into education were investigated. After categorizing the perceived obstacles by factor analysis, associations of those categories with personal and professional differences were further investigated for better contextualizing the findings. Correlations were analyzed by Pearson's product moment coefficient and point-biserial coefficient. The results revealed that it's not the hardware itself that constitute obstacles to technology integration. Insufficiency of in-service and pre-service training, content support, and incentive system emerged as major perceived obstacles to technology integration. Inadequacy of physical and technological infrastructure was also found to be an important obstacle to successful integration. Novelty of the technologies compared to older ones were not found to be an obstacle to technology integration. Moreover, participants stressed the lack of education in teacher training institutions about current technologies that Ministry of National Education officially requires teachers to use as part of their jobs to be another important obstacle. There were no correlations between sex, age, level of education, job position, year of experience in other careers, and any of the categories of perceived obstacles. However, there was a strong negative correlation between year of experience in teaching and insufficiency of resources. Association between year of experience in educational administration and negative psychological state was also strong and negative.
On interplay between excitability and geometry
Venue: arXiv cs.ET 2019
Year: 2019

A commonly accepted feature of an excitable medium is that a local excitation leads to a propagation of circular or spiral excitation wave-fronts. This is indeed the case in fully excitable medium. However, with a decrease of an excitability localised wave-fragments emerge and propagate ballistically. Using FitzhHugh-Nagumo model we numerically study how excitation wave-fronts behave in a geometrically constrained medium and how the wave-fronts explore a random planar graph. We uncover how excitability controls propagation of excitation in angled branches, influences arrest of excitation entering a sudden expansion, and determines patterns of traversing of a random planar graph by an excitation waves.
Statistical-Based Privacy-Preserving Scheme with Malicious Consumers Identification for Smart Grid
Venue: arXiv cs.CR 2019
Year: 2019

As smart grids are getting popular and being widely implemented, preserving the privacy of consumers is becoming more substantial. Power generation and pricing in smart grids depends on the continuously gathered information from the consumers. However, having access to the data relevant to the electricity consumption of each individual consumer is in conflict with its privacy. One common approach for preserving privacy is to aggregate data of different consumers and to use their smart-meters for calculating the bills. But in this approach, malicious consumers who send erroneous data to take advantage or disrupt smart grid cannot be identified. In this paper, we propose a new statistical-based scheme for data gathering and billing in which the privacy of consumers is preserved, and at the same time, if any consumer with erroneous data can be detected. Our simulation results verify these matters.
Design of optimized backstepping controller for the synchronization of chaotic Colpitts oscillator using shark smell algorithm
Venue: arXiv cs.SY 2019
Year: 2019

In this paper, an adaptive backstepping controller has been tuned to synchronize two chaotic Colpitts oscillators in a master slave configuration. The parameters of the controller are determined using shark smell optimization (SSO) algorithm. Numerical results are presented and compared with those of particle swarm optimization (PSO) algorithm. Simulation results show better performance in terms of accuracy and convergence for the proposed optimized method compared to PSO optimized controller or any non-optimized backstepping controller.
Unsupervised Synthesis of Anomalies in Videos: Transforming the Normal
Venue: arXiv cs.CV 2019
Year: 2019

Abnormal activity recognition requires detection of occurrence of anomalous events that suffer from a severe imbalance in data. In a video, normal is used to describe activities that conform to usual events while the irregular events which do not conform to the normal are referred to as abnormal. It is far more common to observe normal data than to obtain abnormal data in visual surveillance. In this paper, we propose an approach where we can obtain abnormal data by transforming normal data. This is a challenging task that is solved through a multi-stage pipeline approach. We utilize a number of techniques from unsupervised segmentation in order to synthesize new samples of data that are transformed from an existing set of normal examples. Further, this synthesis approach has useful applications as a data augmentation technique. An incrementally trained Bayesian convolutional neural network (CNN) is used to carefully select the set of abnormal samples that can be added. Finally through this synthesis approach we obtain a comparable set of abnormal samples that can be used for training the CNN for the classification of normal vs abnormal samples. We show that this method generalizes to multiple settings by evaluating it on two real world datasets and achieves improved performance over other probabilistic techniques that have been used in the past for this task.
Timing in Software-Defined and Centrally-Managed Networks
Venue: arXiv cs.NI 2019
Year: 2019

The work described in this paper explores the use of time and synchronized clocks in centrally-managed and Software Defined Networks (SDNs). One of the main goals of this work is to analyze use cases in which explicit use of time is beneficial. Both theoretical and practical aspects of timed coordination and synchronized clocks in centralized environments are analyzed. Some of the products of this work are already incorporated in the OpenFlow specification, and open source prototypes of the main components are publicly available.
From Theory to Systems: A Grounded Approach to Programming Language Education
Venue: arXiv cs.PL 2019
Year: 2019

I present a new approach to teaching a graduate-level programming languages course focused on using systems programming ideas and languages like WebAssembly and Rust to motivate PL theory. Drawing on students' prior experience with low-level languages, the course shows how type systems and PL theory are used to avoid tricky real-world errors that students encounter in practice. I reflect on the curricular design and lessons learned from two years of teaching at Stanford, showing that integrating systems ideas can provide students a more grounded and enjoyable education in programming languages. The curriculum, course notes, and assignments are freely available: http://cs242.stanford.edu/f18/
Differential Privacy for Eye-Tracking Data
Venue: arXiv cs.CR 2019
Year: 2019

As large eye-tracking datasets are created, data privacy is a pressing concern for the eye-tracking community. De-identifying data does not guarantee privacy because multiple datasets can be linked for inferences. A common belief is that aggregating individuals' data into composite representations such as heatmaps protects the individual. However, we analytically examine the privacy of (noise-free) heatmaps and show that they do not guarantee privacy. We further propose two noise mechanisms that guarantee privacy and analyze their privacy-utility tradeoff. Analysis reveals that our Gaussian noise mechanism is an elegant solution to preserve privacy for heatmaps. Our results have implications for interdisciplinary research to create differentially private mechanisms for eye tracking.
Formalization of Automated Trading Systems in a Concurrent Linear Framework
Venue: arXiv cs.LO 2019
Year: 2019

We present a declarative and modular specification of an automated trading system (ATS) in the concurrent linear framework CLF. We implemented it in Celf, a CLF type checker which also supports executing CLF specifications. We outline the verification of two representative properties of trading systems using generative grammars, an approach to reasoning about CLF specifications.
DuBox: No-Prior Box Objection Detection via Residual Dual Scale Detectors
Venue: arXiv cs.CV 2019
Year: 2019

Traditional neural objection detection methods use multi-scale features that allow multiple detectors to perform detecting tasks independently and in parallel. At the same time, with the handling of the prior box, the algorithm's ability to deal with scale invariance is enhanced. However, too many prior boxes and independent detectors will increase the computational redundancy of the detection algorithm. In this study, we introduce Dubox, a new one-stage approach that detects the objects without prior box. Working with multi-scale features, the designed dual scale residual unit makes dual scale detectors no longer run independently. The second scale detector learns the residual of the first. Dubox has enhanced the capacity of heuristic-guided that can further enable the first scale detector to maximize the detection of small targets and the second to detect objects that cannot be identified by the first one. Besides, for each scale detector, with the new classification-regression progressive strapped loss makes our process not based on prior boxes. Integrating these strategies, our detection algorithm has achieved excellent performance in terms of speed and accuracy. Extensive experiments on the VOC, COCO object detection benchmark have confirmed the effectiveness of this algorithm.
Learning to Guide: Guidance Law Based on Deep Meta-learning and Model Predictive Path Integral Control
Venue: arXiv cs.RO 2019
Year: 2019

In this paper, we present a novel guidance scheme based on model-based deep reinforcement learning (RL) technique. With model-based deep RL method, a deep neural network is trained as a predictive model of guidance dynamics which is incorporated into a model predictive path integral (MPPI) control framework. However the traditional MPPI framework assumes the actual environment similar to the training dataset for the deep neural network which is impractical in practice with different maneuvering of target, other perturbations and actuator failures. To address this problem, our method utilize meta-learning technique to make the deep neural dynamics model adapt to such changes online. With this approach we can alleviate the performance deterioration of standard MPPI control caused by the difference between actual environment and training data. Then, a novel guidance law for a varying velocity interceptor intercepting maneuvering target with desired terminal impact angle under actuator failure is constructed based on aforementioned techniques. Simulation and experiment results under different cases show the effectiveness and robustness of the proposed guidance law in achieving successful interceptions of maneuvering target.
Coverage Analysis of 3-D Dense Cellular Networks with Realistic Propagation Conditions
Venue: arXiv cs.IT 2019
Year: 2019

In recent times, the use of stochastic geometry has become a popular and important tool for performance analysis of next-generation dense small cell wireless networks. Usually, such networks are modeled using 2 dimensional spatial Poisson point processes (SPPP). Moreover, the distinctive effects of line-of-sight (LOS) and non-line-of-sight (NLOS) propagation are also not explicitly taken into account in such analysis. The aim of the current work is to bridge this gap by modeling the access point (AP) and user equipment (UE) locations by 3-dimensional SPPP and considering the realistic LOS/NLOS channel models (path loss and small scale fading) as reported in existing standards. The effect of UE density on downlink coverage probability has also been investigated. In this process, the probabilistic activity of APs has been analytically modeled as a function of AP and UE densities. The derived upper bound of coverage probability is found to be numerically simple as well as extremely tight in nature and thus can be used as a close approximation of the same.
Efficient Feature Selection of Power Quality Events using Two Dimensional (2D) Particle Swarms
Venue: arXiv cs.NE 2019
Year: 2019

A novel two-dimensional (2D) learning framework has been proposed to address the feature selection problem in Power Quality (PQ) events. Unlike the existing feature selection approaches, the proposed 2D learning explicitly incorporates the information about the subset cardinality (i.e., the number of features) as an additional learning dimension to effectively guide the search process. The efficacy of this approach has been demonstrated considering fourteen distinct classes of PQ events which conform to the IEEE Standard 1159. The search performance of the 2D learning approach has been compared to the other six well-known feature selection wrappers by considering two induction algorithms: Naive Bayes (NB) and k-Nearest Neighbors (k-NN). Further, the robustness of the selected/reduced feature subsets has been investigated considering seven different levels of noise. The results of this investigation convincingly demonstrate that the proposed 2D learning can identify significantly better and robust feature subsets for PQ events.
Fatigue design load calculations of the offshore NREL 5MW benchmark turbine using quadrature rule techniques
Venue: arXiv cs.CE 2019
Year: 2019

A novel approach is proposed to reduce, compared to the conventional binning approach, the large number of aeroelastic code evaluations that are necessary to obtain equivalent loads acting on wind turbines. These loads describe the effect of long-term environmental variability on the fatigue loads of a horizontal-axis wind turbine. In particular Design Load Case 1.2, as standardized by IEC, is considered. The approach is based on numerical integration techniques and, more specifically, quadrature rules. The quadrature rule used in this work is a recently proposed "implicit" quadrature rule, which has the main advantage that it can be constructed directly using measurements of the environment. It is demonstrated that the proposed approach yields accurate estimations of the equivalent loads using a significantly reduced number of aeroelastic model evaluations (compared to binning). Moreover the error introduced by the seeds (introduced by averaging over random wind fields and sea states) is incorporated in the quadrature framework, yielding an even further reduction in the number of aeroelastic code evaluations. The reduction in computational time is demonstrated by assessing the fatigue loads on the NREL 5MW reference offshore wind turbine in conjunction with measurement data obtained at the North Sea, both for a simplified and a full load case.
A deep learning framework for quality assessment and restoration in video endoscopy
Venue: arXiv cs.CV 2019
Year: 2019

Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, we contend that the robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution. We propose a fully automatic framework that can: 1) detect and classify six different primary artifacts, 2) provide a quality score for each frame and 3) restore mildly corrupted frames. To detect different artifacts our framework exploits fast multi-scale, single stage convolutional neural network detector. We introduce a quality metric to assess frame quality and predict image restoration success. Generative adversarial networks with carefully chosen regularization are finally used to restore corrupted frames. Our detector yields the highest mean average precision (mAP at 5% threshold) of 49.0 and the lowest computational time of 88 ms allowing for accurate real-time processing. Our restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos we show that our approach preserves an average of 68.7% which is 25% more frames than that retained from the raw videos.