OpenRaiser/Intern-Atlas huggingface dataset:
Static Analysis for Probabilistic Programs
Venue: arXiv cs.PL 2019
Year: 2019

Probabilistic programming is a powerful abstraction for statistical machine learning. Applying static analysis methods to probabilistic programs could serve to optimize the learning process, automatically verify properties of models, and improve the programming interface for users. This field of static analysis for probabilistic programming (SAPP) is young and unorganized, consisting of a constellation of techniques with various goals and limitations. The primary aim of this work is to synthesize the major contributions of the SAPP field within an organizing structure and context. We provide technical background for static analysis and probabilistic programming, suggest a functional taxonomy for probabilistic programming languages, and analyze the applicability of major ideas in the SAPP field. We conclude that, while current static analysis techniques for probabilistic programs have practical limitations, there are a number of future directions with high potential to improve the state of statistical machine learning.
Sociotechnical Considerations for Accessible Visualization Design
Venue: arXiv cs.CY 2019
Year: 2019

Accessibility--the process of designing for people with disabilities (PWD)--is an important but under-explored challenge in the visualization research community. Without careful attention, and if PWD are not included as equal participants throughout the process, there is a danger of perpetuating a vision-first approach to accessible design that marginalizes the lived experience of disability (e.g., by creating overly simplistic "sensory translations" that map visual to non-visual modalities in a one-to-one fashion). In this paper, we present a set of sociotechnical considerations for research in accessible visualization design, drawing on literature in disability studies, tactile information systems, and participatory methods. We identify that using state-of-the-art technologies may introduce more barriers to access than they remove, and that expectations of research novelty may not produce outcomes well-aligned with the needs of disability communities. Instead, to promote a more inclusive design process, we emphasize the importance of clearly communicating goals, following existing accessibility guidelines, and treating PWD as equal participants who are compensated for their specialized skills. To illustrate how these considerations can be applied in practice, we discuss a case study of an inclusive design workshop held in collaboration with the Perkins School for the Blind.
Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees
Venue: arXiv cs.LO 2019
Year: 2019

Reinforcement Learning (RL) has emerged as an efficient method of choice for solving complex sequential decision making problems in automatic control, computer science, economics, and biology. In this paper we present a model-free RL algorithm to synthesize control policies that maximize the probability of satisfying high-level control objectives given as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace properties, the structure of the workspace, and the agent actions, giving rise to a Probabilistically-Labeled Markov Decision Process (PL-MDP) with unknown graph structure and stochastic behaviour, which is even more general case than a fully unknown MDP. We first translate the LTL specification into a Limit Deterministic Buchi Automaton (LDBA), which is then used in an on-the-fly product with the PL-MDP. Thereafter, we define a synchronous reward function based on the acceptance condition of the LDBA. Finally, we show that the RL algorithm delivers a policy that maximizes the satisfaction probability asymptotically. We provide experimental results that showcase the efficiency of the proposed method.
An Auxiliary Classifier Generative Adversarial Framework for Relation Extraction
Venue: arXiv cs.CL 2019
Year: 2019

Relation extraction models suffer from limited qualified training data. Using human annotators to label sentences is too expensive and does not scale well especially when dealing with large datasets. In this paper, we use Auxiliary Classifier Generative Adversarial Networks (AC-GANs) to generate high-quality relational sentences and to improve the performance of relation classifier in end-to-end models. In AC-GAN, the discriminator gives not only a probability distribution over the real source, but also a probability distribution over the relation labels. This helps to generate meaningful relational sentences. Experimental results show that our proposed data augmentation method significantly improves the performance of relation extraction compared to state-of-the-art methods
Overton: A Data System for Monitoring and Improving Machine-Learned Products
Venue: arXiv cs.LG 2019
Year: 2019

We describe a system called Overton, whose main design goal is to support engineers in building, monitoring, and improving production machine learning systems. Key challenges engineers face are monitoring fine-grained quality, diagnosing errors in sophisticated applications, and handling contradictory or incomplete supervision data. Overton automates the life cycle of model construction, deployment, and monitoring by providing a set of novel high-level, declarative abstractions. Overton's vision is to shift developers to these higher-level tasks instead of lower-level machine learning tasks. In fact, using Overton, engineers can build deep-learning-based applications without writing any code in frameworks like TensorFlow. For over a year, Overton has been used in production to support multiple applications in both near-real-time applications and back-of-house processing. In that time, Overton-based applications have answered billions of queries in multiple languages and processed trillions of records reducing errors 1.7-2.9 times versus production systems.
Selecting Data to Clean for Fact Checking: Minimizing Uncertainty vs. Maximizing Surprise
Venue: arXiv cs.DB 2019
Year: 2019

We study the optimization problem of selecting numerical quantities to clean in order to fact-check claims based on such data. Oftentimes, such claims are technically correct, but they can still mislead for two reasons. First, data may contain uncertainty and errors. Second, data can be "fished" to advance particular positions. In practice, fact-checkers cannot afford to clean all data and must choose to clean what "matters the most" to checking a claim. We explore alternative definitions of what "matters the most": one is to ascertain claim qualities (by minimizing uncertainty in these measures), while an alternative is just to counter the claim (by maximizing the probability of finding a counterargument). We show whether the two objectives align with each other, with important implications on when fact-checkers should exercise care in selective data cleaning, to avoid potential bias introduced by their desire to counter claims. We develop efficient algorithms for solving the various variants of the optimization problem, showing significant improvements over naive solutions. The problem is particularly challenging because the objectives in the fact-checking context are complex, non-linear functions over data. We obtain results that generalize to a large class of functions, with potential applications beyond fact-checking.
Tiny Buffer TCP for Data Center Networks
Venue: arXiv cs.NI 2019
Year: 2019

A low and stable buffer occupancy is critical to achieve high throughput, low packet drop rate, low latency, and low jitter for data center networks. It also allows switch chips to support higher port density, larger lookup tables, or richer functions. Tiny Buffer TCP creatively uses the common RED-based ECN with two novel congestion-window adjustment schemes to significantly reduce the required buffer size. Aiming to eliminate the residual packets in the bottleneck queue, Queue Canceling Decrease amortizes the ideal window reduction to the same number of flows so as to minimize the impact to active flows. In order to keep the buffer occupancy low and stable, Reduced Additive Increase recovers the flow window at a slower pace than normal. We implemented TBTCP in Linux kernel and conducted ns2-based simulations and real network-based tests. Our results show that compared to DCTCP, TBTCP reduces the switch buffer requirement by more than 80%, increases the bandwidth utilization by up to 15%, improves the FCT performance by up to 39%, and achieve a 71% better RTT fairness index.
Automated Blood Cell Detection and Counting via Deep Learning for Microfluidic Point-of-Care Medical Devices
Venue: arXiv cs.CV 2019
Year: 2019

Automated in-vitro cell detection and counting have been a key theme for artificial and intelligent biological analysis such as biopsy, drug analysis and decease diagnosis. Along with the rapid development of microfluidics and lab-on-chip technologies, in-vitro live cell analysis has been one of the critical tasks for both research and industry communities. However, it is a great challenge to obtain and then predict the precise information of live cells from numerous microscopic videos and images. In this paper, we investigated in-vitro detection of white blood cells using deep neural networks, and discussed how state-of-the-art machine learning techniques could fulfil the needs of medical diagnosis. The approach we used in this study was based on Faster Region-based Convolutional Neural Networks (Faster RCNNs), and a transfer learning process was applied to apply this technique to the microscopic detection of blood cells. Our experimental results demonstrated that fast and efficient analysis of blood cells via automated microscopic imaging can achieve much better accuracy and faster speed than the conventionally applied methods, implying a promising future of this technology to be applied to the microfluidic point-of-care medical devices.
Feedback Learning for Improving the Robustness of Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Recent research studies revealed that neural networks are vulnerable to adversarial attacks. State-of-the-art defensive techniques add various adversarial examples in training to improve models' adversarial robustness. However, these methods are not universal and can't defend unknown or non-adversarial evasion attacks. In this paper, we analyze the model robustness in the decision space. A feedback learning method is then proposed, to understand how well a model learns and to facilitate the retraining process of remedying the defects. The evaluations according to a set of distance-based criteria show that our method can significantly improve models' accuracy and robustness against different types of evasion attacks. Moreover, we observe the existence of inter-class inequality and propose to compensate it by changing the proportions of examples generated in different classes.
Feature Engineering and Forecasting via Derivative-free Optimization and Ensemble of Sequence-to-sequence Networks with Applications in Renewable Energy
Venue: arXiv cs.LG 2019
Year: 2019

This study introduces a framework for the forecasting, reconstruction and feature engineering of multivariate processes along with its renewable energy applications. We integrate derivative-free optimization with an ensemble of sequence-to-sequence networks and design a new resampling technique called additive resampling, which, along with Bootstrap aggregating (bagging) resampling, are applied to initialize the ensemble structure. Moreover, we explore the proposed framework performance on three renewable energy sources---wind, solar and ocean wave---and conduct several short- to long-term forecasts showing the superiority of the proposed method compared to numerous machine learning techniques. The findings indicate that the introduced method performs more accurately when the forecasting horizon becomes longer. In addition, we modify the framework for automated feature selection. The model represents a clear interpretation of the selected features. Furthermore, we investigate the effects of different environmental and marine factors on the wind speed and ocean output power, respectively, and report the selected features. Finally, we explore the online forecasting setting and illustrate that the model outperforms alternatives through different measurement errors.
On the similarities and differences between the Cloud, Fog and the Edge
Venue: arXiv cs.DC 2019
Year: 2019

The field of edge and fog computing is growing, but there are still many inconsistent and loosely-defined terms in current literature. With many articles comparing theoretical architectures and evaluating implementations, there is a need to understand the underlying meaning of information condensed into fog, edge, and similar terms. Through our review of current literature, we discuss these differences and extract key characteristics for basic concepts that appear throughout. The similarities to existing IaaS, PaaS and SaaS models are presented, contrasted against similar models modified for the specifics of edge devices and workloads. We also evaluate the different aspects existing evaluation and comparison works investigate, including the compute, networking, storage, security, and ease-of-use capabilities of the target implementations. Following that, we make a broad overview of currently available commercial and open-source platforms implementing the edge or fog paradigms, identifying key players, successful niche actors and general trends for feature-level and technical development of these platforms.
Reinforcing Edge Computing with Multipath TCP Enabled Mobile Device Clouds
Venue: arXiv cs.NI 2019
Year: 2019

In recent years, enormous growth has been witnessed in the computational and storage capabilities of mobile devices. However, much of this computational and storage capabilities are not always fully used. On the other hand, popularity of mobile edge computing which aims to replace the traditional centralized powerful cloud with multiple edge servers is rapidly growing. In particular, applications having strict latency requirements can be best served by the mobile edge clouds due to a reduced round-trip delay. In this paper we propose a Multi-Path TCP (MPTCP) enabled mobile device cloud (MDC) as a replacement to the existing TCP based or D2D device cloud techniques, as it effectively makes use of the available bandwidth by providing much higher throughput as well as ensures robust wireless connectivity. We investigate the congestion in mobile-device cloud formation resulting mainly due to the message passing for service providing nodes at the time of discovery, service continuity and formation of cloud composition. We propose a user space agent called congestion handler that enable offloading of packets from one sub-flow to the other under link quality constraints. Further, we discuss the benefits of this design and perform preliminary analysis of the system.
Human-Machine Collaborative Design for Accelerated Design of Compact Deep Neural Networks for Autonomous Driving
Venue: arXiv cs.CV 2019
Year: 2019

An effective deep learning development process is critical for widespread industrial adoption, particularly in the automotive sector. A typical industrial deep learning development cycle involves customizing and re-designing an off-the-shelf network architecture to meet the operational requirements of the target application, leading to considerable trial and error work by a machine learning practitioner. This approach greatly impedes development with a long turnaround time and the unsatisfactory quality of the created models. As a result, a development platform that can aid engineers in greatly accelerating the design and production of compact, optimized deep neural networks is highly desirable. In this joint industrial case study, we study the efficacy of the GenSynth AI-assisted AI design platform for accelerating the design of custom, optimized deep neural networks for autonomous driving through human-machine collaborative design. We perform a quantitative examination by evaluating 10 different compact deep neural networks produced by GenSynth for the purpose of object detection via a NASNet-based user network prototype design, targeted at a low-cost GPU-based accelerated embedded system. Furthermore, we quantitatively assess the talent hours and GPU processing hours used by the GenSynth process and three other approaches based on the typical industrial development process. In addition, we quantify the annual cloud cost savings for comprehensive testing using networks produced by GenSynth. Finally, we assess the usability and merits of the GenSynth process through user feedback. The findings of this case study showed that GenSynth is easy to use and can be effective at accelerating the design and production of compact, customized deep neural network.
Hidden Structure in the Solutions Set of the N Queens Problem
Venue: arXiv cs.AI 2019
Year: 2019

Some preliminary results are reported on the equivalence of any n-queens problem with the roots of a Boolean valued quadratic form via a generic dimensional reduction scheme. It is then proven that the solutions set is encoded in the entries of a special matrix. Further examination reveals a direct association with pointwise Boolean fractal operators applied on certain integer sequences associated with this matrix suggesting the presence of an underlying special geometry of the solutions set.
What can computational models learn from human selective attention? A review from an audiovisual crossmodal perspective
Venue: arXiv cs.CV 2019
Year: 2019

Selective attention plays an essential role in information acquisition and utilization from the environment. In the past 50 years, research on selective attention has been a central topic in cognitive science. Compared with unimodal studies, crossmodal studies are more complex but necessary to solve real-world challenges in both human experiments and computational modeling. Although an increasing number of findings on crossmodal selective attention have shed light on humans' behavioral patterns and neural underpinnings, a much better understanding is still necessary to yield the same benefit for computational intelligent agents. This article reviews studies of selective attention in unimodal visual and auditory and crossmodal audiovisual setups from the multidisciplinary perspectives of psychology and cognitive neuroscience, and evaluates different ways to simulate analogous mechanisms in computational models and robotics. We discuss the gaps between these fields in this interdisciplinary review and provide insights about how to use psychological findings and theories in artificial intelligence from different perspectives.
Adaptive Wasserstein Hourglass for Weakly Supervised Hand Pose Estimation from Monocular RGB
Venue: arXiv cs.CV 2019
Year: 2019

Insufficient labeled training datasets is one of the bottlenecks of 3D hand pose estimation from monocular RGB images. Synthetic datasets have a large number of images with precise annotations, but the obvious difference with real-world datasets impacts the generalization. Little work has been done to bridge the gap between two domains over their wide difference. In this paper, we propose a domain adaptation method called Adaptive Wasserstein Hourglass (AW Hourglass) for weakly-supervised 3D hand pose estimation, which aims to distinguish the difference and explore the common characteristics (e.g. hand structure) of synthetic and real-world datasets. Learning the common characteristics helps the network focus on pose-related information. The similarity of the characteristics makes it easier to enforce domain-invariant constraints. During training, based on the relation between these common characteristics and 3D pose learned from fully-annotated synthetic datasets, it is beneficial for the network to restore the 3D pose of weakly labeled real-world datasets with the aid of 2D annotations and depth images. While in testing, the network predicts the 3D pose with the input of RGB.
Accelerating Training using Tensor Decomposition
Venue: arXiv cs.CV 2019
Year: 2019

Tensor decomposition is one of the well-known approaches to reduce the latency time and number of parameters of a pre-trained model. However, in this paper, we propose an approach to use tensor decomposition to reduce training time of training a model from scratch. In our approach, we train the model from scratch (i.e., randomly initialized weights) with its original architecture for a small number of epochs, then the model is decomposed, and then continue training the decomposed model till the end. There is an optional step in our approach to convert the decomposed architecture back to the original architecture. We present results of using this approach on both CIFAR10 and Imagenet datasets, and show that there can be upto 2x speed up in training time with accuracy drop of upto 1.5% only, and in other cases no accuracy drop. This training acceleration approach is independent of hardware and is expected to have similar speed ups on both CPU and GPU platforms.
Augmented Data Science: Towards Industrialization and Democratization of Data Science
Venue: arXiv cs.AI 2019
Year: 2019

Conversion of raw data into insights and knowledge requires substantial amounts of effort from data scientists. Despite breathtaking advances in Machine Learning (ML) and Artificial Intelligence (AI), data scientists still spend the majority of their effort in understanding and then preparing the raw data for ML/AI. The effort is often manual and ad hoc, and requires some level of domain knowledge. The complexity of the effort increases dramatically when data diversity, both in form and context, increases. In this paper, we introduce our solution, Augmented Data Science (ADS), towards addressing this "human bottleneck" in creating value from diverse datasets. ADS is a data-driven approach and relies on statistics and ML to extract insights from any data set in a domain-agnostic way to facilitate the data science process. Key features of ADS are the replacement of rudimentary data exploration and processing steps with automation and the augmentation of data scientist judgment with automatically-generated insights. We present building blocks of our end-to-end solution and provide a case study to exemplify its capabilities.
E-Crime Legal Brief: A Case Study on Talk Talk Hacking
Venue: arXiv cs.CR 2019
Year: 2019

E-crime has had various definitions for different countries and organisations. There is no universal definition of E-crime and therefore the interpretation is left to cybercrime investigators and judges to apply related crimes to within the scope where possible. E-crime legal brief should include Citation, facts of the case, issues, reasoning, decision of the judges and analysis. The analysis outlines applicable laws in e-crime, case laws relevant to the facts of the case and crime committed.
SQLR: Short-Term Memory Q-Learning for Elastic Provisioning
Venue: arXiv cs.NI 2019
Year: 2019

As more and more application providers transition to the cloud and deliver their services on a Software as a Service (SaaS) basis, cloud providers need to make their provisioning systems agile enough to meet Service Level Agreements. At the same time they should guard against over-provisioning which limits their capacity to accommodate more tenants. To this end we propose SQLR, a dynamic provisioning system employing a customized model-free reinforcement learning algorithm that is capable of reusing contextual knowledge learned from one workload to optimize resource provisioning for other workload patterns. SQLR achieves results comparable to those where resources are unconstrained, with minimal overhead. Our experiments show that we can reduce the amount of provisioned resources by almost 25% with less than 1% overall service unavailability (due to blocking) while delivering similar response times as those of an over-provisioned system.