OpenRaiser/Intern-Atlas huggingface dataset:
Ten Simple Rules for Reproducible Research in Jupyter Notebooks
Venue: arXiv cs.OH 2018
Year: 2018

Reproducibility of computational studies is a hallmark of scientific methodology. It enables researchers to build with confidence on the methods and findings of others, reuse and extend computational pipelines, and thereby drive scientific progress. Since many experimental studies rely on computational analyses, biologists need guidance on how to set up and document reproducible data analyses or simulations. In this paper, we address several questions about reproducibility. For example, what are the technical and non-technical barriers to reproducible computational studies? What opportunities and challenges do computational notebooks offer to overcome some of these barriers? What tools are available and how can they be used effectively? We have developed a set of rules to serve as a guide to scientists with a specific focus on computational notebook systems, such as Jupyter Notebooks, which have become a tool of choice for many applications. Notebooks combine detailed workflows with narrative text and visualization of results. Combined with software repositories and open source licensing, notebooks are powerful tools for transparent, collaborative, reproducible, and reusable data analyses.
She's Reddit: A source of statistically significant gendered interest information?
Venue: arXiv cs.CY 2018
Year: 2018

Information about gender differences in interests is necessary to disentangle the effects of discrimination and choice when gender inequalities occur, such as in employment. This article assesses gender differences in interests within the popular social news and entertainment site Reddit. A method to detect terms that are statistically significantly used more by males or females in 181 million comments in 100 subreddits shows that gender affects both the selection of subreddits and activities within most of them. The method avoids the hidden gender biases of topic modelling for this task. Although the method reveals statistically significant gender differences in interests for topics that are extensively discussed on Reddit, it cannot give definitive causes, and imitation and sharing within the site mean that additional checking is needed to verify the results. Nevertheless, with care, Reddit can serve as a useful source of insights into gender differences in interests.
Stochastic Distance Transform
Venue: arXiv cs.CV 2018
Year: 2018

The distance transform (DT) and its many variations are ubiquitous tools for image processing and analysis. In many imaging scenarios, the images of interest are corrupted by noise. This has a strong negative impact on the accuracy of the DT, which is highly sensitive to spurious noise points. In this study, we consider images represented as discrete random sets and observe statistics of DT computed on such representations. We, thus, define a stochastic distance transform (SDT), which has an adjustable robustness to noise. Both a stochastic Monte Carlo method and a deterministic method for computing the SDT are proposed and compared. Through a series of empirical tests, we demonstrate that the SDT is effective not only in improving the accuracy of the computed distances in the presence of noise, but also in improving the performance of template matching and watershed segmentation of partially overlapping objects, which are examples of typical applications where DTs are utilized.
DeepLens: Shallow Depth Of Field From A Single Image
Venue: arXiv cs.CV 2018
Year: 2018

We aim to generate high resolution shallow depth-of-field (DoF) images from a single all-in-focus image with controllable focal distance and aperture size. To achieve this, we propose a novel neural network model comprised of a depth prediction module, a lens blur module, and a guided upsampling module. All modules are differentiable and are learned from data. To train our depth prediction module, we collect a dataset of 2462 RGB-D images captured by mobile phones with a dual-lens camera, and use existing segmentation datasets to improve border prediction. We further leverage a synthetic dataset with known depth to supervise the lens blur and guided upsampling modules. The effectiveness of our system and training strategies are verified in the experiments. Our method can generate high-quality shallow DoF images at high resolution, and produces significantly fewer artifacts than the baselines and existing solutions for single image shallow DoF synthesis. Compared with the iPhone portrait mode, which is a state-of-the-art shallow DoF solution based on a dual-lens depth camera, our method generates comparable results, while allowing for greater flexibility to choose focal points and aperture size, and is not limited to one capture setup.
Approximate Dynamic Programming for Planning a Ride-Sharing System using Autonomous Fleets of Electric Vehicles
Venue: arXiv cs.AI 2018
Year: 2018

Within a decade, almost every major auto company, along with fleet operators such as Uber, have announced plans to put autonomous vehicles on the road. At the same time, electric vehicles are quickly emerging as a next-generation technology that is cost effective, in addition to offering the benefits of reducing the carbon footprint. The combination of a centrally managed fleet of driverless vehicles, along with the operating characteristics of electric vehicles, is creating a transformative new technology that offers significant cost savings with high service levels. This problem involves a dispatch problem for assigning riders to cars, a surge pricing problem for deciding on the price per trip and a planning problem for deciding on the fleet size. We use approximate dynamic programming to develop high-quality operational dispatch strategies to determine which car is best for a particular trip, when a car should be recharged, and when it should be re-positioned to a different zone which offers a higher density of trips. We prove that the value functions are monotone in the battery and time dimensions and use hierarchical aggregation to get better estimates of the value functions with a small number of observations. Then, surge pricing is discussed using an adaptive learning approach to decide on the price for each trip. Finally, we discuss the fleet size problem which depends on the previous two problems.
Private Machine Learning in TensorFlow using Secure Computation
Venue: arXiv cs.CR 2018
Year: 2018

We present a framework for experimenting with secure multi-party computation directly in TensorFlow. By doing so we benefit from several properties valuable to both researchers and practitioners, including tight integration with ordinary machine learning processes, existing optimizations for distributed computation in TensorFlow, high-level abstractions for expressing complex algorithms and protocols, and an expanded set of familiar tooling. We give an open source implementation of a state-of-the-art protocol and report on concrete benchmarks using typical models from private machine learning.
Contextual Topic Modeling For Dialog Systems
Venue: arXiv cs.CL 2018
Year: 2018

Accurate prediction of conversation topics can be a valuable signal for creating coherent and engaging dialog systems. In this work, we focus on context-aware topic classification methods for identifying topics in free-form human-chatbot dialogs. We extend previous work on neural topic classification and unsupervised topic keyword detection by incorporating conversational context and dialog act features. On annotated data, we show that incorporating context and dialog acts leads to relative gains in topic classification accuracy by 35% and on unsupervised keyword detection recall by 11% for conversational interactions where topics frequently span multiple utterances. We show that topical metrics such as topical depth is highly correlated with dialog evaluation metrics such as coherence and engagement implying that conversational topic models can predict user satisfaction. Our work for detecting conversation topics and keywords can be used to guide chatbots towards coherent dialog.
Deep Learning vs. Human Graders for Classifying Severity Levels of Diabetic Retinopathy in a Real-World Nationwide Screening Program
Venue: arXiv cs.CV 2018
Year: 2018

Deep learning algorithms have been used to detect diabetic retinopathy (DR) with specialist-level accuracy. This study aims to validate one such algorithm on a large-scale clinical population, and compare the algorithm performance with that of human graders. 25,326 gradable retinal images of patients with diabetes from the community-based, nation-wide screening program of DR in Thailand were analyzed for DR severity and referable diabetic macular edema (DME). Grades adjudicated by a panel of international retinal specialists served as the reference standard. Across different severity levels of DR for determining referable disease, deep learning significantly reduced the false negative rate (by 23%) at the cost of slightly higher false positive rates (2%). Deep learning algorithms may serve as a valuable tool for DR screening.
Compositional Verification for Autonomous Systems with Deep Learning Components
Venue: arXiv cs.AI 2018
Year: 2018

As autonomy becomes prevalent in many applications, ranging from recommendation systems to fully autonomous vehicles, there is an increased need to provide safety guarantees for such systems. The problem is difficult, as these are large, complex systems which operate in uncertain environments, requiring data-driven machine-learning components. However, learning techniques such as Deep Neural Networks, widely used today, are inherently unpredictable and lack the theoretical foundations to provide strong assurance guarantees. We present a compositional approach for the scalable, formal verification of autonomous systems that contain Deep Neural Network components. The approach uses assume-guarantee reasoning whereby {\em contracts}, encoding the input-output behavior of individual components, allow the designer to model and incorporate the behavior of the learning-enabled components working side-by-side with the other components. We illustrate the approach on an example taken from the autonomous vehicles domain.
Reduction of Parameter Redundancy in Biaffine Classifiers with Symmetric and Circulant Weight Matrices
Venue: arXiv cs.CL 2018
Year: 2018

Currently, the biaffine classifier has been attracting attention as a method to introduce an attention mechanism into the modeling of binary relations. For instance, in the field of dependency parsing, the Deep Biaffine Parser by Dozat and Manning has achieved state-of-the-art performance as a graph-based dependency parser on the English Penn Treebank and CoNLL 2017 shared task. On the other hand, it is reported that parameter redundancy in the weight matrix in biaffine classifiers, which has O(n^2) parameters, results in overfitting (n is the number of dimensions). In this paper, we attempted to reduce the parameter redundancy by assuming either symmetry or circularity of weight matrices. In our experiments on the CoNLL 2017 shared task dataset, our model achieved better or comparable accuracy on most of the treebanks with more than 16% parameter reduction.
Sequenced-Replacement Sampling for Deep Learning
Venue: arXiv cs.LG 2018
Year: 2018

We propose sequenced-replacement sampling (SRS) for training deep neural networks. The basic idea is to assign a fixed sequence index to each sample in the dataset. Once a mini-batch is randomly drawn in each training iteration, we refill the original dataset by successively adding samples according to their sequence index. Thus we carry out replacement sampling but in a batched and sequenced way. In a sense, SRS could be viewed as a way of performing "mini-batch augmentation". It is particularly useful for a task where we have a relatively small images-per-class such as CIFAR-100. Together with a longer period of initial large learning rate, it significantly improves the classification accuracy in CIFAR-100 over the current state-of-the-art results. Our experiments indicate that training deeper networks with SRS is less prone to over-fitting. In the best case, we achieve an error rate as low as 10.10%.
NimbRo-OP2X: Adult-sized Open-source 3D Printed Humanoid Robot
Venue: arXiv cs.RO 2018
Year: 2018

Humanoid robotics research depends on capable robot platforms, but recently developed advanced platforms are often not available to other research groups, expensive, dangerous to operate, or closed-source. The lack of available platforms forces researchers to work with smaller robots, which have less strict dynamic constraints or with simulations, which lack many real-world effects. We developed NimbRo-OP2X to address this need. At a height of 135 cm our robot is large enough to interact in a human environment. Its low weight of only 19 kg makes the operation of the robot safe and easy, as no special operational equipment is necessary. Our robot is equipped with a fast onboard computer and a GPU to accelerate parallel computations. We extend our already open-source software by a deep-learning based vision system and gait parameter optimisation. The NimbRo-OP2X was evaluated during RoboCup 2018 in Montréal, Canada, where it won all possible awards in the Humanoid AdultSize class.
Assumption-Based Planning
Venue: arXiv cs.AI 2018
Year: 2018

The purpose of the paper is to introduce a new approach of planning called Assumption-Based Planning. This approach is a very interesting way to devise a planner based on a multi-agent system in which the production of a global shared plan is obtained by conjecture/refutation cycles. Contrary to classical approaches, our contribution relies on the agents reasoning that leads to the production of a plan from planning domains. To take into account complex environments and the partial agents knowledge, we propose to consider the planning problem as a defeasible reasoning where the agents exchange proposals and counter-proposals and are able to reason about uncertainty. The argumentation dialogue between agents must not be viewed as a negotiation process but as an investigation process in order to build a plan. In this paper, we focus on the mechanisms that allow an agent to produce `reasonable' proposals according to its knowledge.
New and Simplified Distributed Algorithms for Weighted All Pairs Shortest Paths
Venue: arXiv cs.DS 2018
Year: 2018

We consider the problem of computing all pairs shortest paths (APSP) and shortest paths for k sources in a weighted graph in the distributed CONGEST model. For graphs with non-negative integer edge weights (including zero weights) we build on a recent pipelined algorithm to obtain $\tilde{O}(λ^{1/4}\cdot n^{5/4})$ in graphs with non-negative integer edge-weight at most $λ$, and $\tilde{O}(n \cdot \bigtriangleup^{1/3})$ rounds for shortest path distances at most $\bigtriangleup$. Additionally, we simplify some of the procedures in the earlier APSP algorithms for non-negative edge weights in [HNS17,ARKP18]. We also present results for computing h-hop shortest paths and shortest paths from $k$ given sources. In other results, we present a randomized exact APSP algorithm for graphs with arbitrary edge weights that runs in $\tilde{O}(n^{4/3})$ rounds w.h.p. in n, which improves the previous best $\tilde{O}(n^{3/2})$ bound, which is deterministic. We also present an $\tilde{O}(n/ε^2)$-round deterministic $(1+ε)$ approximation algorithm for graphs with non-negative $poly(n)$ integer weights (including zero edge-weights), improving results in [Nanongkai14,LP15] that hold only for positive integer weights.
Security-Reliability Tradeoff for Distributed Antenna Systems in Heterogeneous Cellular Networks
Venue: arXiv cs.IT 2018
Year: 2018

In this paper, we investigate physical-layer security for a spectrum-sharing heterogeneous cellular network comprised of a macro cell and a small cell, where a passive eavesdropper is assumed to tap the transmissions of both the macro cell and small cell. In the macro cell, a macro base station (MBS) equipped with multiple distributed antennas sends its confidential information to a macro user (MU) through an opportunistic transmit antenna. Meanwhile, in the small cell, a small base station (SBS) transmits to a small user (SU) over the same spectrum used by MBS. We propose an interference-canceled opportunistic antenna selection (IC-OAS) scheme to enhance physical-layer security for the heterogeneous network. To be specific, when MBS sends its confidential message to MU through an opportunistic distributed antenna, a special signal is artificially designed and emitted at MBS to ensure that the received interference at MU from SBS is canceled out. For comparison, the conventional interference-limited opportunistic antenna selection (IL-OAS) is considered as a benchmark. We characterize the security-reliability tradeoff (SRT) for the proposed IC-OAS and conventional IL-OAS schemes in terms of deriving their closed-form expressions of intercept probability and outage probability. Numerical results show that compared with the conventional IL-OAS, the proposed IC-OAS scheme not only brings SRT benefits to the macro cell, but also has the potential of improving the SRT of small cell by increasing the number of distributed antennas. Additionally, by jointly taking into account the macro cell and small cell, an overall SRT of the proposed IC-OAS scheme is shown to be significantly better than that of the conventional IL-OAS approach in terms of a sum intercept probability versus sum outage probability.
Autonomous Functional Locomotion in a Tendon-Driven Limb via Limited Experience
Venue: arXiv cs.RO 2018
Year: 2018

Robots will become ubiquitously useful only when they can use few attempts to teach themselves to perform different tasks, even with complex bodies and in dynamical environments. Vertebrates, in fact, successfully use trial-and-error to learn multiple tasks in spite of their intricate tendon-driven anatomies. Roboticists find such tendon-driven systems particularly hard to control because they are simultaneously nonlinear, under-determined (many tendon tensions combine to produce few net joint torques), and over-determined (few joint rotations define how many tendons need to be reeled-in/payed-out). We demonstrate---for the first time in simulation and in hardware---how a model-free approach allows few-shot autonomous learning to produce effective locomotion in a 3-tendon/2-joint tendon-driven leg. Initially, an artificial neural network fed by sparsely sampled data collected using motor babbling creates an inverse map from limb kinematics to motor activations, which is analogous to juvenile vertebrates playing during development. Thereafter, iterative reward-driven exploration of candidate motor activations simultaneously refines the inverse map and finds a functional locomotor limit-cycle autonomously. This biologically-inspired algorithm, which we call G2P (General to Particular), enables versatile adaptation of robots to changes in the target task, mechanics of their bodies, and environment. Moreover, this work empowers future studies of few-shot autonomous learning in biological systems, which is the foundation of their enviable functional versatility.
Design Space Exploration of Neural Network Activation Function Circuits
Venue: arXiv cs.NE 2018
Year: 2018

The widespread application of artificial neural networks has prompted researchers to experiment with FPGA and customized ASIC designs to speed up their computation. These implementation efforts have generally focused on weight multiplication and signal summation operations, and less on activation functions used in these applications. Yet, efficient hardware implementations of nonlinear activation functions like Exponential Linear Units (ELU), Scaled Exponential Linear Units (SELU), and Hyperbolic Tangent (tanh), are central to designing effective neural network accelerators, since these functions require lots of resources. In this paper, we explore efficient hardware implementations of activation functions using purely combinational circuits, with a focus on two widely used nonlinear activation functions, i.e., SELU and tanh. Our experiments demonstrate that neural networks are generally insensitive to the precision of the activation function. The results also prove that the proposed combinational circuit-based approach is very efficient in terms of speed and area, with negligible accuracy loss on the MNIST, CIFAR-10 and IMAGENET benchmarks. Synopsys Design Compiler synthesis results show that circuit designs for tanh and SELU can save between 3.13-7.69 and 4.45-8:45 area compared to the LUT/memory-based implementations, and can operate at 5.14GHz and 4.52GHz using the 28nm SVT library, respectively. The implementation is available at: https://github.com/ThomasMrY/ActivationFunctionDemo.
Ockham's Razor in Memetic Computing: Three Stage Optimal Memetic Exploration
Venue: arXiv cs.LG 2018
Year: 2018

Memetic Computing is a subject in computer science which considers complex structures as the combination of simple agents, memes, whose evolutionary interactions lead to intelligent structures capable of problem-solving. This paper focuses on Memetic Computing optimization algorithms and proposes a counter-tendency approach for algorithmic design. Research in the field tends to go in the direction of improving existing algorithms by combining different methods or through the formulation of more complicated structures. Contrary to this trend, we instead focus on simplicity, proposing a structurally simple algorithm with emphasis on processing only one solution at a time. The proposed algorithm, namely Three Stage Optimal Memetic Exploration, is composed of three memes; the first stochastic and with a long search radius, the second stochastic and with a moderate search radius and the third deterministic and with a short search radius. The bottom-up combination of the three operators by means of a natural trial and error logic, generates a robust and efficient optimizer, capable of competing with modern complex and computationally expensive algorithms. This is suggestive of the fact that complexity in algorithmic structures can be unnecessary, if not detrimental, and that simple bottom-up approaches are likely to be competitive is here invoked as an extension to Memetic Computing basing on the philosophical concept of Ockham's Razor. An extensive experimental setup on various test problems and one digital signal processing application is presented. Numerical results show that the proposed approach, despite its simplicity and low computational cost displays a very good performance on several problems, and is competitive with sophisticated algorithms representing the-state-of-the-art in computational intelligence optimization.
Lightweight Convolutional Approaches to Reading Comprehension on SQuAD
Venue: arXiv cs.CL 2018
Year: 2018

Current state-of-the-art reading comprehension models rely heavily on recurrent neural networks. We explored an entirely different approach to question answering: a convolutional model. By their nature, these convolutional models are fast to train and capture local dependencies well, though they can struggle with longer-range dependencies and thus require augmentation to achieve comparable performance to RNN-based models. We conducted over two dozen controlled experiments with convolutional models and various kernel/attention/regularization schemes to determine the precise performance gains of each strategy, while maintaining a focus on speed. We ultimately ensembled three models: crossconv (0.5398 dev F1), attnconv (0.5665), and maybeconv (0.5285). The ensembled model was able to achieve a 0.6238 F1 score using the official SQuAD evaluation script. Our individual convolutional model crossconv was able to exceed the performance of the RNN-plus-attention baseline by 25% while training 6 times faster.