OpenRaiser/Intern-Atlas huggingface dataset:
RuleMatrix: Visualizing and Understanding Classifiers with Rules
Venue: arXiv cs.LG 2018
Year: 2018

With the growing adoption of machine learning techniques, there is a surge of research interest towards making machine learning systems more transparent and interpretable. Various visualizations have been developed to help model developers understand, diagnose, and refine machine learning models. However, a large number of potential but neglected users are the domain experts with little knowledge of machine learning but are expected to work with machine learning systems. In this paper, we present an interactive visualization technique to help users with little expertise in machine learning to understand, explore and validate predictive models. By viewing the model as a black box, we extract a standardized rule-based knowledge representation from its input-output behavior. We design RuleMatrix, a matrix-based visualization of rules to help users navigate and verify the rules and the black-box model. We evaluate the effectiveness of RuleMatrix via two use cases and a usability study.
Domain Adaptation for Deviating Acquisition Protocols in CNN-based Lesion Classification on Diffusion-Weighted MR Images
Venue: arXiv cs.CV 2018
Year: 2018

End-to-end deep learning improves breast cancer classification on diffusion-weighted MR images (DWI) using a convolutional neural network (CNN) architecture. A limitation of CNN as opposed to previous model-based approaches is the dependence on specific DWI input channels used during training. However, in the context of large-scale application, methods agnostic towards heterogeneous inputs are desirable, due to the high deviation of scanning protocols between clinical sites. We propose model-based domain adaptation to overcome input dependencies and avoid re-training of networks at clinical sites by restoring training inputs from altered input channels given during deployment. We demonstrate the method's significant increase in classification performance and superiority over implicit domain adaptation provided by training-schemes operating on model-parameters instead of raw DWI images.
A Dense CNN approach for skin lesion classification
Venue: arXiv cs.CV 2018
Year: 2018

This article presents a Deep CNN, based on the DenseNet architecture jointly with a highly discriminating learning methodology, in order to classify seven kinds of skin lesions: Melanoma, Melanocytic nevus, Basal cell carcinoma, Actinic keratosis / Bowen's disease, Benign keratosis, Dermatofibroma, Vascular lesion. In particular a 61 layers DenseNet, pre-trained on IMAGENET dataset, has been fine-tuned on ISIC 2018 Task 3 Challenge Dataset exploiting a Center Loss function.
On Ternary Coding and Three-Valued Logic
Venue: arXiv cs.AI 2018
Year: 2018

Mathematically, ternary coding is more efficient than binary coding. It is little used in computation because technology for binary processing is already established and the implementation of ternary coding is more complicated, but remains relevant in algorithms that use decision trees and in communications. In this paper we present a new comparison of binary and ternary coding and their relative efficiencies are computed both for number representation and decision trees. The implications of our inability to use optimal representation through mathematics or logic are examined. Apart from considerations of representation efficiency, ternary coding appears preferable to binary coding in classification of many real-world problems of artificial intelligence (AI) and medicine. We examine the problem of identifying appropriate three classes for domain-specific applications.
Improving the "Correct Eventual Consistency" Tool
Venue: arXiv cs.DC 2018
Year: 2018

Preserving invariants while designing distributed applications under weak consistency models is difficult. The CEC (Correct Eventual Consistency Tool) is meant to aid the application designer in this task. It provides information about the errors during concurrent operations and suggestions on how and where to synchronize operations. This report presents two features of the tool: providing a counterexample for debugging and concurrency control suggestions.
Performance Evaluation and Optimization of LPWA IoT Networks: A Stochastic Geometry Approach
Venue: arXiv cs.IT 2018
Year: 2018

Leveraging grant-free radio access for enabling lowpower wide-area (LPWA) Internet of Things (IoT) connectivity has attracted lots of attention in recent years. Regarding lack of research on LPWA IoT networks, this work is devoted to reliability modeling, battery-lifetime analysis, and operation-control of such networks. We derive the interplay amongst density of the access points, communication bandwidth, volume of traffic from heterogeneous sources, and quality of service (QoS) in communications. The presented analytical framework comprises modeling of interference from heterogeneous sources with correlated deployment locations and time-frequency asynchronous radio-resource usage patterns. The derived expressions represent the operation regions and rates in which, energy and cost resources of devices and the access network, respectively, could be traded to achieve a given level of QoS in communications. For example, our expressions indicate the expected increase in QoS by increasing number of transmitted replicas, transmit power, density of the access points, and communication bandwidth. Our results further shed light on scalability of such networks and figure out the bounds up to which, scaling resources can compensate the increase in traffic volume and QoS demand. Finally, we present an energy-optimized operation control policy for IoT devices. The simulation results confirm tightness of the derived analytical expressions, and indicate usefulness of them in planning and operation control of IoT networks.
Chinese Poetry Generation with Flexible Styles
Venue: arXiv cs.CL 2018
Year: 2018

Research has shown that sequence-to-sequence neural models, particularly those with the attention mechanism, can successfully generate classical Chinese poems. However, neural models are not capable of generating poems that match specific styles, such as the impulsive style of Li Bai, a famous poet in the Tang Dynasty. This work proposes a memory-augmented neural model to enable the generation of style-specific poetry. The key idea is a memory structure that stores how poems with a desired style were generated by humans, and uses similar fragments to adjust the generation. We demonstrate that the proposed algorithm generates poems with flexible styles, including styles of a particular era and an individual poet.
Design and optimisation of an efficient HDF5 I/O kernel for massive parallel fluid flow simulations
Venue: arXiv cs.PF 2018
Year: 2018

More and more massive parallel codes running on several hundreds of thousands of cores enter the computational science and engineering domain, allowing high-fidelity computations on up to trillions of unknowns for very detailed analyses of the underlying problems. During such runs, typically gigabytes of data are being produced, hindering both efficient storage and (interactive) data exploration. Here, advanced approaches based on inherently distributed data formats such as HDF5 become necessary in order to avoid long latencies when storing the data and to support fast (random) access when retrieving the data for visual processing. Avoiding file locking and using collective buffering, write bandwidths to a single file close to the theoretical peak on a modern supercomputing cluster were achieved. The structure of the output file supports a very fast interactive visualisation and introduces additional steering functionality.
Comparison of RNN Encoder-Decoder Models for Anomaly Detection
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we compare different types of Recurrent Neural Network (RNN) Encoder-Decoders in anomaly detection viewpoint. We focused on finding the model that can learn the same data more effectively. We compared multiple models under the same conditions, such as the number of parameters, optimizer, and learning rate. However, the difference is whether to predict the future sequence or restore the current sequence. We constructed the dataset with simple vectors and used them for the experiment. Finally, we experimentally confirmed that the model performs better when the model restores the current sequence, rather than predict the future sequence.
Generating Levels That Teach Mechanics
Venue: arXiv cs.AI 2018
Year: 2018

The automatic generation of game tutorials is a challenging AI problem. While it is possible to generate annotations and instructions that explain to the player how the game is played, this paper focuses on generating a gameplay experience that introduces the player to a game mechanic. It evolves small levels for the Mario AI Framework that can only be beaten by an agent that knows how to perform specific actions in the game. It uses variations of a perfect A* agent that are limited in various ways, such as not being able to jump high or see enemies, to test how failing to do certain actions can stop the player from beating the level.
Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings
Venue: arXiv cs.CL 2018
Year: 2018

Unsupervised learning has been an attractive method for easily deriving meaningful data representations from vast amounts of unlabeled data. These representations, or embeddings, often yield superior results in many tasks, whether used directly or as features in subsequent training stages. However, the quality of the embeddings is highly dependent on the assumed knowledge in the unlabeled data and how the system extracts information without supervision. Domain portability is also very limited in unsupervised learning, often requiring re-training on other in-domain corpora to achieve robustness. In this work we present a multitask paradigm for unsupervised contextual learning of behavioral interactions which addresses unsupervised domain adaption. We introduce an online multitask objective into unsupervised learning and show that sentence embeddings generated through this process increases performance of affective tasks.
A data-supported history of bioinformatics tools
Venue: arXiv cs.DL 2018
Year: 2018

Since the advent of next-generation sequencing in the early 2000s, the volume of bioinformatics software tools and databases has exploded and continues to grow rapidly. Documenting this evolution on a global and time-dependent scale is a challenging task, limited by the scarcity of comprehensive tool repositories. We collected data from over ~23,000 references classified in the OMICtools database, spanning the last 26 years of bioinformatics to present a data-supported snapshot of bioinformatics software tool evolution and the current status, to shed light on future directions and opportunities in this field. The present review explores new aspects of computational biology, including country partnerships, trends in technologies and area of development, research and development (R&D) investments and coding languages. This is the most comprehensive systematic overview of the field to date and provides the community with insights and knowledge on the direction of the development and evolution of bioinformatics software tools, highlighting the increasing complexity of analysis.
Practical MIMO-NOMA: Low Complexity & Capacity-Approaching Solution
Venue: arXiv cs.IT 2018
Year: 2018

MIMO-NOMA combines Multiple-Input Multiple-Output (MIMO) and Non-Orthogonal Multiple Access (NOMA), which can address heterogeneous challenges, such as massive connectivity, low latency, and high reliability. In this paper, a practical coded MIMO-NOMA system with capacityapproaching performance as well as low implementation complexity is proposed. Specifically, the employed receiver consists of a multi-user Linear Minimum Mean-Square Error (LMMSE) detector and a bank of single-user message-passing decoders, which decompose the overall signal recovery into distributed low-complexity calculations. An asymptotic extrinsic information transfer analysis is proposed to estimate the performance of iterative receiver, where practical channel codes that match with the LMMSE detector in the iterative decoding perspective are constructed. As a result, the proposed coded MIMO-NOMA system achieves asymptotic performances within 0.2 dB from the theoretical capacity. Simulation results validate the reliability and robustness of the proposed system in practical settings, including various system loads, iteration numbers, code lengths, and channel conditions.
Guaranteed Error Bounds on Approximate Model Abstractions through Reachability Analysis
Venue: arXiv cs.SY 2018
Year: 2018

It is well known that exact notions of model abstraction and reduction for dynamical systems may not be robust enough in practice because they are highly sensitive to the specific choice of parameters. In this paper we consider this problem for nonlinear ordinary differential equations (ODEs) with polynomial derivatives. We introduce approximate differential equivalence as a more permissive variant of a recently developed exact counterpart, allowing ODE variables to be related even when they are governed by nearby derivatives. We develop algorithms to (i) compute the largest approximate differential equivalence; (ii) construct an approximate quotient model from the original one via an appropriate parameter perturbation; and (iii) provide a formal certificate on the quality of the approximation as an error bound, computed as an over-approximation of the reachable set of the perturbed model. Finally, we apply approximate differential equivalences to study the effect of parametric tolerances in models of symmetric electric circuits.
Deep neural network based speech separation optimizing an objective estimator of intelligibility for low latency applications
Venue: arXiv cs.SD 2018
Year: 2018

Mean square error (MSE) has been the preferred choice as loss function in the current deep neural network (DNN) based speech separation techniques. In this paper, we propose a new cost function with the aim of optimizing the extended short time objective intelligibility (ESTOI) measure. We focus on applications where low algorithmic latency ($\leq 10$ ms) is important. We use long short-term memory networks (LSTM) and evaluate our proposed approach on four sets of two-speaker mixtures from extended Danish hearing in noise (HINT) dataset. We show that the proposed loss function can offer improved or at par objective intelligibility (in terms of ESTOI) compared to an MSE optimized baseline while resulting in lower objective separation performance (in terms of the source to distortion ratio (SDR)). We then proceed to propose an approach where the network is first initialized with weights optimized for MSE criterion and then trained with the proposed ESTOI loss criterion. This approach mitigates some of the losses in objective separation performance while preserving the gains in objective intelligibility.
Active Learning for Segmentation by Optimizing Content Information for Maximal Entropy
Venue: arXiv cs.CV 2018
Year: 2018

Segmentation is essential for medical image analysis tasks such as intervention planning, therapy guidance, diagnosis, treatment decisions. Deep learning is becoming increasingly prominent for segmentation, where the lack of annotations, however, often becomes the main limitation. Due to privacy concerns and ethical considerations, most medical datasets are created, curated, and allow access only locally. Furthermore, current deep learning methods are often suboptimal in translating anatomical knowledge between different medical imaging modalities. Active learning can be used to select an informed set of image samples to request for manual annotation, in order to best utilize the limited annotation time of clinical experts for optimal outcomes, which we focus on in this work. Our contributions herein are two fold: (1) we enforce domain-representativeness of selected samples using a proposed penalization scheme to maximize information at the network abstraction layer, and (2) we propose a Borda-count based sample querying scheme for selecting samples for segmentation. Comparative experiments with baseline approaches show that the samples queried with our proposed method, where both above contributions are combined, result in significantly improved segmentation performance for this active learning task.
Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
Venue: arXiv cs.CV 2018
Year: 2018

Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost. In order to reduce this cost, several quantization schemes have gained attention recently with some focusing on weight quantization, and others focusing on quantizing activations. This paper proposes novel techniques that target weight and activation quantizations separately resulting in an overall quantized neural network (QNN). The activation quantization technique, PArameterized Clipping acTivation (PACT), uses an activation clipping parameter $α$ that is optimized during training to find the right quantization scale. The weight quantization scheme, statistics-aware weight binning (SAWB), finds the optimal scaling factor that minimizes the quantization error based on the statistical characteristics of the distribution of weights without the need for an exhaustive search. The combination of PACT and SAWB results in a 2-bit QNN that achieves state-of-the-art classification accuracy (comparable to full precision networks) across a range of popular models and datasets.
The Generalized Lasso for Sub-gaussian Measurements with Dithered Quantization
Venue: arXiv cs.IT 2018
Year: 2018

In the problem of structured signal recovery from high-dimensional linear observations, it is commonly assumed that full-precision measurements are available. Under this assumption, the recovery performance of the popular Generalized Lasso (G-Lasso) is by now well-established. In this paper, we extend these types of results to the practically relevant settings with quantized measurements. We study two extremes of the quantization schemes, namely, uniform and one-bit quantization; the former imposes no limit on the number of quantization bits, while the second only allows for one bit. In the presence of a uniform dithering signal and when measurement vectors are sub-gaussian, we show that the same algorithm (i.e., the G-Lasso) has favorable recovery guarantees for both uniform and one-bit quantization schemes. Our theoretical results, shed light on the appropriate choice of the range of values of the dithering signal and accurately capture the error dependence on the problem parameters. For example, our error analysis shows that the G-Lasso with one-bit uniformly dithered measurements leads to only a logarithmic rate loss compared to the full-precision measurements.
Video Time: Properties, Encoders and Evaluation
Venue: arXiv cs.CV 2018
Year: 2018

Time-aware encoding of frame sequences in a video is a fundamental problem in video understanding. While many attempted to model time in videos, an explicit study on quantifying video time is missing. To fill this lacuna, we aim to evaluate video time explicitly. We describe three properties of video time, namely a) temporal asymmetry, b)temporal continuity and c) temporal causality. Based on each we formulate a task able to quantify the associated property. This allows assessing the effectiveness of modern video encoders, like C3D and LSTM, in their ability to model time. Our analysis provides insights about existing encoders while also leading us to propose a new video time encoder, which is better suited for the video time recognition tasks than C3D and LSTM. We believe the proposed meta-analysis can provide a reasonable baseline to assess video time encoders on equal grounds on a set of temporal-aware tasks.
Is it worth it? Budget-related evaluation metrics for model selection
Venue: arXiv cs.CL 2018
Year: 2018

Creating a linguistic resource is often done by using a machine learning model that filters the content that goes through to a human annotator, before going into the final resource. However, budgets are often limited, and the amount of available data exceeds the amount of affordable annotation. In order to optimize the benefit from the invested human work, we argue that deciding on which model one should employ depends not only on generalized evaluation metrics such as F-score, but also on the gain metric. Because the model with the highest F-score may not necessarily have the best sequencing of predicted classes, this may lead to wasting funds on annotating false positives, yielding zero improvement of the linguistic resource. We exemplify our point with a case study, using real data from a task of building a verb-noun idiom dictionary. We show that, given the choice of three systems with varying F-scores, the system with the highest F-score does not yield the highest profits. In other words, in our case the cost-benefit trade off is more favorable for a system with a lower F-score.
Semantic Parsing: Syntactic assurance to target sentence using LSTM Encoder CFG-Decoder
Venue: arXiv cs.CL 2018
Year: 2018

Semantic parsing can be defined as the process of mapping natural language sentences into a machine interpretable, formal representation of its meaning. Semantic parsing using LSTM encoder-decoder neural networks have become promising approach. However, human automated translation of natural language does not provide grammaticality guarantees for the sentences generate such a guarantee is particularly important for practical cases where a data base query can cause critical errors if the sentence is ungrammatical. In this work, we propose an neural architecture called Encoder CFG-Decoder, whose output conforms to a given context-free grammar. Results are show for any implementation of such architecture display its correctness and providing benchmark accuracy levels better than the literature.
Representational efficiency outweighs action efficiency in human program induction
Venue: arXiv cs.AI 2018
Year: 2018

The importance of hierarchically structured representations for tractable planning has long been acknowledged. However, the questions of how people discover such abstractions and how to define a set of optimal abstractions remain open. This problem has been explored in cognitive science in the problem solving literature and in computer science in hierarchical reinforcement learning. Here, we emphasize an algorithmic perspective on learning hierarchical representations in which the objective is to efficiently encode the structure of the problem, or, equivalently, to learn an algorithm with minimal length. We introduce a novel problem-solving paradigm that links problem solving and program induction under the Markov Decision Process (MDP) framework. Using this task, we target the question of whether humans discover hierarchical solutions by maximizing efficiency in number of actions they generate or by minimizing the complexity of the resulting representation and find evidence for the primacy of representational efficiency.