OpenRaiser/Intern-Atlas huggingface dataset:
Extracting 2D weak labels from volume labels using multiple instance learning in CT hemorrhage detection
Venue: arXiv cs.CV 2019
Year: 2019

Multiple instance learning (MIL) is a supervised learning methodology that aims to allow models to learn instance class labels from bag class labels, where a bag is defined to contain multiple instances. MIL is gaining traction for learning from weak labels but has not been widely applied to 3D medical imaging. MIL is well-suited to clinical CT acquisitions since (1) the highly anisotropic voxels hinder application of traditional 3D networks and (2) patch-based networks have limited ability to learn whole volume labels. In this work, we apply MIL with a deep convolutional neural network to identify whether clinical CT head image volumes possess one or more large hemorrhages (> 20cm$^3$), resulting in a learned 2D model without the need for 2D slice annotations. Individual image volumes are considered separate bags, and the slices in each volume are instances. Such a framework sets the stage for incorporating information obtained in clinical reports to help train a 2D segmentation approach. Within this context, we evaluate the data requirements to enable generalization of MIL by varying the amount of training data. Our results show that a training size of at least 400 patient image volumes was needed to achieve accurate per-slice hemorrhage detection. Over a five-fold cross-validation, the leading model, which made use of the maximum number of training volumes, had an average true positive rate of 98.10%, an average true negative rate of 99.36%, and an average precision of 0.9698. The models have been made available along with source code to enabled continued exploration and adaption of MIL in CT neuroimaging.
Relative contributions of Shakespeare and Fletcher in Henry VIII: An Analysis Based on Most Frequent Words and Most Frequent Rhythmic Patterns
Venue: arXiv cs.CL 2019
Year: 2019

The versified play Henry VIII is nowadays widely recognized to be a collaborative work not written solely by William Shakespeare. We employ combined analysis of vocabulary and versification together with machine learning techniques to determine which authors also took part in the writing of the play and what were their relative contributions. Unlike most previous studies, we go beyond the attribution of particular scenes and use the rolling attribution approach to determine the probabilities of authorship of pieces of texts, without respecting the scene boundaries. Our results highly support the canonical division of the play between William Shakespeare and John Fletcher proposed by James Spedding, but also bring new evidence supporting the modifications proposed later by Thomas Merriam.
Enhancing Programmability, Portability, and Performance with Rich Cross-Layer Abstractions
Venue: arXiv cs.DC 2019
Year: 2019

Programmability, performance portability, and resource efficiency have emerged as critical challenges in harnessing complex and diverse architectures today to obtain high performance and energy efficiency. While there is abundant research, and thus significant improvements, at different levels of the stack that address these very challenges, in this thesis, we observe that we are fundamentally limited by the interfaces and abstractions between the application and the underlying system/hardware--specifically, the hardware-software interface. The existing narrow interfaces pose two critical challenges. First, significant effort and expertise are required to write high-performance code to harness the full potential of today's diverse and sophisticated hardware. Second, as a hardware/system designer, architecting faster and more efficient systems is challenging as the vast majority of the program's semantic content gets lost in translation with today's hardware-software interface. Moving towards the future, these challenges in programmability and efficiency will be even more intractable as we architect increasingly heterogeneous and sophisticated systems. This thesis makes the case for rich low-overhead cross-layer abstractions as a highly effective means to address the above challenges. These abstractions are designed to communicate higher-level program information from the application to the underlying system and hardware in a highly efficient manner, requiring only minor additions to the existing interfaces. In doing so, they enable a rich space of hardware-software cooperative mechanisms to optimize for performance. We propose 4 different approaches to designing richer abstractions between the application, system software, and hardware architecture in different contexts to significantly improve programmability, portability, and performance in CPUs and GPUs.
Point Movement in a DSL for Higher-Order FEM Visualization
Venue: arXiv cs.GR 2019
Year: 2019

Scientific visualization tools tend to be flexible in some ways (e.g., for exploring isovalues) while restricted in other ways, such as working only on regular grids, or only on unstructured meshes (as used in the finite element method, FEM). Our work seeks to expose the common structure of visualization methods, apart from the specifics of how the fields being visualized are formed. Recognizing that previous approaches to FEM visualization depend on efficiently updating computed positions within a mesh, we took an existing visualization domain-specific language, and added a mesh position type and associated arithmetic operators. These are orthogonal to the visualization method itself, so existing programs for visualizing regular grid data work, with minimal changes, on higher-order FEM data. We reproduce the efficiency gains of an earlier guided search method of mesh position update for computing streamlines, and we demonstrate a novel ability to uniformly sample ridge surfaces of higher-order FEM solutions defined on curved meshes.
Learning Representations in Reinforcement Learning:An Information Bottleneck Approach
Venue: arXiv cs.LG 2019
Year: 2019

The information bottleneck principle is an elegant and useful approach to representation learning. In this paper, we investigate the problem of representation learning in the context of reinforcement learning using the information bottleneck framework, aiming at improving the sample efficiency of the learning algorithms. %by accelerating the process of discarding irrelevant information when the %input states are extremely high-dimensional. We analytically derive the optimal conditional distribution of the representation, and provide a variational lower bound. Then, we maximize this lower bound with the Stein variational (SV) gradient method. We incorporate this framework in the advantageous actor critic algorithm (A2C) and the proximal policy optimization algorithm (PPO). Our experimental results show that our framework can improve the sample efficiency of vanilla A2C and PPO significantly. Finally, we study the information bottleneck (IB) perspective in deep RL with the algorithm called mutual information neural estimation(MINE) . We experimentally verify that the information extraction-compression process also exists in deep RL and our framework is capable of accelerating this process. We also analyze the relationship between MINE and our method, through this relationship, we theoretically derive an algorithm to optimize our IB framework without constructing the lower bound.
Schedule Earth Observation satellites with Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Optical Earth observation satellites acquire images worldwide , covering up to several million square kilometers every day. The complexity of scheduling acquisitions for such systems increases exponentially when considering the interoperabil-ity of several satellite constellations together with the uncertainties from weather forecasts. In order to deliver valid images to customers as fast as possible, it is crucial to acquire cloud-free images. Depending on weather forecasts, up to 50% of images acquired by operational satellites can be trashed due to excessive cloud covers, showing there is room for improvement. We propose an acquisition scheduling approach based on Deep Reinforcement Learning and experiment on a simplified environment. We find that it challenges classical methods relying on human-expert heuristic.
Building Effective Large-Scale Traffic State Prediction System: Traffic4cast Challenge Solution
Venue: arXiv cs.LG 2019
Year: 2019

How to build an effective large-scale traffic state prediction system is a challenging but highly valuable problem. This study focuses on the construction of an effective solution designed for spatio-temporal data to predict large-scale traffic state. Considering the large data size in Traffic4cast Challenge and our limited computational resources, we emphasize model design to achieve a relatively high prediction performance within acceptable running time. We adopt a structure similar to U-net and use a mask instead of spatial attention to address the data sparsity. Then, combined with the experience of time series prediction problem, we design a number of features, which are input into the model as different channels. Region cropping is used to decrease the difference between the size of the receptive field and the study area, and the models can be specially optimized for each sub-region. The fusion of interdisciplinary knowledge and experience is an emerging demand in classical traffic research. Several interdisciplinary studies we have been studying are also discussed in the Complementary Challenges. The source codes are available in https://github.com/wufanyou/traffic4cast-TLab.
RAPDARTS: Resource-Aware Progressive Differentiable Architecture Search
Venue: arXiv cs.LG 2019
Year: 2019

Early neural network architectures were designed by so-called "grad student descent". Since then, the field of Neural Architecture Search (NAS) has developed with the goal of algorithmically designing architectures tailored for a dataset of interest. Recently, gradient-based NAS approaches have been created to rapidly perform the search. Gradient-based approaches impose more structure on the search, compared to alternative NAS methods, enabling faster search phase optimization. In the real-world, neural architecture performance is measured by more than just high accuracy. There is increasing need for efficient neural architectures, where resources such as model size or latency must also be considered. Gradient-based NAS is also suitable for such multi-objective optimization. In this work we extend a popular gradient-based NAS method to support one or more resource costs. We then perform in-depth analysis on the discovery of architectures satisfying single-resource constraints for classification of CIFAR-10.
On the Relativized Alon Second Eigenvalue Conjecture III: Asymptotic Expansions for Tangle-Free Hashimoto Traces
Venue: arXiv cs.DM 2019
Year: 2019

This is the third in a series of articles devoted to showing that a typical covering map of large degree to a fixed, regular graph has its new adjacency eigenvalues within the bound conjectured by Alon for random regular graphs. In this paper we consider random graphs that are random covering graphs of large degree $n$ of a fixed base graph. We prove the existence of asympototic expansion in $1/n$ for the expected value of the number of strictly non-backtracking closed walks of length $k$ times the indicator function that the graph is free of certain {\em tangles}; moreover, we prove that the coefficients of these expansions are "nice functions" of $k$, namely approximately equal to a sum of polynomials in $k$ times exponential functions of $k$. Our results use the methods of Friedman used to resolve Alon's original conjecture, combined with the results of Article~II in this series of articles. One simplification in this article over the previous methods of Friedman is that the "regularlized traces" used in this article, which we call {\em certified traces}, are far easier to define and work with than the previously utilized {\em selective traces}.
A Discreet Wearable IoT Sensor for Continuous Transdermal Alcohol Monitoring -- Challenges and Opportunities
Venue: arXiv cs.HC 2019
Year: 2019

Non-invasive continuous alcohol monitoring has potential applications in both population research and in clinical management of acute alcohol intoxication or chronic alcoholism. Current wearable monitors based on transdermal alcohol content (TAC) sensing are relatively bulky and have limited quantification accuracy. Here we describe the development of a discreet wearable transdermal alcohol (TAC) sensor in the form of a wristband or armband. This novel sensor can detect vapor-phase alcohol in perspiration from 0.09 ppm (equivalent to 0.09 mg/dL sweat alcohol concentration at 25 °C under Henry's Law equilibrium) to over 500 ppm at one-minute time resolution. The TAC sensor is powered by a 110 mAh lithium battery that lasts for over 7 days. In addition, the sensor can function as a medical "internet-of-things" (IoT) device by connecting to an Android smartphone gateway via Bluetooth Low Energy (BLE) and upload data to a cloud informatics system. Such wearable IoT sensors may enable large-scale alcohol-related research and personalized management. We also present evidence suggesting a hypothesis that perspiration rate is the dominant factor leading to TAC measurement variabilities, which may inform more reproducible and accurate TAC sensor designs in the future.
Robots Assembling Machines: Learning from the World Robot Summit 2018 Assembly Challenge
Venue: arXiv cs.RO 2019
Year: 2019

The Industrial Assembly Challenge at the World Robot Summit was held in 2018 to showcase the state-of-the-art of autonomous manufacturing systems. The challenge included various tasks, such as bin picking, kitting, and assembly of standard industrial parts into 2D and 3D assemblies. Some of the tasks were only revealed at the competition itself, representing the challenge of "level 5" automation, i. e., programming and setting up an autonomous assembly system in less than one day. We conducted a survey among the teams that participated in the challenge and investigated aspects such as team composition, development costs, system setups as well as the teams' strategies and approaches. An analysis of the survey results reveals that the competitors have been in two camps: those constructing conventional robotic work cells with off-the-shelf tools, and teams who mostly relied on custom-made end effectors and novel software approaches in combination with collaborative robots. While both camps performed reasonably well, the winning team chose a middle ground in between, combining the efficiency of established play-back programming with the autonomy gained by CAD-based object detection and force control for assembly operations.
A Dynamic Preference Logic for reasoning about Agent Programming
Venue: arXiv cs.MA 2019
Year: 2019

In this work, we investigate the use of Dynamic Preference Logic to encode BDI mental attitudes. Further, exploring this codification and the representation of preferences over possible worlds by preferences over propositional formulas, here called priority graphs, we comment on how to interpret BDI agent programs in this logic. Also, using the connection between dynamic operations defined over preference models and their encoding as transformations on priority graphs, we show how our logic can be used not only to reason about agent programs, but as a tool to specify reasoning mechanisms to guarantee certain properties in the theory of rationality for the programming language.
Recent Advances in Algorithmic High-Dimensional Robust Statistics
Venue: arXiv cs.DS 2019
Year: 2019

Learning in the presence of outliers is a fundamental problem in statistics. Until recently, all known efficient unsupervised learning algorithms were very sensitive to outliers in high dimensions. In particular, even for the task of robust mean estimation under natural distributional assumptions, no efficient algorithm was known. Recent work in theoretical computer science gave the first efficient robust estimators for a number of fundamental statistical tasks, including mean and covariance estimation. Since then, there has been a flurry of research activity on algorithmic high-dimensional robust estimation in a range of settings. In this survey article, we introduce the core ideas and algorithmic techniques in the emerging area of algorithmic high-dimensional robust statistics with a focus on robust mean estimation. We also provide an overview of the approaches that have led to computationally efficient robust estimators for a range of broader statistical tasks and discuss new directions and opportunities for future work.
Optimal Server Selection for Straggler Mitigation
Venue: arXiv cs.NI 2019
Year: 2019

The performance of large-scale distributed compute systems is adversely impacted by stragglers when the execution time of a job is uncertain. To manage stragglers, we consider a multi-fork approach for job scheduling, where additional parallel servers are added at forking instants. In terms of the forking instants and the number of additional servers, we compute the job completion time and the cost of server utilization when the task processing times are assumed to have a shifted exponential distribution. We use this study to provide insights into the scheduling design of the forking instants and the associated number of additional servers to be started. Numerical results demonstrate orders of magnitude improvement in cost in the regime of low completion times as compared to the prior works.
Progressive Feature Polishing Network for Salient Object Detection
Venue: arXiv cs.CV 2019
Year: 2019

Feature matters for salient object detection. Existing methods mainly focus on designing a sophisticated structure to incorporate multi-level features and filter out cluttered features. We present Progressive Feature Polishing Network (PFPN), a simple yet effective framework to progressively polish the multi-level features to be more accurate and representative. By employing multiple Feature Polishing Modules (FPMs) in a recurrent manner, our approach is able to detect salient objects with fine details without any post-processing. A FPM parallelly updates the features of each level by directly incorporating all higher level context information. Moreover, it can keep the dimensions and hierarchical structures of the feature maps, which makes it flexible to be integrated with any CNN-based models. Empirical experiments show that our results are monotonically getting better with increasing number of FPMs. Without bells and whistles, PFPN outperforms the state-of-the-art methods significantly on five benchmark datasets under various evaluation metrics.
Contextual Recurrent Units for Cloze-style Reading Comprehension
Venue: arXiv cs.CL 2019
Year: 2019

Recurrent Neural Networks (RNN) are known as powerful models for handling sequential data, and especially widely utilized in various natural language processing tasks. In this paper, we propose Contextual Recurrent Units (CRU) for enhancing local contextual representations in neural networks. The proposed CRU injects convolutional neural networks (CNN) into the recurrent units to enhance the ability to model the local context and reducing word ambiguities even in bi-directional RNNs. We tested our CRU model on sentence-level and document-level modeling NLP tasks: sentiment classification and reading comprehension. Experimental results show that the proposed CRU model could give significant improvements over traditional CNN or RNN models, including bidirectional conditions, as well as various state-of-the-art systems on both tasks, showing its promising future of extensibility to other NLP tasks as well.
Efficient Animation of Sparse Voxel Octrees for Real-Time Ray Tracing
Venue: arXiv cs.GR 2019
Year: 2019

A considerable limitation of employing sparse voxels octrees (SVOs) as a model format for ray tracing has been that the octree data structure is inherently static. Due to traversal algorithms' dependence on the strict hierarchical structure of octrees, it has been challenging to achieve real-time performance of SVO model animation in ray tracing since the octree data structure would typically have to be regenerated every frame. Presented in this article is a novel method for animation of models specified on the SVO format. The method distinguishes itself by permitting model transformations such as rotation, translation, and anisotropic scaling, while preserving the hierarchical structure of SVO models so that they may be efficiently traversed. Due to its modest memory footprint and straightforward arithmetic operations, the method is well-suited for implementation in hardware. A software ray tracing implementation of animated SVO models demonstrates real-time performance on current-generation desktop GPUs, and shows that the animation method does not substantially slow down the rendering procedure compared to rendering static SVOs.
Conjugate Gradients for Kernel Machines
Venue: arXiv cs.LG 2019
Year: 2019

Regularized least-squares (kernel-ridge / Gaussian process) regression is a fundamental algorithm of statistics and machine learning. Because generic algorithms for the exact solution have cubic complexity in the number of datapoints, large datasets require to resort to approximations. In this work, the computation of the least-squares prediction is itself treated as a probabilistic inference problem. We propose a structured Gaussian regression model on the kernel function that uses projections of the kernel matrix to obtain a low-rank approximation of the kernel and the matrix. A central result is an enhanced way to use the method of conjugate gradients for the specific setting of least-squares regression as encountered in machine learning. Our method improves the approximation of the kernel ridge regressor / Gaussian process posterior mean over vanilla conjugate gradients and, allows computation of the posterior variance and the log marginal likelihood (evidence) without further overhead.
Supplementary material for Uncorrected least-squares temporal difference with lambda-return
Venue: arXiv cs.LG 2019
Year: 2019

Here, we provide a supplementary material for Takayuki Osogami, "Uncorrected least-squares temporal difference with lambda-return," which appears in {\it Proceedings of the 34th AAAI Conference on Artificial Intelligence} (AAAI-20).
Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling
Venue: arXiv cs.LG 2019
Year: 2019

Convolutional neural networks (CNNs) with dilated filters such as the Wavenet or the Temporal Convolutional Network (TCN) have shown good results in a variety of sequence modelling tasks. However, efficiently modelling long-term dependencies in these sequences is still challenging. Although the receptive field of these models grows exponentially with the number of layers, computing the convolutions over very long sequences of features in each layer is time and memory-intensive, prohibiting the use of longer receptive fields in practice. To increase efficiency, we make use of the "slow feature" hypothesis stating that many features of interest are slowly varying over time. For this, we use a U-Net architecture that computes features at multiple time-scales and adapt it to our auto-regressive scenario by making convolutions causal. We apply our model ("Seq-U-Net") to a variety of tasks including language and audio generation. In comparison to TCN and Wavenet, our network consistently saves memory and computation time, with speed-ups for training and inference of over 4x in the audio generation experiment in particular, while achieving a comparable performance in all tasks.