OpenRaiser/Intern-Atlas huggingface dataset:
BGrowth: an efficient approach for the segmentation of vertebral compression fractures in magnetic resonance imaging
Venue: arXiv cs.CV 2019
Year: 2019

Segmentation of medical images is a critical issue: several process of analysis and classification rely on this segmentation. With the growing number of people presenting back pain and problems related to it, the automatic or semi-automatic segmentation of fractured vertebral bodies became a challenging task. In general, those fractures present several regions with non-homogeneous intensities and the dark regions are quite similar to the structures nearby. Aimed at overriding this challenge, in this paper we present a semi-automatic segmentation method, called Balanced Growth (BGrowth). The experimental results on a dataset with 102 crushed and 89 normal vertebrae show that our approach significantly outperforms well-known methods from the literature. We have achieved an accuracy up to 95% while keeping acceptable processing time performance, that is equivalent to the state-of-the-artmethods. Moreover, BGrowth presents the best results even with a rough (sloppy) manual annotation (seed points).
Exploring Model-based Planning with Policy Networks
Venue: arXiv cs.LG 2019
Year: 2019

Model-based reinforcement learning (MBRL) with model-predictive control or online planning has shown great potential for locomotion control tasks in terms of both sample efficiency and asymptotic performance. Despite their initial successes, the existing planning methods search from candidate sequences randomly generated in the action space, which is inefficient in complex high-dimensional environments. In this paper, we propose a novel MBRL algorithm, model-based policy planning (POPLIN), that combines policy networks with online planning. More specifically, we formulate action planning at each time-step as an optimization problem using neural networks. We experiment with both optimization w.r.t. the action sequences initialized from the policy network, and also online optimization directly w.r.t. the parameters of the policy network. We show that POPLIN obtains state-of-the-art performance in the MuJoCo benchmarking environments, being about 3x more sample efficient than the state-of-the-art algorithms, such as PETS, TD3 and SAC. To explain the effectiveness of our algorithm, we show that the optimization surface in parameter space is smoother than in action space. Further more, we found the distilled policy network can be effectively applied without the expansive model predictive control during test time for some environments such as Cheetah. Code is released in https://github.com/WilsonWangTHU/POPLIN.
Gray-box Monitoring of Hyperproperties (Extended Version)
Venue: arXiv cs.LO 2019
Year: 2019

Many important system properties, particularly in security and privacy, cannot be verified statically. Therefore, runtime verification is an appealing alternative. Logics for hyperproperties, such as HyperLTL, support a rich set of such properties. We first show that black-box monitoring of HyperLTL is in general unfeasible, and suggest a gray-box approach. Gray-box monitoring implies performing analysis of the system at run-time, which brings new limitations to monitorabiliy (the feasibility of solving the monitoring problem). Thus, as another contribution of this paper we refine the classic notions of monitorability, both for trace properties and hyperproperties, taking into account the computability of the monitor. We then apply our approach to monitor a privacy hyperproperty called distributed data minimality, expressed as a HyperLTL property, by using an SMT-based static verifier at runtime.
Latent Distribution Assumption for Unbiased and Consistent Consensus Modelling
Venue: arXiv cs.HC 2019
Year: 2019

We study the problem of aggregation noisy labels. Usually, it is solved by proposing a stochastic model for the process of generating noisy labels and then estimating the model parameters using the observed noisy labels. A traditional assumption underlying previously introduced generative models is that each object has one latent true label. In contrast, we introduce a novel latent distribution assumption, implying that a unique true label for an object might not exist, but rather each object might have a specific distribution generating a latent subjective label each time the object is observed. Our experiments showed that the novel assumption is more suitable for difficult tasks, when there is an ambiguity in choosing a "true" label for certain objects.
Evolving Self-taught Neural Networks: The Baldwin Effect and the Emergence of Intelligence
Venue: arXiv cs.NE 2019
Year: 2019

The so-called Baldwin Effect generally says how learning, as a form of ontogenetic adaptation, can influence the process of phylogenetic adaptation, or evolution. This idea has also been taken into computation in which evolution and learning are used as computational metaphors, including evolving neural networks. This paper presents a technique called evolving self-taught neural networks - neural networks that can teach themselves without external supervision or reward. The self-taught neural network is intrinsically motivated. Moreover, the self-taught neural network is the product of the interplay between evolution and learning. We simulate a multi-agent system in which neural networks are used to control autonomous agents. These agents have to forage for resources and compete for their own survival. Experimental results show that the interaction between evolution and the ability to teach oneself in self-taught neural networks outperform evolution and self-teaching alone. More specifically, the emergence of an intelligent foraging strategy is also demonstrated through that interaction. Indications for future work on evolving neural networks are also presented.
Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches
Venue: arXiv cs.NE 2019
Year: 2019

Machine learning algorithms have made significant advances in many applications. However, their hardware implementation on the state-of-the-art platforms still faces several challenges and are limited by various factors, such as memory volume, memory bandwidth and interconnection overhead. The adoption of the crossbar architecture with emerging memory technology partially solves the problem but induces process variation and other concerns. In this paper, we will present novel solutions to two fundamental issues in crossbar implementation of Artificial Intelligence (AI) algorithms: device variation and insufficient interconnections. These solutions are inspired by the statistical properties of algorithms themselves, especially the redundancy in neural network nodes and connections. By Random Sparse Adaptation and pruning the connections following the Small-World model, we demonstrate robust and efficient performance on representative datasets such as MNIST and CIFAR-10. Moreover, we present Continuous Growth and Pruning algorithm for future learning and adaptation on hardware.
Learning Discriminative features using Center Loss and Reconstruction as Regularizer for Speech Emotion Recognition
Venue: arXiv cs.SD 2019
Year: 2019

This paper proposes a Convolutional Neural Network (CNN) inspired by Multitask Learning (MTL) and based on speech features trained under the joint supervision of softmax loss and center loss, a powerful metric learning strategy, for the recognition of emotion in speech. Speech features such as Spectrograms and Mel-frequency Cepstral Coefficient s (MFCCs) help retain emotion-related low-level characteristics in speech. We experimented with several Deep Neural Network (DNN) architectures that take in speech features as input and trained them under both softmax and center loss, which resulted in highly discriminative features ideal for Speech Emotion Recognition (SER). Our networks also employ a regularizing effect by simultaneously performing the auxiliary task of reconstructing the input speech features. This sharing of representations among related tasks enables our network to better generalize the original task of SER. Some of our proposed networks contain far fewer parameters when compared to state-of-the-art architectures.
SGANVO: Unsupervised Deep Visual Odometry and Depth Estimation with Stacked Generative Adversarial Networks
Venue: arXiv cs.RO 2019
Year: 2019

Recently end-to-end unsupervised deep learning methods have achieved an effect beyond geometric methods for visual depth and ego-motion estimation tasks. These data-based learning methods perform more robustly and accurately in some of the challenging scenes. The encoder-decoder network has been widely used in the depth estimation and the RCNN has brought significant improvements in the ego-motion estimation. Furthermore, the latest use of Generative Adversarial Nets(GANs) in depth and ego-motion estimation has demonstrated that the estimation could be further improved by generating pictures in the game learning process. This paper proposes a novel unsupervised network system for visual depth and ego-motion estimation: Stacked Generative Adversarial Network(SGANVO). It consists of a stack of GAN layers, of which the lowest layer estimates the depth and ego-motion while the higher layers estimate the spatial features. It can also capture the temporal dynamic due to the use of a recurrent representation across the layers. See Fig.1 for details. We select the most commonly used KITTI [1] data set for evaluation. The evaluation results show that our proposed method can produce better or comparable results in depth and ego-motion estimation.
Intrinsic Weight Learning Approach for Multi-view Clustering
Venue: arXiv cs.LG 2019
Year: 2019

Exploiting different representations, or views, of the same object for better clustering has become very popular these days, which is conventionally called multi-view clustering. Generally, it is essential to measure the importance of each individual view, due to some noises, or inherent capacities in description. Many previous works model the view importance as weight, which is simple but effective empirically. In this paper, instead of following the traditional thoughts, we propose a new weight learning paradigm in context of multi-view clustering in virtue of the idea of re-weighted approach, and we theoretically analyze its working mechanism. Meanwhile, as a carefully achieved example, all of the views are connected by exploring a unified Laplacian rank constrained graph, which will be a representative method to compare with other weight learning approaches in experiments. Furthermore, the proposed weight learning strategy is much suitable for multi-view data, and it can be naturally integrated with many existing clustering learners. According to the numerical experiments, the proposed intrinsic weight learning approach is proved effective and practical to use in multi-view clustering.
Zero Latency for Emergencies: A Machine Learning based Approach to Quantify Impact of Construction Projects on Emergency Response in Urban Settings
Venue: arXiv cs.CY 2019
Year: 2019

Continuous construction and rehabilitation in urban settings have unavoidable impacts on arrival times of first responders to emergency locations. Current research efforts on emergency response assessments focus on case studies, where specific periods (e.g., super storm Sandy) of emergency response times are analyzed. Simulation based studies that aim to evaluate response times in relation to various constraints/fleet sizes also exist. However, they do not analyze how specific changes (e.g., new and ongoing construction projects) in urban settings impact emergency response times of first responders. This paper aims to fill the gap and proposes a novel approach to predict the expected emergency response time for a given location using the fabric of zones regarding construction activities. This approach relies on historical records of emergency response and construction permits issued by city agencies. The approach first defines the signature of a zone (by zip codes) for construction activities based on the distribution of historical construction work types permitted in that zone over time. Then, zones that share similar signatures are clustered to find if there exists a relationship between construction signatures and emergency response times. Next, supervised learning algorithms are deployed to predict the average emergency response times for each cluster. The approach was tested using New York City's construction permit and emergency response records, and can be easily replicated for other cities with similar public datasets. This study serves as the first step towards quantitatively understanding construction projects' impact on a quality of life (QoL) indicator (specifically emergency response times) in urban settings.
Toward a Standard Interface for User-Defined Scheduling in OpenMP
Venue: arXiv cs.DC 2019
Year: 2019

Parallel loops are an important part of OpenMP programs. Efficient scheduling of parallel loops can improve performance of the programs. The current OpenMP specification only offers three options for loop scheduling, which are insufficient in certain instances. Given the large number of other possible scheduling strategies, it is infeasible to standardize each one. A more viable approach is to extend the OpenMP standard to allow for users to define loop scheduling strategies. The approach will enable standard-compliant application-specific scheduling. This work analyzes the principal components required by user-defined scheduling and proposes two competing interfaces as candidates for the OpenMP standard. We conceptually compare the two proposed interfaces with respect to the three host languages of OpenMP, i.e., C, C++, and Fortran. These interfaces serve the OpenMP community as a basis for discussion and prototype implementation for user-defined scheduling.
Understanding and Classifying Cultural Music Using Melodic Features Case Of Hindustani, Carnatic And Turkish Music
Venue: arXiv cs.SD 2019
Year: 2019

We present a melody based classification of musical styles by exploiting the pitch and energy based characteristics derived from the audio signal. Three prominent musical styles were chosen which have improvisation as integral part with similar melodic principles, theme, and structure of concerts namely, Hindustani, Carnatic and Turkish music. Listeners of one or more of these genres can discriminate between these based on the melodic contour alone. Listening tests were carried out using melodic attributes alone, on similar melodic pieces with respect to raga/makam, and removing any instrumentation cue to validate our hypothesis that style distinction is evident in the melody. Our method is based on finding a set of highly discriminatory features, derived from musicology, to capture distinct characteristics of the melodic contour. Behavior in terms of transitions of the pitch contour, the presence of micro-tonal notes and the nature of variations in the vocal energy are exploited. The automatically classified style labels are found to correlate well with subjective listening judgments. This was verified by using statistical tests to compare the labels from subjective and objective judgments. The melody based features, when combined with timbre based features, were seen to improve the classification performance.
A bibliometric analysis of Bitcoin scientific production
Venue: arXiv cs.DL 2019
Year: 2019

Blockchain technology, and more specifically Bitcoin (one of its foremost applications), have been receiving increasing attention in the scientific community. The first publications with Bitcoin as a topic, can be traced back to 2012. In spite of this short time span, the production magnitude (1162 papers) makes it necessary to make a bibliometric study in order to observe research clusters, emerging topics, and leading scholars. Our paper is aimed at studying the scientific production only around bitcoin, excluding other blockchain applications. Thus, we restricted our search to papers indexed in the Web of Science Core Collection, whose topic is "bitcoin". This database is suitable for such diverse disciplines such as economics, engineering, mathematics, and computer science. This bibliometric study draws the landscape of the current state and trends of Bitcoin-related research in different scientific disciplines.
Pixel-Accurate Depth Evaluation in Realistic Driving Scenarios
Venue: arXiv cs.CV 2019
Year: 2019

This work introduces an evaluation benchmark for depth estimation and completion using high-resolution depth measurements with angular resolution of up to 25" (arcsecond), akin to a 50 megapixel camera with per-pixel depth available. Existing datasets, such as the KITTI benchmark, provide only sparse reference measurements with an order of magnitude lower angular resolution - these sparse measurements are treated as ground truth by existing depth estimation methods. We propose an evaluation methodology in four characteristic automotive scenarios recorded in varying weather conditions (day, night, fog, rain). As a result, our benchmark allows us to evaluate the robustness of depth sensing methods in adverse weather and different driving conditions. Using the proposed evaluation data, we demonstrate that current stereo approaches provide significantly more stable depth estimates than monocular methods and lidar completion in adverse weather. Data and code are available at https://github.com/gruberto/PixelAccurateDepthBenchmark.git.
Backpropagation-Friendly Eigendecomposition
Venue: arXiv cs.LG 2019
Year: 2019

Eigendecomposition (ED) is widely used in deep networks. However, the backpropagation of its results tends to be numerically unstable, whether using ED directly or approximating it with the Power Iteration method, particularly when dealing with large matrices. While this can be mitigated by partitioning the data in small and arbitrary groups, doing so has no theoretical basis and makes its impossible to exploit the power of ED to the full. In this paper, we introduce a numerically stable and differentiable approach to leveraging eigenvectors in deep networks. It can handle large matrices without requiring to split them. We demonstrate the better robustness of our approach over standard ED and PI for ZCA whitening, an alternative to batch normalization, and for PCA denoising, which we introduce as a new normalization strategy for deep networks, aiming to further denoise the network's features.
Optimal Message Bundling with Delay and Synchronization Constraints in Wireless Sensor Networks
Venue: arXiv cs.NI 2019
Year: 2019

Message bundling is an effective way to reduce the energy consumption for message transmissions in wireless sensor networks. However, bundling more messages could increase both end-to-end delay and message transmission interval; the former needs to be maintained within a certain value for time-sensitive applications like environmental monitoring, while the latter affects time synchronization accuracy when the bundling includes synchronization messages as well. Taking as an example a novel time synchronization scheme recently proposed for energy efficiency, we propose an optimal message bundling approach to reduce the message transmissions while maintaining the user-defined requirements on end-to-end delay and time synchronization accuracy. Through translating the objective of joint maintenance to an integer linear programming problem, we compute a set of optimal bundling numbers for the sensor nodes to constrain their link-level delays, thereby achieve and maintain the required end-to-end delay and synchronization accuracy while the message transmission is minimized.
An alternative approach to the calculation of fundamental groups based on labeled natural deduction
Venue: arXiv cs.LO 2019
Year: 2019

In this work, we use a labelled deduction system based on the concept of computational paths (sequence of rewrites) as equalities between two terms of the same type. We also define a term rewriting system that is used to make computations between these computational paths, establishing equalities between equalities. We use a labelled deduction system based on the concept of computational paths (sequence of rewrites) to obtain some results of algebraic topology and with support of the Seifet-Van Kampen Theorem we will calculate, in a way less complex than the one made in mathematics \cite{Munkres} and the technique of homotopy type theory \cite{hott}, the fundamental group of Klein Blottle $\mathbb{K}^2$, of the Torus $\mathbb{T}^2$ and Two holed Torus $\mathbb{M}_2=\mathbb{T}^2\# \mathbb{T}^2$ (the connected sum two torus).
Fully Decoupled Neural Network Learning Using Delayed Gradients
Venue: arXiv cs.CV 2019
Year: 2019

Training neural networks with back-propagation (BP) requires a sequential passing of activations and gradients, which forces the network modules to work in a synchronous fashion. This has been recognized as the lockings (i.e., the forward, backward and update lockings) inherited from the BP. In this paper, we propose a fully decoupled training scheme using delayed gradients (FDG) to break all these lockings. The FDG splits a neural network into multiple modules and trains them independently and asynchronously using different workers (e.g., GPUs). We also introduce a gradient shrinking process to reduce the stale gradient effect caused by the delayed gradients. In addition, we prove that the proposed FDG algorithm guarantees a statistical convergence during training. Experiments are conducted by training deep convolutional neural networks to perform classification tasks on benchmark datasets, showing comparable or better results against the state-of-the-art methods as well as the BP in terms of both generalization and acceleration abilities. In particular, we show that the FDG is also able to train very wide networks (e.g., WRN-28-10) and extremely deep networks (e.g., ResNet-1202). Code is available at https://github.com/ZHUANGHP/FDG.
Near-optimal Bayesian Solution For Unknown Discrete Markov Decision Process
Venue: arXiv cs.LG 2019
Year: 2019

We tackle the problem of acting in an unknown finite and discrete Markov Decision Process (MDP) for which the expected shortest path from any state to any other state is bounded by a finite number $D$. An MDP consists of $S$ states and $A$ possible actions per state. Upon choosing an action $a_t$ at state $s_t$, one receives a real value reward $r_t$, then one transits to a next state $s_{t+1}$. The reward $r_t$ is generated from a fixed reward distribution depending only on $(s_t, a_t)$ and similarly, the next state $s_{t+1}$ is generated from a fixed transition distribution depending only on $(s_t, a_t)$. The objective is to maximize the accumulated rewards after $T$ interactions. In this paper, we consider the case where the reward distributions, the transitions, $T$ and $D$ are all unknown. We derive the first polynomial time Bayesian algorithm, BUCRL{} that achieves up to logarithm factors, a regret (i.e the difference between the accumulated rewards of the optimal policy and our algorithm) of the optimal order $\tilde{\mathcal{O}}(\sqrt{DSAT})$. Importantly, our result holds with high probability for the worst-case (frequentist) regret and not the weaker notion of Bayesian regret. We perform experiments in a variety of environments that demonstrate the superiority of our algorithm over previous techniques. Our work also illustrates several results that will be of independent interest. In particular, we derive a sharper upper bound for the KL-divergence of Bernoulli random variables. We also derive sharper upper and lower bounds for Beta and Binomial quantiles. All the bound are very simple and only use elementary functions.