OpenRaiser/Intern-Atlas huggingface dataset:
The effects of negative adaptation in Model-Agnostic Meta-Learning
Venue: arXiv cs.LG 2018
Year: 2018

The capacity of meta-learning algorithms to quickly adapt to a variety of tasks, including ones they did not experience during meta-training, has been a key factor in the recent success of these methods on few-shot learning problems. This particular advantage of using meta-learning over standard supervised or reinforcement learning is only well founded under the assumption that the adaptation phase does improve the performance of our model on the task of interest. However, in the classical framework of meta-learning, this constraint is only mildly enforced, if not at all, and we only see an improvement on average over a distribution of tasks. In this paper, we show that the adaptation in an algorithm like MAML can significantly decrease the performance of an agent in a meta-reinforcement learning setting, even on a range of meta-training tasks.
GADGET SVM: A Gossip-bAseD sub-GradiEnT Solver for Linear SVMs
Venue: arXiv cs.LG 2018
Year: 2018

In the era of big data, an important weapon in a machine learning researcher's arsenal is a scalable Support Vector Machine (SVM) algorithm. SVMs are extensively used for solving classification problems. Traditional algorithms for learning SVMs often scale super linearly with training set size which becomes infeasible very quickly for large data sets. In recent years, scalable algorithms have been designed which study the primal or dual formulations of the problem. This often suggests a way to decompose the problem and facilitate development of distributed algorithms. In this paper, we present a distributed algorithm for learning linear Support Vector Machines in the primal form for binary classification called Gossip-bAseD sub-GradiEnT (GADGET) SVM. The algorithm is designed such that it can be executed locally on nodes of a distributed system. Each node processes its local homogeneously partitioned data and learns a primal SVM model. It then gossips with random neighbors about the classifier learnt and uses this information to update the model. Extensive theoretical and empirical results suggest that this anytime algorithm has performance comparable to its centralized and online counterparts.
Generalizability of predictive models for intensive care unit patients
Venue: arXiv cs.LG 2018
Year: 2018

A large volume of research has considered the creation of predictive models for clinical data; however, much existing literature reports results using only a single source of data. In this work, we evaluate the performance of models trained on the publicly-available eICU Collaborative Research Database. We show that cross-validation using many distinct centers provides a reasonable estimate of model performance in new centers. We further show that a single model trained across centers transfers well to distinct hospitals, even compared to a model retrained using hospital-specific data. Our results motivate the use of multi-center datasets for model development and highlight the need for data sharing among hospitals to maximize model performance.
RDEC: Integrating Regularization into Deep Embedded Clustering for Imbalanced Datasets
Venue: arXiv cs.LG 2018
Year: 2018

Clustering is a fundamental machine learning task and can be used in many applications. With the development of deep neural networks (DNNs), combining techniques from DNNs with clustering has become a new research direction and achieved some success. However, few studies have focused on the imbalanced-data problem which commonly occurs in real-world applications. In this paper, we propose a clustering method, regularized deep embedding clustering (RDEC), that integrates virtual adversarial training (VAT), a network regularization technique, with a clustering method called deep embedding clustering (DEC). DEC optimizes cluster assignments by pushing data more densely around centroids in latent space, but it is sometimes sensitive to the initial location of centroids, especially in the case of imbalanced data, where the minor class has less chance to be assigned a good centroid. RDEC introduces regularization using VAT to ensure the model's robustness to local perturbations of data. VAT pushes data that are similar in the original space closer together in the latent space, bunching together data from minor classes and thereby facilitating cluster identification by RDEC. Combining the advantages of DEC and VAT, RDEC attains state-of-the-art performance on both balanced and imbalanced benchmark/real-world datasets. For example, accuracies are as high as 98.41% on MNIST dataset and 85.45% on a highly imbalanced dataset derived from the MNIST, which is nearly 8% higher than the current best result.
On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition
Venue: arXiv cs.CL 2018
Year: 2018

End-to-end automatic speech recognition (ASR) commonly transcribes audio signals into sequences of characters while its performance is evaluated by measuring the word-error rate (WER). This suggests that predicting sequences of words directly may be helpful instead. However, training with word-level supervision can be more difficult due to the sparsity of examples per label class. In this paper we analyze an end-to-end ASR model that combines a word-and-character representation in a multi-task learning (MTL) framework. We show that it improves on the WER and study how the word-level model can benefit from character-level supervision by analyzing the learned inductive preference bias of each model component empirically. We find that by adding character-level supervision, the MTL model interpolates between recognizing more frequent words (preferred by the word-level model) and shorter words (preferred by the character-level model).
The MeSH-gram Neural Network Model: Extending Word Embedding Vectors with MeSH Concepts for UMLS Semantic Similarity and Relatedness in the Biomedical Domain
Venue: arXiv cs.CL 2018
Year: 2018

Eliciting semantic similarity between concepts in the biomedical domain remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they risen to efficiently capture semantic relationships The underlying idea is that two words that have close meaning gather similar contexts. In this study, we propose a new neural network model named MeSH-gram which relies on a straighforward approach that extends the skip-gram neural network model by considering MeSH (Medical Subject Headings) descriptors instead words. Trained on publicly available corpus PubMed MEDLINE, MeSH-gram is evaluated on reference standards manually annotated for semantic similarity. MeSH-gram is first compared to skip-gram with vectors of size 300 and at several windows contexts. A deeper comparison is performed with tewenty existing models. All the obtained results of Spearman's rank correlations between human scores and computed similarities show that MeSH-gram outperforms the skip-gram model, and is comparable to the best methods but that need more computation and external resources.
Multi-Task Learning with Multi-View Attention for Answer Selection and Knowledge Base Question Answering
Venue: arXiv cs.CL 2018
Year: 2018

Answer selection and knowledge base question answering (KBQA) are two important tasks of question answering (QA) systems. Existing methods solve these two tasks separately, which requires large number of repetitive work and neglects the rich correlation information between tasks. In this paper, we tackle answer selection and KBQA tasks simultaneously via multi-task learning (MTL), motivated by the following motivations. First, both answer selection and KBQA can be regarded as a ranking problem, with one at text-level while the other at knowledge-level. Second, these two tasks can benefit each other: answer selection can incorporate the external knowledge from knowledge base (KB), while KBQA can be improved by learning contextual information from answer selection. To fulfill the goal of jointly learning these two tasks, we propose a novel multi-task learning scheme that utilizes multi-view attention learned from various perspectives to enable these tasks to interact with each other as well as learn more comprehensive sentence representations. The experiments conducted on several real-world datasets demonstrate the effectiveness of the proposed method, and the performance of answer selection and KBQA is improved. Also, the multi-view attention scheme is proved to be effective in assembling attentive information from different representational perspectives.
Reduced-Dimension Design of MIMO Over-the-Air Computing for Data Aggregation in Clustered IoT Networks
Venue: arXiv cs.IT 2018
Year: 2018

One basic operation of Internet-of-Things (IoT) networks is aggregating distributed sensing-data over wireless-channels for function-computation, called wireless-data-aggregation (WDA). Targeting dense sensors, a recently developed technology called over-the-air computing (AirComp) can dramatically reduce the WDA latency by aggregating distributed data "over-the-air" using the waveform-superposition property of a multi-access channel. In this work, we design multiple-input-multiple-output (MIMO) AirComp for computing a vector-valued function in a clustered IoT network with multi-antenna sensors forming clusters and a multi-antenna access-point (AP) performing WDA. The resultant high-dimensional but low-rank MIMO channels makes it important to reduce channel or signal dimensionality in AirComp to avoid exposure to noise from channel null-spaces. Motivated by this, we develop a framework of reduced-dimension MIMO AirComp, featuring decomposed-aggregation-beamforming (DAB). Consider the case of separable channel-clusters with non-overlapping angle-of-arrival ranges. The optimal DAB has the structure where inner-components extract the dominant eigen-spaces of corresponding channel-clusters and outer-components jointly equalize the resultant low-dimensional channels. Consider the more complex case of inseparable clusters. We propose a suboptimal DAB design where the inner-component performs both dimension-reduction and joint-equalization over clustered-channel covariance matrices and the outer-component jointly equalizes the small-scale fading-channels. Furthermore, efficient algorithms for rank-optimization of individual DAB components and channel-feedback leveraging the AirComp principle are developed.
Segmentation of Head and Neck Organs at Risk Using CNN with Batch Dice Loss
Venue: arXiv cs.CV 2018
Year: 2018

This paper deals with segmentation of organs at risk (OAR) in head and neck area in CT images which is a crucial step for reliable intensity modulated radiotherapy treatment. We introduce a convolution neural network with encoder-decoder architecture and a new loss function, the batch soft Dice loss function, used to train the network. The resulting model produces segmentations of every OAR in the public MICCAI 2015 Head And Neck Auto-Segmentation Challenge dataset. Despite the heavy class imbalance in the data, we improve accuracy of current state-of-the-art methods by 0.33 mm in terms of average surface distance and by 0.11 in terms of Dice overlap coefficient on average.
Generic machine identification and maximum efficiency operation of induction machines
Venue: arXiv cs.SY 2018
Year: 2018

This paper proposes an advanced machine identification method for inverter fed squirrel-cage induction machines, based on steady-state measurements in the rotor flux oriented dq-reference frame. The measured machine maps are used to extract maximum efficiency per torque (MEPT) look-up tables, which guarantee the maximum achievable efficiency in every operating point. Furthermore, it is shown, that comparable results can be achieved, even without a torque sensor. The main advantage of the described method is its generality, which implicitly covers magnetic saturation, iron losses and other nonlinear effects that are typically neglected or approximated by complex models. Finally, the efficiencies of V/Hz and field-oriented control (FOC) are calculated for different speeds and load torques, allowing for quantitative assessment and comparison of both methods.
Automatically Segmenting the Left Atrium from Cardiac Images Using Successive 3D U-Nets and a Contour Loss
Venue: arXiv cs.CV 2018
Year: 2018

Radiological imaging offers effective measurement of anatomy, which is useful in disease diagnosis and assessment. Previous study has shown that the left atrial wall remodeling can provide information to predict treatment outcome in atrial fibrillation. Nevertheless, the segmentation of the left atrial structures from medical images is still very time-consuming. Current advances in neural network may help creating automatic segmentation models that reduce the workload for clinicians. In this preliminary study, we propose automated, two-stage, three-dimensional U-Nets with convolutional neural network, for the challenging task of left atrial segmentation. Unlike previous two-dimensional image segmentation methods, we use 3D U-Nets to obtain the heart cavity directly in 3D. The dual 3D U-Net structure consists of, a first U-Net to coarsely segment and locate the left atrium, and a second U-Net to accurately segment the left atrium under higher resolution. In addition, we introduce a Contour loss based on additional distance information to adjust the final segmentation. We randomly split the data into training datasets (80 subjects) and validation datasets (20 subjects) to train multiple models, with different augmentation setting. Experiments show that the average Dice coefficients for validation datasets are around 0.91 - 0.92, the sensitivity around 0.90-0.94 and the specificity 0.99. Compared with traditional Dice loss, models trained with Contour loss in general offer smaller Hausdorff distance with similar Dice coefficient, and have less connected components in predictions. Finally, we integrate several trained models in an ensemble prediction to segment testing datasets.
Joint Target Detection, Tracking and Classification with Forward-Backward PHD Smoothing
Venue: arXiv cs.SY 2018
Year: 2018

Forward-backward Probability Hypothesis Density (PHD) smoothing is an efficient way for target tracking in dense clutter environment. Although the target class has been widely viewed as useful information to enhance the target tracking, there is no existing work in literature which incorporates the feature information into PHD smoothing. In this paper, we generalized the PHD smoothing by extending the general mode, which includes kinematic mode, class mode or their combinations etc., to forward-backward PHD filter. Through a top-down method, the general mode augmented forward-backward PHD smoothing is derived. The evaluation results show that our approach out-performs the state-of-art joint detection, tracking and classification algorithm in target state estimation, number estimation and classification. The reduction of OSPA distance is up to 40%.
Tube-CNN: Modeling temporal evolution of appearance for object detection in video
Venue: arXiv cs.CV 2018
Year: 2018

Object detection in video is crucial for many applications. Compared to images, video provides additional cues which can help to disambiguate the detection problem. Our goal in this paper is to learn discriminative models for the temporal evolution of object appearance and to use such models for object detection. To model temporal evolution, we introduce space-time tubes corresponding to temporal sequences of bounding boxes. We propose two CNN architectures for generating and classifying tubes, respectively. Our tube proposal network (TPN) first generates a large number of spatio-temporal tube proposals maximizing object recall. The Tube-CNN then implements a tube-level object detector in the video. Our method improves state of the art on two large-scale datasets for object detection in video: HollywoodHeads and ImageNet VID. Tube models show particular advantages in difficult dynamic scenes.
Formal Synthesis of Analytic Controllers for Sampled-Data Systems via Genetic Programming
Venue: arXiv cs.SY 2018
Year: 2018

This paper presents an automatic formal controller synthesis method for nonlinear sampled-data systems with safety and reachability specifications. Fundamentally, the presented method is not restricted to polynomial systems and controllers. We consider periodically switched controllers based on a Control Lyapunov Barrier-like functions. The proposed method utilizes genetic programming to synthesize these functions as well as the controller modes. Correctness of the controller are subsequently verified by means of a Satisfiability Modulo Theories solver. Effectiveness of the proposed methodology is demonstrated on multiple systems.
An Improved Cost Function for Hierarchical Cluster Trees
Venue: arXiv cs.DS 2018
Year: 2018

Hierarchical clustering has been a popular method in various data analysis applications. It partitions a data set into a hierarchical collection of clusters, and can provide a global view of (cluster) structure behind data across different granularity levels. A hierarchical clustering (HC) of a data set can be naturally represented by a tree, called a HC-tree, where leaves correspond to input data and subtrees rooted at internal nodes correspond to clusters. Many hierarchical clustering algorithms used in practice are developed in a procedure manner. Dasgupta proposed to study the hierarchical clustering problem from an optimization point of view, and introduced an intuitive cost function for similarity-based hierarchical clustering with nice properties as well as natural approximation algorithms. We observe that while Dasgupta's cost function is effective at differentiating a good HC-tree from a bad one for a fixed graph, the value of this cost function does not reflect how well an input similarity graph is consistent to a hierarchical structure. In this paper, we present a new cost function, which is developed based on Dasgupta's cost function, to address this issue. The optimal tree under the new cost function remains the same as the one under Dasgupta's cost function. However, the value of our cost function is more meaningful. The new way of formulating the cost function also leads to a polynomial time algorithm to compute the optimal cluster tree when the input graph has a perfect HC-structure, or an approximation algorithm when the input graph 'almost' has a perfect HC-structure. Finally, we provide further understanding of the new cost function by studying its behavior for random graphs sampled from an edge probability matrix.
Visual Object Networks: Image Generation with Disentangled 3D Representation
Venue: arXiv cs.CV 2018
Year: 2018

Recent progress in deep generative models has led to tremendous breakthroughs in image generation. However, while existing models can synthesize photorealistic images, they lack an understanding of our underlying 3D world. We present a new generative model, Visual Object Networks (VON), synthesizing natural images of objects with a disentangled 3D representation. Inspired by classic graphics rendering pipelines, we unravel our image formation process into three conditionally independent factors---shape, viewpoint, and texture---and present an end-to-end adversarial learning framework that jointly models 3D shapes and 2D images. Our model first learns to synthesize 3D shapes that are indistinguishable from real shapes. It then renders the object's 2.5D sketches (i.e., silhouette and depth map) from its shape under a sampled viewpoint. Finally, it learns to add realistic texture to these 2.5D sketches to generate natural images. The VON not only generates images that are more realistic than state-of-the-art 2D image synthesis methods, but also enables many 3D operations such as changing the viewpoint of a generated image, editing of shape and texture, linear interpolation in texture and shape space, and transferring appearance across different objects and viewpoints.
Embedding-reparameterization procedure for manifold-valued latent variables in generative models
Venue: arXiv cs.LG 2018
Year: 2018

Conventional prior for Variational Auto-Encoder (VAE) is a Gaussian distribution. Recent works demonstrated that choice of prior distribution affects learning capacity of VAE models. We propose a general technique (embedding-reparameterization procedure, or ER) for introducing arbitrary manifold-valued variables in VAE model. We compare our technique with a conventional VAE on a toy benchmark problem. This is work in progress.
Applications of Blockchain in Healthcare: Current Landscape & Challenges
Venue: arXiv cs.CY 2018
Year: 2018

Several problems in healthcare stem from the complex network of intermediaries and the lack of traceability of transactions. To mention a few: healthcare data is fragmented across several silos negatively affecting research and services, about half of the clinical trials are never reported, the cost of drug discovery is ever increasing, and substandard and fake medicines are still a huge problem. Blockchain has the potential to solve these problems as it provides trust without any intermediaries, has traceability as a default feature, and promises new business models by enabling novel incentive structures. Due to its potential, blockchain has gathered significant interest in the healthcare industry. In this paper, we review major use cases of blockchain in healthcare: patient data management, pharmaceutical research, supply chain management of medical goods, prescription management, billing claims management, analytics, and telemedicine alongside the related projects. We found that most of the blockchain projects are limited as white-papers, proof of concepts, and products with a limited user base. However, we observed that the quantity, quality, and maturity of the projects are increasing. We also discuss technical, regulatory, and business challenges to the adoption of blockchain in the healthcare industry
Tri-axial Self-Attention for Concurrent Activity Recognition
Venue: arXiv cs.CV 2018
Year: 2018

We present a system for concurrent activity recognition. To extract features associated with different activities, we propose a feature-to-activity attention that maps the extracted global features to sub-features associated with individual activities. To model the temporal associations of individual activities, we propose a transformer-network encoder that models independent temporal associations for each activity. To make the concurrent activity prediction aware of the potential associations between activities, we propose self-attention with an association mask. Our system achieved state-of-the-art or comparable performance on three commonly used concurrent activity detection datasets. Our visualizations demonstrate that our system is able to locate the important spatial-temporal features for final decision making. We also showed that our system can be applied to general multilabel classification problems.
Adversarial Defense of Image Classification Using a Variational Auto-Encoder
Venue: arXiv cs.CV 2018
Year: 2018

Deep neural networks are known to be vulnerable to adversarial attacks. This exposes them to potential exploits in security-sensitive applications and highlights their lack of robustness. This paper uses a variational auto-encoder (VAE) to defend against adversarial attacks for image classification tasks. This VAE defense has a few nice properties: (1) it is quite flexible and its use of randomness makes it harder to attack; (2) it can learn disentangled representations that prevent blurry reconstruction; and (3) a patch-wise VAE defense strategy is used that does not require retraining for different size images. For moderate to severe attacks, this system outperforms or closely matches the performance of JPEG compression, with the best quality parameter. It also has more flexibility and potential for improvement via training.