OpenRaiser/Intern-Atlas huggingface dataset:
Combining Multiple Algorithms in Classifier Ensembles using Generalized Mixture Functions
Venue: arXiv cs.LG 2018
Year: 2018

Classifier ensembles are pattern recognition structures composed of a set of classification algorithms (members), organized in a parallel way, and a combination method with the aim of increasing the classification accuracy of a classification system. In this study, we investigate the application of a generalized mixture (GM) functions as a new approach for providing an efficient combination procedure for these systems through the use of dynamic weights in the combination process. Therefore, we present three GM functions to be applied as a combination method. The main advantage of these functions is that they can define dynamic weights at the member outputs, making the combination process more efficient. In order to evaluate the feasibility of the proposed approach, an empirical analysis is conducted, applying classifier ensembles to 25 different classification data sets. In this analysis, we compare the use of the proposed approaches to ensembles using traditional combination methods as well as the state-of-the-art ensemble methods. Our findings indicated gains in terms of performance when comparing the proposed approaches to the traditional ones as well as comparable results with the state-of-the-art methods.
Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
Venue: arXiv cs.CV 2018
Year: 2018

Learning rate scheduler has been a critical issue in the deep neural network training. Several schedulers and methods have been proposed, including step decay scheduler, adaptive method, cosine scheduler and cyclical scheduler. This paper proposes a new scheduling method, named hyperbolic-tangent decay (HTD). We run experiments on several benchmarks such as: ResNet, Wide ResNet and DenseNet for CIFAR-10 and CIFAR-100 datasets, LSTM for PAMAP2 dataset, ResNet on ImageNet and Fashion-MNIST datasets. In our experiments, HTD outperforms step decay and cosine scheduler in nearly all cases, while requiring less hyperparameters than step decay, and more flexible than cosine scheduler. Code is available at https://github.com/BIGBALLON/HTD.
Training Generative Reversible Networks
Venue: arXiv cs.LG 2018
Year: 2018

Generative models with an encoding component such as autoencoders currently receive great interest. However, training of autoencoders is typically complicated by the need to train a separate encoder and decoder model that have to be enforced to be reciprocal to each other. To overcome this problem, by-design reversible neural networks (RevNets) had been previously used as generative models either directly optimizing the likelihood of the data under the model or using an adversarial approach on the generated data. Here, we instead investigate their performance using an adversary on the latent space in the adversarial autoencoder framework. We investigate the generative performance of RevNets on the CelebA dataset, showing that generative RevNets can generate coherent faces with similar quality as Variational Autoencoders. This first attempt to use RevNets inside the adversarial autoencoder framework slightly underperformed relative to recent advanced generative models using an autoencoder component on CelebA, but this gap may diminish with further optimization of the training setup of generative RevNets. In addition to the experiments on CelebA, we show a proof-of-principle experiment on the MNIST dataset suggesting that adversary-free trained RevNets can discover meaningful latent dimensions without pre-specifying the number of dimensions of the latent sampling distribution. In summary, this study shows that RevNets can be employed in different generative training settings. Source code for this study is at https://github.com/robintibor/generative-reversible
Numerical Integration as an Initial Value Problem
Venue: arXiv cs.NA 2018
Year: 2018

Numerical integration (NI) packages commonly used in scientific research are limited to returning the value of a definite integral at the upper integration limit, also commonly referred to as numerical quadrature. These quadrature algorithms are typically of a fixed accuracy and have only limited ability to adapt to the application. In this article, we will present a highly adaptive algorithm that not only can efficiently compute definite integrals encountered in physical problems but also can be applied to other problems such as indefinite integrals, integral equations and linear and non-linear eigenvalue problems. More specifically, a finite element based algorithm is presented that numerically solves first order ordinary differential equations (ODE) by propagating the solution function from a given initial value (lower integration value). The algorithm incorporates powerful techniques including, adaptive step size choice of elements, local error checking and enforces continuity of both the integral and the integrand across consecutive elements.
Hierarchical Graph Clustering using Node Pair Sampling
Venue: arXiv cs.SI 2018
Year: 2018

We present a novel hierarchical graph clustering algorithm inspired by modularity-based clustering techniques. The algorithm is agglomerative and based on a simple distance between clusters induced by the probability of sampling node pairs. We prove that this distance is reducible, which enables the use of the nearest-neighbor chain to speed up the agglomeration. The output of the algorithm is a regular dendrogram, which reveals the multi-scale structure of the graph. The results are illustrated on both synthetic and real datasets.
Adapting Neural Text Classification for Improved Software Categorization
Venue: arXiv cs.SE 2018
Year: 2018

Software Categorization is the task of organizing software into groups that broadly describe the behavior of the software, such as "editors" or "science." Categorization plays an important role in several maintenance tasks, such as repository navigation and feature elicitation. Current approaches attempt to cast the problem as text classification, to make use of the rich body of literature from the NLP domain. However, as we will show in this paper, text classification algorithms are generally not applicable off-the-shelf to source code; we found that they work well when high-level project descriptions are available, but suffer very large performance penalties when classifying source code and comments only. We propose a set of adaptations to a state-of-the-art neural classification algorithm and perform two evaluations: one with reference data from Debian end-user programs, and one with a set of C/C++ libraries that we hired professional programmers to annotate. We show that our proposed approach achieves performance exceeding that of previous software classification techniques as well as a state-of-the-art neural text classification technique.
Deployment of a Blockchain-Based Self-Sovereign Identity
Venue: arXiv cs.DC 2018
Year: 2018

Digital identity is unsolved: after many years of research there is still no trusted communication over the Internet. To provide identity within the context of mutual distrust, this paper presents a blockchain-based digital identity solution. Without depending upon a single trusted third party, the proposed solution achieves passport-level legally valid identity. This solution for making identities Self-Sovereign, builds on a generic provable claim model for which attestations of truth from third parties need to be collected. The claim model is then shown to be both blockchain structure and proof method agnostic. Four different implementations in support of these two claim model properties are shown to offer sub-second performance for claim creation and claim verification. Through the properties of Self-Sovereign Identity, legally valid status and acceptable performance, our solution is considered to be fit for adoption by the general public.
Automating Network Error Detection using Long-Short Term Memory Networks
Venue: arXiv cs.NI 2018
Year: 2018

In this work, we investigate the current flaws with identifying network-related errors, and examine how K-Means and Long-Short Term Memory Networks solve these problems. We demonstrate that K-Means is able to classify messages, but not necessary provide meaningful clusters. However, Long-Short Term Memory Networks are able to meet our goals of providing an intelligent clustering of messages by grouping messages that are temporally related. Additionally, Long-Short Term Memory Networks can provide the ability to understand and visualize temporal causality, which unlocks the ability to warn about errors before they happen. We show that LSTMs have a 70% accuracy on classifying network errors, and provide some suggestions on future work.
Another Proof of Cuckoo hashing with New Variants
Venue: arXiv cs.DS 2018
Year: 2018

We show a new proof for the load of obtained by a Cuckoo Hashing data structure. Our proof is arguably simpler than previous proofs and allows for new generalizations. The proof first appeared in Pinkas et. al. \cite{PSWW19} in the context of a protocol for private set intersection. We present it here separately to improve its readability.
Swift-Link: A compressive beam alignment algorithm for practical mmWave radios
Venue: arXiv cs.IT 2018
Year: 2018

Next generation wireless networks will exploit the large amount of spectrum available at millimeter wave (mmWave) frequencies. Design of mmWave systems, however, is challenging due to strict power, cost and hardware constraints at higher bandwidths. To achieve a good SNR for communication, mmWave systems use large antenna arrays. Beamforming with highly directional beams is one way to use the antennas. As the channel changes over time, the beams that maximize the SNR have to be estimated quickly to reduce the training overhead. Prior work has exploited the observation that mmWave channels are sparse to perform compressed sensing (CS) based beam alignment with few channel measurements. Most of the existing CS-based algorithms, however, assume perfect synchronization and fail in the presence of carrier frequency offset (CFO). This paper presents Swift-Link, a fast beam alignment algorithm that is robust against the offset. Swift-Link includes a novel randomized beam training sequence that minimizes the beam alignment errors due to CFO and a low-complexity algorithm that corrects these errors. Even with strict hardware constraints, our algorithm uses fewer channel measurements than comparable CS algorithms and has analytical guarantees. Swift-Link requires a small output dynamic range at the analog-to-digital converter compared to beam-scanning techniques.
Distributed Hypothesis Testing with Privacy Constraints
Venue: arXiv cs.IT 2018
Year: 2018

We revisit the distributed hypothesis testing (or hypothesis testing with communication constraints) problem from the viewpoint of privacy. Instead of observing the raw data directly, the transmitter observes a sanitized or randomized version of it. We impose an upper bound on the mutual information between the raw and randomized data. Under this scenario, the receiver, which is also provided with side information, is required to make a decision on whether the null or alternative hypothesis is in effect. We first provide a general lower bound on the type-II exponent for an arbitrary pair of hypotheses. Next, we show that if the distribution under the alternative hypothesis is the product of the marginals of the distribution under the null (i.e., testing against independence), then the exponent is known exactly. Moreover, we show that the strong converse property holds. Using ideas from Euclidean information theory, we also provide an approximate expression for the exponent when the communication rate is low and the privacy level is high. Finally, we illustrate our results with a binary and a Gaussian example.
Binary linear code weight distribution estimation by random bit stream compression
Venue: arXiv cs.IT 2018
Year: 2018

A statistical estimation algorithm of the weight distribution of a linear code is shown, based on using its generator matrix as a compression function on random bit strings.
Efficient Differentiable Programming in a Functional Array-Processing Language
Venue: arXiv cs.MS 2018
Year: 2018

We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations. Thanks to this feature, we demonstrate how for some real-world machine learning and computer vision benchmarks, the system outperforms the state-of-the-art automatic differentiation tools.
StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks
Venue: arXiv cs.SD 2018
Year: 2018

This paper proposes a method that allows non-parallel many-to-many voice conversion (VC) by using a variant of a generative adversarial network (GAN) called StarGAN. Our method, which we call StarGAN-VC, is noteworthy in that it (1) requires no parallel utterances, transcriptions, or time alignment procedures for speech generator training, (2) simultaneously learns many-to-many mappings across different attribute domains using a single generator network, (3) is able to generate converted speech signals quickly enough to allow real-time implementations and (4) requires only several minutes of training examples to generate reasonably realistic-sounding speech. Subjective evaluation experiments on a non-parallel many-to-many speaker identity conversion task revealed that the proposed method obtained higher sound quality and speaker similarity than a state-of-the-art method based on variational autoencoding GANs.
Joint Altitude, Beamwidth, Location and Bandwidth Optimization for UAV-Enabled Communications
Venue: arXiv cs.IT 2018
Year: 2018

This letter investigates an uplink power control problem for unmanned aerial vehicles (UAVs) assisted wireless communications. We jointly optimize the UAV's flying altitude, antenna beamwidth, UAV's location and ground terminals' allocated bandwidth and transmit power to minimize the sum uplink power subject to the minimal rate demand. An iterative algorithm is proposed with low complexity to obtain a suboptimal solution. Numerical results show that the proposed algorithm can achieve good performance in terms of uplink sum power saving.
A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation
Venue: arXiv cs.CV 2018
Year: 2018

Recent advances in deep learning, like 3D fully convolutional networks (FCNs), have improved the state-of-the-art in dense semantic segmentation of medical images. However, most network architectures require severely downsampling or cropping the images to meet the memory limitations of today's GPU cards while still considering enough context in the images for accurate segmentation. In this work, we propose a novel approach that utilizes auto-context to perform semantic segmentation at higher resolutions in a multi-scale pyramid of stacked 3D FCNs. We train and validate our models on a dataset of manually annotated abdominal organs and vessels from 377 clinical CT images used in gastric surgery, and achieve promising results with close to 90% Dice score on average. For additional evaluation, we perform separate testing on datasets from different sources and achieve competitive results, illustrating the robustness of the model and approach.
The Limitations of Cross-language Word Embeddings Evaluation
Venue: arXiv cs.CL 2018
Year: 2018

The aim of this work is to explore the possible limitations of existing methods of cross-language word embeddings evaluation, addressing the lack of correlation between intrinsic and extrinsic cross-language evaluation methods. To prove this hypothesis, we construct English-Russian datasets for extrinsic and intrinsic evaluation tasks and compare performances of 5 different cross-language models on them. The results say that the scores even on different intrinsic benchmarks do not correlate to each other. We can conclude that the use of human references as ground truth for cross-language word embeddings is not proper unless one does not understand how do native speakers process semantics in their cognition.
Deep Vessel Segmentation By Learning Graphical Connectivity
Venue: arXiv cs.CV 2018
Year: 2018

We propose a novel deep-learning-based system for vessel segmentation. Existing methods using CNNs have mostly relied on local appearances learned on the regular image grid, without considering the graphical structure of vessel shape. To address this, we incorporate a graph convolutional network into a unified CNN architecture, where the final segmentation is inferred by combining the different types of features. The proposed method can be applied to expand any type of CNN-based vessel segmentation method to enhance the performance. Experiments show that the proposed method outperforms the current state-of-the-art methods on two retinal image datasets as well as a coronary artery X-ray angiography dataset.
Dilatation of Lateral Ventricles with Brain Volumes in Infants with 3D Transfontanelle US
Venue: arXiv cs.CV 2018
Year: 2018

Ultrasound (US) can be used to assess brain development in newborns, as MRI is challenging due to immobilization issues, and may require sedation. Dilatation of the lateral ventricles in the brain is a risk factor for poorer neurodevelopment outcomes in infants. Hence, 3D US has the ability to assess the volume of the lateral ventricles similar to clinically standard MRI, but manual segmentation is time consuming. The objective of this study is to develop an approach quantifying the ratio of lateral ventricular dilatation with respect to total brain volume using 3D US, which can assess the severity of macrocephaly. Automatic segmentation of the lateral ventricles is achieved with a multi-atlas deformable registration approach using locally linear correlation metrics for US-MRI fusion, followed by a refinement step using deformable mesh models. Total brain volume is estimated using a 3D ellipsoid modeling approach. Validation was performed on a cohort of 12 infants, ranging from 2 to 8.5 months old, where 3D US and MRI were used to compare brain volumes and segmented lateral ventricles. Automatically extracted volumes from 3D US show a high correlation and no statistically significant difference when compared to ground truth measurements. Differences in volume ratios was 6.0 +/- 4.8% compared to MRI, while lateral ventricular segmentation yielded a mean Dice coefficient of 70.8 +/- 3.6% and a mean absolute distance (MAD) of 0.88 +/- 0.2mm, demonstrating the clinical benefit of this tool in paediatric ultrasound.
Design of CMOS-memristor Circuits for LSTM architecture
Venue: arXiv cs.ET 2018
Year: 2018

Long Short-Term memory (LSTM) architecture is a well-known approach for building recurrent neural networks (RNN) useful in sequential processing of data in application to natural language processing. The near-sensor hardware implementation of LSTM is challenged due to large parallelism and complexity. We propose a 0.18 m CMOS, GST memristor LSTM hardware architecture for near-sensor processing. The proposed system is validated in a forecasting problem based on Keras model.
Predictive Analysis on Twitter: Techniques and Applications
Venue: arXiv cs.SI 2018
Year: 2018

Predictive analysis of social media data has attracted considerable attention from the research community as well as the business world because of the essential and actionable information it can provide. Over the years, extensive experimentation and analysis for insights have been carried out using Twitter data in various domains such as healthcare, public health, politics, social sciences, and demographics. In this chapter, we discuss techniques, approaches and state-of-the-art applications of predictive analysis of Twitter data. Specifically, we present fine-grained analysis involving aspects such as sentiment, emotion, and the use of domain knowledge in the coarse-grained analysis of Twitter data for making decisions and taking actions, and relate a few success stories.
Finding Convincing Arguments Using Scalable Bayesian Preference Learning
Venue: arXiv cs.CL 2018
Year: 2018

We introduce a scalable Bayesian preference learning method for identifying convincing arguments in the absence of gold-standard rat- ings or rankings. In contrast to previous work, we avoid the need for separate methods to perform quality control on training data, predict rankings and perform pairwise classification. Bayesian approaches are an effective solution when faced with sparse or noisy training data, but have not previously been used to identify convincing arguments. One issue is scalability, which we address by developing a stochastic variational inference method for Gaussian process (GP) preference learning. We show how our method can be applied to predict argument convincingness from crowdsourced data, outperforming the previous state-of-the-art, particularly when trained with small amounts of unreliable data. We demonstrate how the Bayesian approach enables more effective active learning, thereby reducing the amount of data required to identify convincing arguments for new users and domains. While word embeddings are principally used with neural networks, our results show that word embeddings in combination with linguistic features also benefit GPs when predicting argument convincingness.
Human-aided Multi-Entity Bayesian Networks Learning from Relational Data
Venue: arXiv cs.LG 2018
Year: 2018

An Artificial Intelligence (AI) system is an autonomous system which emulates human mental and physical activities such as Observe, Orient, Decide, and Act, called the OODA process. An AI system performing the OODA process requires a semantically rich representation to handle a complex real world situation and ability to reason under uncertainty about the situation. Multi-Entity Bayesian Networks (MEBNs) combines First-Order Logic with Bayesian Networks for representing and reasoning about uncertainty in complex, knowledge-rich domains. MEBN goes beyond standard Bayesian networks to enable reasoning about an unknown number of entities interacting with each other in various types of relationships, a key requirement for the OODA process of an AI system. MEBN models have heretofore been constructed manually by a domain expert. However, manual MEBN modeling is labor-intensive and insufficiently agile. To address these problems, an efficient method is needed for MEBN modeling. One of the methods is to use machine learning to learn a MEBN model in whole or in part from data. In the era of Big Data, data-rich environments, characterized by uncertainty and complexity, have become ubiquitous. The larger the data sample is, the more accurate the results of the machine learning approach can be. Therefore, machine learning has potential to improve the quality of MEBN models as well as the effectiveness for MEBN modeling. In this research, we study a MEBN learning framework to develop a MEBN model from a combination of domain expert's knowledge and data. To evaluate the MEBN learning framework, we conduct an experiment to compare the MEBN learning framework and the existing manual MEBN modeling in terms of development efficiency.
NumtaDB - Assembled Bengali Handwritten Digits
Venue: arXiv cs.CV 2018
Year: 2018

To benchmark Bengali digit recognition algorithms, a large publicly available dataset is required which is free from biases originating from geographical location, gender, and age. With this aim in mind, NumtaDB, a dataset consisting of more than 85,000 images of hand-written Bengali digits, has been assembled. This paper documents the collection and curation process of numerals along with the salient statistics of the dataset.