OpenRaiser/Intern-Atlas huggingface dataset:
One-Class Feature Learning Using Intra-Class Splitting
Venue: arXiv cs.LG 2019
Year: 2019

This paper proposes a novel generic one-class feature learning method based on intra-class splitting. In one-class classification, feature learning is challenging, because only samples of one class are available during training. Hence, state-of-the-art methods require reference multi-class datasets to pretrain feature extractors. In contrast, the proposed method realizes feature learning by splitting the given normal class into typical and atypical normal samples. By introducing closeness loss and dispersion loss, an intra-class joint training procedure between the two subsets after splitting enables the extraction of valuable features for one-class classification. Various experiments on three well-known image classification datasets demonstrate the effectiveness of our method which outperformed other baseline models in average.
Intertemporal Connections Between Query Suggestions and Search Engine Results for Politics Related Queries
Venue: arXiv cs.IR 2019
Year: 2019

This short paper deals with the combination and comparison of two data sources: Search engine results and query suggestions for 16 terms related to political candidates and parties. The data was collected before the federal election in Germany in September 2017 for a period of two months. The rank biased overlap (RBO) statistic is used to measure the similarity of the top-weighted rankings. For each search term and for both the search results and query auto-completions we study the stability of the rankings over time.
Steerable $e$PCA: Rotationally Invariant Exponential Family PCA
Venue: arXiv cs.CV 2019
Year: 2019

In photon-limited imaging, the pixel intensities are affected by photon count noise. Many applications, such as 3-D reconstruction using correlation analysis in X-ray free electron laser (XFEL) single molecule imaging, require an accurate estimation of the covariance of the underlying 2-D clean images. Accurate estimation of the covariance from low-photon count images must take into account that pixel intensities are Poisson distributed, hence the classical sample covariance estimator is sub-optimal. Moreover, in single molecule imaging, including in-plane rotated copies of all images could further improve the accuracy of covariance estimation. In this paper we introduce an efficient and accurate algorithm for covariance matrix estimation of count noise 2-D images, including their uniform planar rotations and possibly reflections. Our procedure, steerable $e$PCA, combines in a novel way two recently introduced innovations. The first is a methodology for principal component analysis (PCA) for Poisson distributions, and more generally, exponential family distributions, called $e$PCA. The second is steerable PCA, a fast and accurate procedure for including all planar rotations for PCA. The resulting principal components are invariant to the rotation and reflection of the input images. We demonstrate the efficiency and accuracy of steerable $e$PCA in numerical experiments involving simulated XFEL datasets and rotated Yale B face data.
Design and Implementation of Open LoRa for IoT
Venue: arXiv cs.NI 2019
Year: 2019

Long Range (LoRa) network is emerging as one of the most promising Low Power Wide Area (LPWA) networks, since it enables the energy-constraint devices distributed over wide areas to establish affordable connectivity. However, how to implement a cost-effective and flexible LoRa network is still an open challenge. This paper aims at exposing a feasible solution of design and implementation, allowing users to conveniently build a private LoRa network for various IoT applications. Firstly, several typical application scenarios of LoRa network are discussed. Then, the LoRa system architecture is presented with the functionality of each component. We address the hardware design and implementation of LoRa Gateway, which is the bridge between LoRa nodes and LoRa network server. Especially, the paper contributes by proposing an improved software architecture of LoRa network server whose source codes are open on GitHub. Under the architecture, LoRa network server is divided into four decoupled modules and uses the messaging system based on streaming data for the interaction between modules to guarantee scalability and flexibility. Finally, extensive experiments are conducted to evaluate the performance of LoRa networks in typical environments.
Age of Information Performance of Multiaccess Strategies with Packet Management
Venue: arXiv cs.NI 2019
Year: 2019

We consider a system consisting of $N$ source nodes communicating with a common receiver. Each source node has a buffer of infinite capacity to store incoming bursty traffic in the form of status updates transmitted in packets, which should maintain the status information at the receiver fresh. Packets waiting for transmission can be discarded to avoid wasting network resources for the transmission of stale information. We investigate the age of information (AoI) performance of the system under scheduled and random access. Moreover, we present analysis of the AoI with and without packet management at the transmission queue of the source nodes, where as packet management we consider the capability to replace unserved packets at the queue whenever newer ones arrive. Finally, we provide simulation results that illustrate the impact of the network operating parameters on the age performance of the different access protocols.
Distributed Economic Dispatch for Energy Internet Based on Multi-Agent Consensus Control
Venue: arXiv cs.SY 2019
Year: 2019

We consider the economic dispatch (ED) for an Energy Internet composed of energy routers (ERs), interconnected microgrids and main grid. The microgrid consists of several bus nodes associated with distributed generators (DGs) and intelligent control units (ICUs). We propose a distributed ED algorithm for the grid-connected microgrid, where each ICU iterates the estimated electricity price of the distribution system and the estimation for the average power mismatch of the whole microgrid by leader-following and average consensus algorithms, respectively. The ER iterates the incremental power exchanged with the distribution system. By constructing an auxiliary consensus system, we prove that if the communication topology of the Energy Internet contains a spanning tree with the ER as the root and there is a path from each ICU to the ER, then the estimated electricity price of the distribution system converges to its real value, the power supply and demand achieves balance and the ED achieves optimal asymptotically. Furthermore, we propose an autonomous distributed ED algorithm covering both grid-connected and isolated modes of the microgrid by feeding back the estimated average power mismatch for updating the incremental costs with penalty factor. It is proved that if the communication topology of the microgrid is connected and there exists an ICU bi-directionally neighboring the ER, then the microgrid can switches between the two modes reliably. The simulation results demonstrate the effectiveness of the proposed algorithms.
Fault Location in Power Distribution Systems via Deep Graph Convolutional Networks
Venue: arXiv cs.LG 2019
Year: 2019

This paper develops a novel graph convolutional network (GCN) framework for fault location in power distribution networks. The proposed approach integrates multiple measurements at different buses while taking system topology into account. The effectiveness of the GCN model is corroborated by the IEEE 123 bus benchmark system. Simulation results show that the GCN model significantly outperforms other widely-used machine learning schemes with very high fault location accuracy. In addition, the proposed approach is robust to measurement noise and data loss errors. Data visualization results of two competing neural networks are presented to explore the mechanism of GCN's superior performance. A data augmentation procedure is proposed to increase the robustness of the model under various levels of noise and data loss errors. Further experiments show that the model can adapt to topology changes of distribution networks and perform well with a limited number of measured buses.
Writer-Aware CNN for Parsimonious HMM-Based Offline Handwritten Chinese Text Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Recently, the hybrid convolutional neural network hidden Markov model (CNN-HMM) has been introduced for offline handwritten Chinese text recognition (HCTR) and has achieved state-of-the-art performance. However, modeling each of the large vocabulary of Chinese characters with a uniform and fixed number of hidden states requires high memory and computational costs and makes the tens of thousands of HMM state classes confusing. Another key issue of CNN-HMM for HCTR is the diversified writing style, which leads to model strain and a significant performance decline for specific writers. To address these issues, we propose a writer-aware CNN based on parsimonious HMM (WCNN-PHMM). First, PHMM is designed using a data-driven state-tying algorithm to greatly reduce the total number of HMM states, which not only yields a compact CNN by state sharing of the same or similar radicals among different Chinese characters but also improves the recognition accuracy due to the more accurate modeling of tied states and the lower confusion among them. Second, WCNN integrates each convolutional layer with one adaptive layer fed by a writer-dependent vector, namely, the writer code, to extract the irrelevant variability in writer information to improve recognition performance. The parameters of writer-adaptive layers are jointly optimized with other network parameters in the training stage, while a multiple-pass decoding strategy is adopted to learn the writer code and generate recognition results. Validated on the ICDAR 2013 competition of CASIA-HWDB database, the more compact WCNN-PHMM of a 7360-class vocabulary can achieve a relative character error rate (CER) reduction of 16.6% over the conventional CNN-HMM without considering language modeling. By adopting a powerful hybrid language model (N-gram language model and recurrent neural network language model), the CER of WCNN-PHMM is reduced to 3.17%.
Dynamic Runtime Feature Map Pruning
Venue: arXiv cs.LG 2019
Year: 2019

High bandwidth requirements are an obstacle for accelerating the training and inference of deep neural networks. Most previous research focuses on reducing the size of kernel maps for inference. We analyze parameter sparsity of six popular convolutional neural networks - AlexNet, MobileNet, ResNet-50, SqueezeNet, TinyNet, and VGG16. Of the networks considered, those using ReLU (AlexNet, SqueezeNet, VGG16) contain a high percentage of 0-valued parameters and can be statically pruned. Networks with Non-ReLU activation functions in some cases may not contain any 0-valued parameters (ResNet-50, TinyNet). We also investigate runtime feature map usage and find that input feature maps comprise the majority of bandwidth requirements when depth-wise convolution and point-wise convolutions used. We introduce dynamic runtime pruning of feature maps and show that 10% of dynamic feature map execution can be removed without loss of accuracy. We then extend dynamic pruning to allow for values within an epsilon of zero and show a further 5% reduction of feature map loading with a 1% loss of accuracy in top-1.
Joint Embedding Learning and Low-Rank Approximation: A Framework for Incomplete Multi-view Learning
Venue: arXiv cs.LG 2019
Year: 2019

In real-world applications, not all instances in multi-view data are fully represented. To deal with incomplete data, Incomplete Multi-view Learning (IML) rises. In this paper, we propose the Joint Embedding Learning and Low-Rank Approximation (JELLA) framework for IML. The JELLA framework approximates the incomplete data by a set of low-rank matrices and learns a full and common embedding by linear transformation. Several existing IML methods can be unified as special cases of the framework. More interestingly, some linear transformation based complete multi-view methods can be adapted to IML directly with the guidance of the framework. Thus, the JELLA framework improves the efficiency of processing incomplete multi-view data, and bridges the gap between complete multi-view learning and IML. Moreover, the JELLA framework can provide guidance for developing new algorithms. For illustration, within the framework, we propose the Incomplete Multi-view Learning with Block Diagonal Representation (IML-BDR) method. Assuming that the sampled examples have approximate linear subspace structure, IML-BDR uses the block diagonal structure prior to learn the full embedding, which would lead to more correct clustering. A convergent alternating iterative algorithm with the Successive Over-Relaxation optimization technique is devised for optimization. Experimental results on various datasets demonstrate the effectiveness of IML-BDR.
Second-Order Converses via Reverse Hypercontractivity
Venue: arXiv cs.IT 2019
Year: 2019

A strong converse shows that no procedure can beat the asymptotic (as blocklength $n\to\infty$) fundamental limit of a given information-theoretic problem for any fixed error probability. A second-order converse strengthens this conclusion by showing that the asymptotic fundamental limit cannot be exceeded by more than $O(\tfrac{1}{\sqrt{n}})$. While strong converses are achieved in a broad range of information-theoretic problems by virtue of the "blowing-up method"---a powerful methodology due to Ahlswede, Gács and Körner (1976) based on concentration of measure---this method is fundamentally unable to attain second-order converses and is restricted to finite-alphabet settings. Capitalizing on reverse hypercontractivity of Markov semigroups and functional inequalities, this paper develops the "smoothing-out" method, an alternative to the blowing-up approach that does not rely on finite alphabets and that leads to second-order converses in a variety of information-theoretic problems that were out of reach of previous methods.
A Survey of Deep Facial Attribute Analysis
Venue: arXiv cs.CV 2019
Year: 2019

Facial attribute analysis has received considerable attention when deep learning techniques made remarkable breakthroughs in this field over the past few years. Deep learning based facial attribute analysis consists of two basic sub-issues: facial attribute estimation (FAE), which recognizes whether facial attributes are present in given images, and facial attribute manipulation (FAM), which synthesizes or removes desired facial attributes. In this paper, we provide a comprehensive survey of deep facial attribute analysis from the perspectives of both estimation and manipulation. First, we summarize a general pipeline that deep facial attribute analysis follows, which comprises two stages: data preprocessing and model construction. Additionally, we introduce the underlying theories of this two-stage pipeline for both FAE and FAM. Second, the datasets and performance metrics commonly used in facial attribute analysis are presented. Third, we create a taxonomy of state-of-the-art methods and review deep FAE and FAM algorithms in detail. Furthermore, several additional facial attribute related issues are introduced, as well as relevant real-world applications. Finally, we discuss possible challenges and promising future research directions.
Spotting Micro-Expressions on Long Videos Sequences
Venue: arXiv cs.CV 2019
Year: 2019

This paper presents baseline results for the first Micro-Expression Spotting Challenge 2019 by evaluating local temporal pattern (LTP) on SAMM and CAS(ME)2. The proposed LTP patterns are extracted by applying PCA in a temporal window on several facial local regions. The micro-expression sequences are then spotted by a local classification of LTP and a global fusion. The performance is evaluated by Leave-One-Subject-Out cross validation. Furthermore, we define the criteria of determining true positives in one video by overlap rate and set the metric F1-score for spotting performance of the whole database. The F1-score of baseline results for SAMM and CAS(ME)2 are 0.0316 and 0.0179, respectively.
Identification of Cancer -- Mesothelioma Disease Using Logistic Regression and Association Rule
Venue: arXiv cs.CY 2019
Year: 2019

Malignant Pleural Mesothelioma (MPM) or malignant mesothelioma (MM) is an atypical, aggressive tumor that matures into cancer in the pleura, a stratum of tissue bordering the lungs. Diagnosis of MPM is difficult and it accounts for about seventy-five percent of all mesothelioma diagnosed yearly in the United States of America. Being a fatal disease, early identification of MPM is crucial for patient survival. Our study implements logistic regression and develops association rules to identify early stage symptoms of MM. We retrieved medical reports generated by Dicle University and implemented logistic regression to measure the model accuracy. We conducted (a) logistic correlation, (b) Omnibus test and (c) Hosmer and Lemeshow test for model evaluation. Moreover, we also developed association rules by confidence, rule support, lift, condition support and deployability. Categorical logistic regression increases the training accuracy from 72.30% to 81.40% with a testing accuracy of 63.46%. The study also shows the top 5 symptoms that is mostly likely indicates the presence in MM. This study concludes that using predictive modeling can enhance primary presentation and diagnosis of MM.
Bridge Damage Detection using a Single-Stage Detector and Field Inspection Images
Venue: arXiv cs.CY 2019
Year: 2019

Detecting concrete surface damages is a vital task for maintaining the structural health and reliability of highway bridges. Currently, most of these tasks are conducted manually which could be cumbersome and time-consuming. Recent rapid advancement in convolution neural network has enabled the development of deep learning-based visual inspection techniques for detecting multiple structural damages. However, most deep learning-based techniques are built on two-stage, proposal-driven detectors and using less complex image data, which is not promising to promote practical applications and integration within intelligent autonomous inspection systems. In this study, a faster, simpler single-stage detector is proposed based on YOLOv3 for detecting multiple concrete damages of highway bridges. To realize this, a bridge damage dataset consisting of field inspection images is prepared. These images have large variations in damage appearance and monitoring scene and are labeled with four types of concrete damages: crack, pop-out, spalling and exposed rebar. The original YOLOv3 is further improved by introducing a novel transfer learning method, Batch Renormalization and Focal Loss. The improved YOLOv3 is evaluated in terms of average precision and speed. The results show that the improved YOLOv3 has a detection accuracy of up to 80%. Its performance is about 13% better than the original YOLOv3.
Stanza: Layer Separation for Distributed Training in Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

The parameter server architecture is prevalently used for distributed deep learning. Each worker machine in a parameter server system trains the complete model, which leads to a hefty amount of network data transfer between workers and servers. We empirically observe that the data transfer has a non-negligible impact on training time. To tackle the problem, we design a new distributed training system called Stanza. Stanza exploits the fact that in many models such as convolution neural networks, most data exchange is attributed to the fully connected layers, while most computation is carried out in convolutional layers. Thus, we propose layer separation in distributed training: the majority of the nodes just train the convolutional layers, and the rest train the fully connected layers only. Gradients and parameters of the fully connected layers no longer need to be exchanged across the cluster, thereby substantially reducing the data transfer volume. We implement Stanza on PyTorch and evaluate its performance on Azure and EC2. Results show that Stanza accelerates training significantly over current parameter server systems: on EC2 instances with Tesla V100 GPU and 10Gb bandwidth for example, Stanza is 1.34x--13.9x faster for common deep learning models.
TripleAgent: Monitoring, Perturbation and Failure-obliviousness for Automated Resilience Improvement in Java Applications
Venue: arXiv cs.SE 2019
Year: 2019

In this paper, we present a novel resilience improvement system for Java applications. The unique feature of this system is to combine automated monitoring, automated perturbation injection, and automated resilience improvement. The latter is achieved thanks to the failure-oblivious computing, a concept introduced in 2004 by Rinard and colleagues. We design and implement the system as agents for the Java virtual machine. We evaluate the system on two real-world applications: a file transfer client and an email server. Our results show that it is possible to automatically improve the resilience of Java applications with respect to uncaught or mishandled exceptions.
Adaptive Image Sampling using Deep Learning and its Application on X-Ray Fluorescence Image Reconstruction
Venue: arXiv cs.CV 2019
Year: 2019

This paper presents an adaptive image sampling algorithm based on Deep Learning (DL). The adaptive sampling mask generation network is jointly trained with an image inpainting network. The sampling rate is controlled in the mask generation network, and a binarization strategy is investigated to make the sampling mask binary. Besides the image sampling and reconstruction application, we show that the proposed adaptive sampling algorithm is able to speed up raster scan processes such as the X-Ray fluorescence (XRF) image scanning process. Recently XRF laboratory-based systems have evolved to lightweight and portable instruments thanks to technological advancements in both X-Ray generation and detection. However, the scanning time of an XRF image is usually long due to the long exposures requires (e.g., $100 μs-1ms$ per point). We propose an XRF image inpainting approach to address the issue of long scanning time, thus speeding up the scanning process while still maintaining the possibility to reconstruct a high quality XRF image. The proposed adaptive image sampling algorithm is applied to the RGB image of the scanning target to generate the sampling mask. The XRF scanner is then driven according to the sampling mask to scan a subset of the total image pixels. Finally, we inpaint the scanned XRF image by fusing the RGB image to reconstruct the full scan XRF image. The experiments show that the proposed adaptive sampling algorithm is able to effectively sample the image and achieve a better reconstruction accuracy than that of the existing methods.
On Computation and Generalization of GANs with Spectrum Control
Venue: arXiv cs.LG 2019
Year: 2019

Generative Adversarial Networks (GANs), though powerful, is hard to train. Several recent works (brock2016neural,miyato2018spectral) suggest that controlling the spectra of weight matrices in the discriminator can significantly improve the training of GANs. Motivated by their discovery, we propose a new framework for training GANs, which allows more flexible spectrum control (e.g., making the weight matrices of the discriminator have slow singular value decays). Specifically, we propose a new reparameterization approach for the weight matrices of the discriminator in GANs, which allows us to directly manipulate the spectra of the weight matrices through various regularizers and constraints, without intensively computing singular value decompositions. Theoretically, we further show that the spectrum control improves the generalization ability of GANs. Our experiments on CIFAR-10, STL-10, and ImageNet datasets confirm that compared to other methods, our proposed method is capable of generating images with competitive quality by utilizing spectral normalization and encouraging the slow singular value decay.