Reviewerly/exHarmony huggingface dataset:
Wasserstein GAN With Quadratic Transport Cost

Wasserstein GANs are increasingly used in Computer Vision applications as they are easier to train. Previous WGAN variants mainly use the lι transport cost to compute the Wasserstein distance between the real and synthetic data distributions. The lι transport cost restricts the discriminator to be 1-Lipschitz. However, WGANs with lι transport cost were recently shown to not always converge. In this paper, we propose WGAN-QC, a WGAN with quadratic transport cost. Based on the quadratic transport cost, we propose an Optimal Transport Regularizer (OTR) to stabilize the training process of WGAN-QC. We prove that the objective of the discriminator during each generator update computes the exact quadratic Wasserstein distance between real and synthetic data distributions. We also prove that WGAN-QC converges to a local equilibrium point with finite discriminator updates per generator update. We show experimentally on a Dirac distribution that WGAN-QC converges, when many of the lι cost WGANs fail to [22]. Qualitative and quantitative results on the CelebA, CelebA-HQ, LSUN and the ImageNet dog datasets show that WGAN-QC is better than state-of-art GAN methods. WGAN-QC has much faster runtime than other WGAN variants.
Anomaly Detections for Manufacturing Systems Based on Sensor Data—Insights into Two Challenging Real-World Production Settings

To build, run, and maintain reliable manufacturing machines, the condition of their components has to be continuously monitored. When following a fine-grained monitoring of these machines, challenges emerge pertaining to the (1) feeding procedure of large amounts of sensor data to downstream processing components and the (2) meaningful analysis of the produced data. Regarding the latter aspect, manifold purposes are addressed by practitioners and researchers. Two analyses of real-world datasets that were generated in production settings are discussed in this paper. More specifically, the analyses had the goals (1) to detect sensor data anomalies for further analyses of a pharma packaging scenario and (2) to predict unfavorable temperature values of a 3D printing machine environment. Based on the results of the analyses, it will be shown that a proper management of machines and their components in industrial manufacturing environments can be efficiently supported by the detection of anomalies. The latter shall help to support the technical evangelists of the production companies more properly.
A New Feature Pyramid Network for Object Detection

Aiming at the problems of high computational cost based on deep backbones (e.g., ResNet-50, ResNet-101, DenseNet-169) in the state-of-the-art method about object detector, this paper improves the capability of feature representations by using New Feature Pyramid module on the basis of fast lightweight backbone network (vgg-16), and finally establishes a fast and accurate detector. The architecture of our model is named New FPN (New Feature Pyramid Network). Based on the structure of Feature Pyramid Network, we design a novel New Feature Pyramid Network, which consists of a combination of top-down and bottom-up connections to fuse features across scales, and achieves high-level semantic feature map at all scales. The experimental results show that New FPN achieves state-of-the-art detection accuracy (i.e. 79.2%mAP) on PASCAL VOC 2007 with high efficiency (i.e. 73FPS).
Gated CNN: Integrating multi-scale feature layers for object detection

Different convolutional layers in an explainable CNN usually encode different kinds of semantic information for an image, thus the feature fusion approaches like SSD, DSSD, and FPN are widely employed to enhance the detection performance by integrating different results based on multiple convolutional layers. However, the typical fusion approaches first need to independently detect objects based on one convolutional layer before fusion, and this single layer may exist noises or be irrelevant to objects, resulting in detection failure. To tackle the above problem, this paper proposes "Gated CNN" (short for "G-CNN") to introduce a "gate" structure to integrate multiple convolutional layers for object detection. Injected by multi-scale feature layers, a gate employs several filters to extract useful information and block noises by executing one more convolutional or deconvolutional operation simultaneously, thus a gate-based feature layer is more effective and efficient as compared to the convolutional one. Besides, G-CNN employs a detector with two branches to predict the locations and categories of objects, respectively, as well as an inter-class loss to help detectors learn discrepant information among categories. Therefore, the learned detectors could better differentiate similar objects of different categories. Extensive experiments are conducted on two image datasets (PASCAL VOC and COCO), and the results demonstrate that G-CNN outperforms the state-of-the-art approaches, with a mAP of 40.9% at 10.6 FPS.
Zero-Shot Semantic Segmentation via Variational Mapping

We have witnessed the explosive success of deep neural networks (DNNs). However, DNNs typically assume a large amount of training data, and this is not always available in practical scenarios. In this paper, we present zero-shot semantic segmentation, where a model that has never seen the target class during training. For this purpose, we propose variational mapping, which facilitates effective learning by mapping the class label embedding vectors from the semantic space to the visual space. Experimental results using Pascal VOC 2012 show that our proposed method can achieve a mean intersection over union (mIoU) of 42.2, and we believe that this can serve as a baseline for similar research in the future.
Distributed K-Means clustering guaranteeing local differential privacy

In many cases, a service provider might require to aggregate data from end-users to perform mining tasks such as K-means clustering. Nevertheless, since such data often contain sensitive information. In this paper, we propose the first locally differentially private K-means mechanism under this distributed scenario. Differing from standard differentially private clustering mechanisms, the proposed mechanism doesn't need any trusted third party to collect and preprocess users data. Our mechanism first perturbs users data locally to satisfy local differential privacy (LDP). Then it revises the traditional K-means algorithm to allow the service provider to obtain high-quality clustering results by collaborating with users based on the highly perturbed data. We prove that our mechanism can enable high utility clustering while guaranteeing local differential privacy for each user. We also propose an extended mechanism to improve our basic model in terms of privacy and utility. In this mechanism, we perturb both users' sensitive data and the intermediate results of users' clusters in each iteration. Moreover, we consider a more general setting where the users may have different privacy requirements. Extensive experiments are conducted on two real-world datasets, and the results show that our proposal can well preserve the quality of clustering results.
Introduction to convolutional neural network using Keras; an understanding from a statistician

Deep Learning is one of the machine learning methods to find features from a huge data using non-linear transformation. It is now commonly used for supervised learning in many fields. In particular, Convolutional Neural Network (CNN) is the best technique for the image classification since 2012. For users who consider deep learning models for real-world applications, Keras is a popular API for neural networks written in Python and also can be used in R. We try examine the parameter estimation procedures of Deep Neural Network and structures of CNN models from basics to advanced techniques. We also try to figure out some crucial steps in CNN that can improve image classification performance in the CIFAR10 dataset using Keras. We found that several stacks of convolutional layers and batch normalization could improve prediction performance. We also compared image classification performances with other machine learning methods, including K-Nearest Neighbors (K-NN), Random Forest, and XGBoost, in both MNIST and CIFAR10 dataset.
Unsupervised Online Anomaly Detection With Parameter Adaptation for KPI Abrupt Changes

IT companies need to monitor various Key Performance Indicators (KPIs) and detect anomalies in real time to ensure the quality and reliability of Internet-based services. However, due to the diversity of KPIs, the ambiguity and scarcity of anomalies and the lack of labels, anomaly detection for various KPIs has been a great challenge. Existing KPI anomaly detection methods have not explored the properties of anomalies in KPIs in detail to our best knowledge. Therefore, we explore anomalies in KPIs and recognize a common and important form of anomalies named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">abrupt changes</i> , which often indicate potential failures in the relevant services. For <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">abrupt changes</i> in various KPIs, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDCOL</i> , an unsupervised online anomaly detection algorithm with parameter adaptation from the perspective of anomalies for the first time. We propose three techniques: high order <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${D}$ </tex-math></inline-formula> ifference extraction and combination, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${D}$ </tex-math></inline-formula> ensity-based <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${C}$ </tex-math></inline-formula> lustering with parameter adaptation and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${O}\text{n}{L}$ </tex-math></inline-formula> ine detection with subsampling ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDCOL</i> ). Compared with traditional statistical methods and unsupervised learning methods, extensive experimental results and analysis on a large number of public KPIs show the competitive performance of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDCOL</i> and the significance of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">abrupt changes</i> . Furthermore, we provide an interpretation for the promising results, which shows that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DDCOL</i> can be robust to KPI expected concept drifts, and obtain a good feature distribution of normal data in KPIs.
Multiclass imbalanced learning with one-versus-one decomposition and spectral clustering

In many real-world applications, an algorithm needs to learn multiclass classification models from data with imbalanced class distributions. Multiclass imbalanced learning is currently receiving increased attention from researchers. In contrast to traditional imbalanced learning on binary datasets, multiclass imbalanced learning faces great challenges from the variety of changes in the class distributions as well as the inadequate performance of multiclass classification algorithms. In this paper, we propose a novel data preprocessing-based method to solve this problem. The proposed method combines a one-versus-one (OVO) decomposition of class pairs and a spectral clustering technique. This method first decomposes a multiclass dataset into several binary-class datasets. Then, it uses spectral clustering to divide the minority classes of binary-class subsets into subspaces and oversamples them according to the characteristics of the data. Sampling based on spectral clustering takes into account the distribution of the data and effectively avoids oversampling outliers. After the data approximately reaches the equilibrium point, multiclass classifiers can be trained from these rebalanced data. We compared the proposed method with five state-of-the-art multiclass imbalanced learning methods on seven multiclass datasets, using multiclass area under the ROC curve (MAUC), the precision of minor classes (Pmin) and the average precision of all classes (Pavg) as the performance metrics. The experimental results show that our proposed method has the best overall performance.
Mixed Precision Neural Architecture Search for Energy Efficient Deep Learning

Large scale deep neural networks (DNNs) have achieved remarkable successes in various artificial intelligence applications. However, high computational complexity and energy costs of DNNs impede their deployment on edge devices with a limited energy budget. Two major approaches have been investigated for learning compact and energy-efficient DNNs. Neural architecture search (NAS) enables the design automation of neural network structures to achieve both high accuracy and energy efficiency. The other one, model quantization, leverages low-precision representation and arithmetic to trade off efficiency against accuracy. Although NAS and quantization are both critical components of the DNN design closure, limited research considered them collaboratively. In this paper, we propose a new methodology to perform end-to-end joint optimization over the neural architecture and quantization space. Our approach searches for the optimal combinations of architectures and precisions (bit-widths) to directly optimize both the prediction accuracy and hardware energy consumption. Our framework improves and automatizes the flow across neural architecture design and hardware deployment. Experimental results demonstrate that our proposed approach achieves better energy efficiency than advanced quantization approaches and efficiency-aware NAS methods on CIFAR-100 and ImageNet. We study different search and quantization policies, and offer insights for both neural architecture and hardware designs.
An Intelligent Swarm Based Prediction Approach For Predicting Cloud Computing User Resource Needs

Cloud computing aimed at offering elastic resource allocation on demand to cloud consumers. Building a cloud resource demand prediction model is a challenging task because cloud consumer's demand changes over time. Most of the current works in cloud resource demand prediction study the problem from the cloud provider perspective. In this paper, we study the problem from the cloud consumer perspective with the aim to help the consumers to meet their resource needs, derive estimates for the required IT budget, select the best cloud providers, and get better pricing through the advanced reservation of required cloud resources. We develop a new Swarm Intelligence Based Prediction Approach (SIBPA) to predict with higher accuracy the resource needs of a cloud consumer in terms of CPU, memory, and disk storage utilization. The SIBPA is also able to predict the response time and throughput which in turn enables the cloud consumers to make a better scaling decision. It also takes into account the dynamic behavior of consumer requests in a long term period and the seasonal or/and trend patterns in time series. The SIBPA uses the Particle Swarm Optimization (PSO) approach for selecting the best features from the dataset and for estimating the parameters of the prediction algorithms. The experimental results reveal that the prediction accuracy of the SIBPA outperforms the current prediction models. In terms of CPU utilization prediction, the accuracy of SIBPA outperforms the accuracy of the existing cloud consumer prediction frameworks that use Linear Regression, Neural Network, and Support Vector Machines approaches by 56.95%, 80.42%, and 63.86% respectively according to RMSE, and by 72.66%, 44.24%, and 56.78% according to MAPE. The accuracy of SIBPA also outperforms the accuracy of the existing cloud consumer prediction frameworks in terms of response time, throughput, and memory utilization predictions. The analysis and experiment results of SIBPA are discussed in detail in this paper.
Learning Backtrackless Aligned-Spatial Graph Convolutional Networks for Graph Classification

In this paper, we develop a novel backtrackless aligned-spatial graph convolutional network (BASGCN) model to learn effective features for graph classification. Our idea is to transform arbitrary-sized graphs into fixed-sized backtrackless aligned grid structures and define a new spatial graph convolution operation associated with the grid structures. We show that the proposed BASGCN model not only reduces the problems of information loss and imprecise information representation arising in existing spatially-based graph convolutional network (GCN) models, but also bridges the theoretical gap between traditional convolutional neural network (CNN) models and spatially-based GCN models. Furthermore, the proposed BASGCN model can both adaptively discriminate the importance between specified vertices during the convolution process and reduce the notorious tottering problem of existing spatially-based GCNs related to the Weisfeiler-Lehman algorithm, explaining the effectiveness of the proposed model. Experiments on standard graph datasets demonstrate the effectiveness of the proposed model.
An Audio Encryption Algorithm Based on DNA Coding and Chaotic System

Transferring multimedia files like audio is a common problem with information security. Therefore, various encryption technologies are needed to protect these contents. This paper proposes a new audio encryption scheme that provides a high degree of security. The novelty of this scheme is the use of chaotic systems and DNA coding to confuse and diffuse audio data. The initial value of the chaotic system is controlled by the hash value of the audio, making the chaotic trajectory unpredictable. Comparison experiments using different types of audio show that the algorithm works well and is secure enough to withstand many common attacks and can be recommended for multi-channel audio processing.
Unimodal regularized neuron stick-breaking for ordinal classification

This paper targets for the ordinal regression/classification, which objective is to learn a rule to predict labels from a discrete but ordered set. For instance, the classification for medical diagnosis usually involves inherently ordered labels corresponding to the level of health risk. Previous multi-task classifiers on ordinal data often use several binary classification branches to compute a series of cumulative probabilities. However, these cumulative probabilities are not guaranteed to be monotonically decreasing. It also introduces a large number of hyper-parameters to be fine-tuned manually. This paper aims to eliminate or at least largely reduce the effects of those problems. We propose a simple yet efficient way to rephrase the output layer of the conventional deep neural network. Besides, in order to alleviate the effects of label noise in ordinal datasets, we propose a unimodal label regularization strategy. It also explicitly encourages the class predictions to distribute on nearby classes of ground truth. We show that our methods lead to the state-of-the-art accuracy on the medical diagnose task (e.g., Diabetic Retinopathy and Ultrasound Breast dataset) as well as the face age prediction (e.g., Adience face and MORPH Album II) with very little additional cost.
Deep learning for automated sentiment analysis of social media

The spread of information on Facebook and Twitter is much more efficient than on traditional social media platforms. For word-of-mouth (WOM) marketing, social media have become a rich information source for companies or scholars to design models to examine this repository and mine useful insights for marketing strategies. However, social media language is relatively short and contains special words and symbols. Most natural language processing (NLP) methods focus on processing formal sentences and are not well-suited to such short messages. In this study we propose a novel sentiment analysis framework based on deep learning models to extract sentiment from social media. We collect data from which we compile a dataset. After processing these special terms, we seek to establish a semantic dataset for further research. The extracted information will be useful for many future applications. The experimental data have been obtained by crawling several social media platforms.
Trace element contents and in situ sulfur isotope analyses of pyrite in the Baiyun gold deposit, NE China: Implication for the genesis of intrusion-related gold deposits

Trace elements distribution in minerals and formation mechanism of visible gold are poorly understood in intrusion-related gold systems. Here, we choose the Baiyun gold deposit, which is hosted within the Paleoproterozoic metamorphic rocks in the Liaodong Peninsula, NE North China Craton (NCC), as a case study, to address these issues. The ore deposit geology characteristics and paragenesis are described in details; In situ geochemical and sulfur isotopic compositions of pyrite and bulk Pb isotopic ratios of pyrite were presented to define the genesis of pyrite that is closely associated with gold mineralization. Three generations of pyrite were recognized based on petrographic studies: metamorphic pyrite (Py0), hydrothermal pyrite (Py1) coexisting with milky quartz, and hydrothermal pyrite (Py2) coexisting with smoky quartz. In situ laser ablation-inductively coupled plasma-mass spectrometer (LA-ICP-MS) spot analysis suggests that both Py0 and Py1 contain low contents of invisible gold (mean 0.19 ppm and 0.17 ppm, respectively). However, Py1 is low in most other trace element contents and contains Au-Ag-Cu-Pb-Bi micro-inclusions. In contrast, Py2 contains both invisible gold (mean 0.50 ppm) and Au-Ag-Cu-Pb-Bi micro-inclusions. Furthermore, LA-ICP-MS mapping reveals the distributions of Ag, Cu, Pb, Bi and Te closely mimic that of Au, indicating an episode of ore-forming fluid rich in Au, Ag, Cu, Pb, Bi and Te. In situ sulfur isotope analysis of pyrite shows that Py0 has significantly positive δ34S values (+11.74–17.33‰), suggesting that sulfur was derived from sedimentary sources. Py1 and Py2 have negative δ34S values (–8.53––6.19‰, –10.44––6.86‰, respectively), indicating that sulfur was derived from oxidized magmatic fluids. The magmatic fluids are concluded to be released from the early Cretaceous microdiorite magmas, providing most of the sulfur and ore-forming metals. Proofs include the close spatial association of the ore bodies and the dykes, as well the uniform Pb isotopic ratios of Py1 and the early Cretaceous microdiorite. Lines of geological, mineralogical, geochemical and isotopic evidence consistently suggest that the gold mineralization in the Baiyun deposit is intrusion-related, and was resulted from the ~126 Ma magmatic-hydrothermal activities in the region. Visible gold precipitating in pyrite and fractures was from metal-rich magmatic fluids rather than being remobilized from the metamorphic pyrite.