Reviewerly/exHarmony huggingface dataset:
Detect the Unexpected: Novelty Detection in Large Astrophysical Surveys using Fisher Vectors

Finding novelties in an untagged high dimensional dataset poses an open question. In this work, we present an innovative method for detecting such novelties using Fisher Vectors. Our dataset distribution is modeled using a Gaussian Mixture Model. An anomaly score that stems from the theory of Fisher Vector is computed for each of the samples. We compute the anomaly score on the SDSS galaxies spectra dataset and present the different types of novelties found. We compare our findings with other outlier detection algorithms from the literature, and demonstrate the ability of our method to distinguish between samples taken from intersecting probability distributions.
RPR-BP: A Deep Reinforcement Learning Method for Automatic Hyperparameter Optimization

We introduce a new deep reinforcement learning architecture - RPR-BP to optimize hyperparameter for any machine learning model on a given data set. In this method, an agent constructed by a Long Short-Term Memory Network aims at maximizing the expected accuracy of a machine learning model on a validation set. At each iteration, it selects a set of hyperparameters and uses the accuracy of the model on the validation set as the reward signal to update its internal parameters. After multiple iterations, the agent learns how to improve its decisions. However, the computation of the reward requires significant time and leads to low sample efficiency. To speed up training, we employ a neural network to predict the reward. The training process for the agent and the prediction network is divided into three phases: Real-Predictive-Real (RPR). First, the agent and the prediction network are trained by the real experience; then, the agent is trained by the reward generated from the prediction network; finally, the agent is trained again by the real experience. In this way, we can speed up training and make the agent achieve a high accuracy. Besides, to reduce the variance, we propose a Bootstrap Pool (BP) to guide the exploration in the search space. The experiment was carried out by optimizing hyperparameters of two widely used machine learning models: Random Forest and XGBoost. Experimental results show that the proposed method outperforms random search, Bayesian optimization and Tree-structured Parzen Estimator in terms of accuracy, time efficiency and stability.
Distributed Differentially Private Matrix Factorization Based on ADMM

Matrix factorization (MF) is an essential technique to implement intelligent recommender systems widely applied in industry. Privacy and efficiency are two essential issues concerning MF. We leverage two techniques, differential privacy (DP) and distributed computing, to address the two concerns, respectively. (1) Differentially private MF is still challenging since conventional strategies lead to significant error accumulation; we adopt the objective function perturbation technique to tackle such a challenge. (2) We adopt the alternating direction method of multipliers (ADMM) framework to parallelize the factorization to improve performance; to implement this parallelization, we adopt the effective matrix split method and introduce a novel integration strategy for distributed DP based on the post-processing theorem. We identify our work as distributed differentially private MF based on ADMM. The novelty of the work rests in that it is the first to successfully integrate the two innovative techniques to address both privacy and efficiency for MF. We establish the mathematical model and conduct experiments to validate the soundness of our idea. The experimental results based on industrial datasets show that the distributed differentially private MF algorithm provides scalable speedup performance within a limited precision loss while preserving user privacy.
On Arabic Stop-Words: A Comprehensive List and a Dedicated Morphological Analyzer

Stop-words detection is a key preprocessing step and an important component for many Natural Language Processing applications. For Arabic language, stop-words detection is a complex task due to Arabic morphology richness and to the nonexistence of a commonly accepted list. In this paper, we compile a new comprehensive Arabic stop-words list along a stop-words analyzer that combines that list with a machine-learning-based approach to get the most probable stop-word. The first step in our approach provides a context-free analysis and the most appropriate stop-word according to the sentence context is detected in the second step using the Hidden Markov Model. The developed analyzer evaluation yields to over than 97% of accuracy. This achievement outperforms the state of the art analyzers.
On Class Imbalance and Background Filtering in Visual Relationship Detection

In this paper we investigate the problems of class imbalance and irrelevant relationships in Visual Relationship Detection (VRD). State-of-the-art deep VRD models still struggle to predict uncommon classes, limiting their applicability. Moreover, many methods are incapable of properly filtering out background relationships while predicting relevant ones. Although these problems are very apparent, they have both been overlooked so far. We analyse why this is the case and propose modifications to both model and training to alleviate the aforementioned issues, as well as suggesting new measures to complement existing ones and give a more holistic picture of the efficacy of a model.
Hardware implementation of a shape recognition algorithm based on invariant moments

The present work shows the description of a simple fast shape detection algorithm and its implementation in hardware in a FPGA system. The detection algorithm is based on the concepts of Hu's moments which are invariant to similarity transformations. The recognition algorithm is implemented by using a non-local means filter. The algorithm is implemented on a FPGA system by using a hardware description language. We present the different design stages of the algorithm implementation which is based on the finite state machine technique. This algorithm is able to recognize a target shape over a test image. Furthermore, this work, describes the advantages of the implementation in hardware, such as speed and parallelism in signal processing. Finally, we show some results of the implementation of this algorithm.
Synaptic Learning of Long-Term Cognitive Networks with Inputs

In contrast with the extense variety of machine learning algorithms, to fully automate the reasoning process, only a few can take advantage of the expert knowledge. Fuzzy Cognitive Maps (FCMs) are neural networks that can naturally integrate this kind of knowledge in the inference process. Nevertheless, FCMs have serious drawbacks difficult to overcome from the absence of an intrinsically learning algorithm or limited prediction horizon of the activation space of the neurons. Recently, some variants of the FCMs like Short-Term Cognitive Networks (STCN) and Long Term Cognitive Networks (LTCN) have been proposed to solve these problems. In this paper, we propose a new neural network model as a variant of LTCNs called Long-Term Cognitive Networks with Inputs (LTCNIs). A new kind of input neuron which is not present in the traditional FCMs approach or the derived algorithms STCNs and LTCNs is introduced, in order to model inputs like energy or mass in physical systems. The performance of the method is discussed through the modeling of a passive circuit problem. As a second contribution, a new flexible reasoning strategy, which preserves the expert knowledge through synaptic learning is presented. A synaptic learning based on a gradient descent method is implemented limited by a set of restrictions that preserves the model semantics.
Assessing the Learning Difficulty of Text-Based Learning Materials

To have an effective e-learning system, it is important to recommend the appropriate learning material on the basis of suitable difficulty level to the learner for comprehension. For this, assessing the difficulty level of the learning material in terms of reading and comprehension is necessary, which is not trivial. This paper proposes a novel methodology to assess the difficulty level of text-based learning material, in general. The three parameters that are considered as the evaluator of the learning difficulty of a text are: (a) readability of the text, (b) uncommon/rare words used in the text, and (c) degree of information embedded in the text. The approach considers the psycholinguist features of the given text. The experiment is conducted on learning materials for Java. The estimated difficulty closely matches with that of human assessment of the difficulty of the same text.
High-Elasticity Virtual Cluster Placement in Multi-Tenant Cloud Data Centers

In recent years, Data Center Network (DCN) has become a promising and efficient data processing infrastructure for cloud computing. One important mission of DCN is to serve the ever-growing demand for computation, storage, and networking for multiple tenants in cloud computing. This paper uses the notion of elasticity to measure the potential growth of multiple tenants in terms of both computation and communication resources. Our objective is to maximize the elasticity for DCNs. We consider the multiple virtual cluster placement problem with the hose model under the computation and communication constraints. We first formulate this problem as an Integer Linear Programming (ILP) problem. Unfortunately, the formulated ILP problem cannot be solved by the simplex or eclipse methods because of a large number of variables and constraints. Therefore, we propose an efficient scheme based on the Dynamic Programming (DP) and analyze its optimality and complexity. Furthermore, we propose a heuristic algorithm for placement that maximizes the elasticity and guarantees the bandwidth demand as well as lower complexity. Extensive evaluations demonstrate that our schemes outperform existing state-of-the-art methods in terms of both elasticity and efficiency.
The Cramming, Softening and Integrating Learning Algorithm with Parametric ReLU Activation Function for Binary Input/Output Problems

Rare Artificial Neural Networks studies address simultaneously the challenges of (1) systematically adjusting the amount of used hidden layer nodes within the learning process, (2) adopting Parametric ReLU activation function instead of tanh function for fast learning, and (3) guaranteeing learning all training data. This study will address these challenges through deriving the CSI (Cramming, Softening and Integrating) learning algorithm for the single-hidden layer feed-forward neural networks with the binary input/output and making the technical justification. To further verify the proposed learning algorithm, this study conducts an empirical experiment using SPECT heart diagnosis data set from UCI Machine Learning repository. The learning algorithm is implemented via the advanced TensorFlow and GPU.
Efficient on-line learning with diagonal approximation of loss function Hessian

The subject of this paper is stochastic optimization as a tool for on-line learning. New ingredients are introduced to Nesterov's Accelerated Gradient that increase efficiency of this algorithm and determine its parameters that are otherwise tuned manually: step-size and momentum decay factor. In this order a diagonal approximation of the Hessian of the loss function is estimated. In the experimental study the approach is applied to various types of neural networks, deep ones among others.
Speckle Noise Reduction in SAR Images Using Fuzzy Inference System

In recent years, image processing has played a vital role in major research areas. In this article, a new approach using a fuzzy inference system is proposed for speckle reduction in SAR images. In general, SAR images are predominantly used to monitor coastal regions to detect oil spills, ship wake, sea shores and climate changes. In this article, a gamma distribution model is used in a fuzzy inference system to remove speckle noise from SAR images. The performance of the proposed model is tested using fuzzy inference systems, such as mamdani and sugeno. The experimental results proved the efficiency of the proposed system through objective metrics.
MIPCNF: Multi-iterative partitioning class noise filter

Attribute and class noises are the two important sources of Corruptions (noise) contained in real-world datasets which may deteriorate data interpretation and accuracy. Class noise has potentially serious negative impacts compared to attribute noise, however, the existing major class noise detectio n methods are not able to address this problem efficiently. To overcome issues related to detection and the elimination of class noise, we suggest a new noise filtering approach able to identify and remove class noise, called Multi-Iterative Partitioning Class Noise Filter (MIPCNF). Since there is no single filter that consistently outperforms its counterparts in all database types and in different levels of noise, our approach relies on an algorithm in which several rounds of class noise detection are performed on different partitions of the data using several classifiers. Therefore, we use different filtering strategies: iterative noise filter, partitioning filter and ensemble-based filter. The experimental results, on 14 real-world datasets, and statistical analysis, show that our method is not only overcoming the higher noise but also over-performing latest class noise detection and elimination strategies in different levels of noise.
Deeper Monocular Depth Prediction via Long and Short Skip Connection

This paper presents a fully convolutional neural network to tackle the mapping between single view RGB images and depth maps. To regress the depth maps from monocular images, we leverage deep short skip connections in residual learning for extracting features rather than using hand-crafted features. We further propose long skip connections in up-sampling stage to reuse the feature maps which is proved to enhance the result experimentally. To show the impact of loss functions in monocular depth map predictions, we train our model with kind of loss functions and compare the results qualitatively and quantitatively. The proposed model outperforms all current state-of-the-art results with less training data as well as less than half of training epochs in two standard benchmark data sets without any post-processing procedures or other refinement steps.
Discriminative Regularization with Conditional Generative Adversarial Nets for Semi-Supervised Learning

Existing generative adversarial networks (GANs) with manifold regularization for semi-supervised learning (SSL) have shown promising performance in image generation and semi-supervised learning (SSL), which penalize the smoothness of classifier over data manifold based on the smoothness assumption. However, the smoothness assumption is valid for data points in high density region while not hold for data points in low density region, thus they tend to misclassify boundary instances in low density region. In this paper, we propose a novel discriminative regularization method for semi-supervised learning with conditional generative adversarial nets (CGANs). In our method, the discriminative information from class conditional data distribution captured by CGANs is utilized to improve the discrimination of classifier. Different from regular manifold regularization, the discriminative regularization encourages the classifier invariance to local perturbations on the sub-manifold of each cluster, and distinct classification outputs for data points in different clusters. Moreover, our method can be easily implemented via the stochastic approximation without constructing the Laplacian graph or computing the Jacobian of classifier explicitly. Experimental results on benchmark datasets show that our method can achieve competitive performance against previous advanced methods.
From Content Text Encoding Perspective: A Hybrid Deep Matrix Factorization Approach for Recommender System

Recommender systems usually make personalized recommendation for users by analyzing interaction ratings between users and items. In plenty of application domains, some implicit feedback, content text, and other additional auxiliary information are widely used for improving the performance of recommendation. Memory-based collaborative filtering approaches that are widely used like matrix factorization (MF) predicts a personalized ranking for an individual user by leveraging latent factor models to handle common problems in recommender systems. However, most of previous MF methods adopt only explicit ratings to make personalized recommendation, ignoring the importance of implicit feedback on both users and items in-formation. Latent factor model can be easily extended with content text by mapping text information into factors, therefore we propose an approach by using neural networks to acquire the latent factors. Meanwhile, recurrent neural networks (RNNs) are applied to convert the textual data to an auxiliary factor feature to promote the representation of items. The experimental results prove that our model is effective on two benchmark datasets, outperforming some state-of-art approaches.
Gaussian Model of Electrooculogram Signals

Modeling of biomedical signals is crucial and vital for compression, transmission, understanding, feature extraction, and prediction. Researchers devoted their efforts on modeling cardiac related biosignals such as electrocardiography (ECG), photoplethysmography (PPG), and arterial blood pressure(APP) without concerning other types of non-cardiac related biosignals such as electrooculogram (EOG). This paper focus on the modeling of EOG signal using a novel method that exploits a linear combination of seven gaussian basis functions. The proposed method succeeded on modeling of fifteen different eye movements directions combined from right, left, and center with average root mean squared (RMS) error of 1%.
Stochastic Resonance Enables BPP/log* Complexity and Universal Approximation in Analog Recurrent Neural Networks

Stochastic resonance (SR) is a natural process that without limit increases the precision of signal measurements in biological and physical sciences. Most artificial neural networks (NNs) are implemented on digital computers of fixed-precision. A NN accessing universal approximation and a computational complexity class more powerful that of a Turing machine needs analog signals utilizing SR’s limitless precision increase. This paper links an analog recurrent (AR) NN theorem, SR, BPP/log* (a physically realizable, super-Turing computation class), and universal approximation so NNs following them can be made computationally more powerful. An optical neural network mimicking chaos indicates super-Turing computation has been achieved. Additional tests are needed which can verify superTuring computation, show its superiority, and demonstrate its practical benefits. Truly powerful cognitively inspired computation needs to access the combination of ARNNs, SR, super-Turing mathematical complexity, and universal approximation.
Deep Capsule Network based Automatic Batch Code Identification Pipeline for a Real-life Industrial Application

Automatic recognition of text, such as a batch code printed on a box placed on a moving conveyor belt, is still a challenging problem. This paper proposes an end-to-end character recognition technique while addressing the major challenges encountered in a real environment, such as motion blur in the acquired images, slanted or oriented characters, creased batch codes due to wear and tear of boxes, variations in label formats, and variations in printing styles. The major contribution of this work lies in development of three sequential modules: text localization using Connectionist Text Proposal Network(CTPN), character detection and character recognition using a modified version of the capsule network (CapsNet). In contrast to CapsNet, where only a standard single convolution is used, the proposed method uses a series of feature blocks, making it a deep CapsNet which is later proven to generate more comprehensive and better separable feature vectors over its counterpart. The feature generation module is further enhanced by setting a smaller kernel size than CapsNet. The proposed system is validated on a real-world box / packet dataset generated in a retail manufacturing industry. The proposed recognition network architecture is also validated on a standard public dataset (ICDAR 2013). The comparative results are presented with statistical analysis in the experimental results section.
Learning Diachronic Word Embeddings with Iterative Stable Information Alignment

Diachronic word embedding aims to reveal the semantic evolution of words over time. Previous works learned word embeddings in different time periods first, and then aligned all the word embeddings into a same vector space. Different from previous works, we iteratively identify stable words, meanings of which remain acceptably stable even in different time periods, as anchors to ensure the performances of both embedding learning and alignment. To learn word embeddings in the same vector space, two different cross-time constraints are used during training. Initially, we identify the most obvious stable words with an unconstrained model, and then use hard constraint to restrain them in related stable time periods. In the iterative process, we identify new stable words from previously trained model and use soft constraint on them to fine-tune the model. We use COHA dataset ( https://corpus.byu.edu/coha/ ) [14], which consists of texts from 1810s to 2000s. Both qualitative and quantitative evaluations show our model can capture meanings in each single time period accurately and model the changes of word meaning. Experimental results indicate that our proposed model outperforms all baseline methods in terms of diachronic text evaluation.
BitFlow-Net: Toward Fully Binarized Convolutional Neural Networks

Binarization can greatly compress and accelerate deep convolutional neural networks (CNNs) for real-time industrial applications. However, existing binarized CNNs (BCNNs) rely on scaling factor (SF) and batch normalization (BatchNorm) that still involve resource-consuming floating-point multiplication operations. Addressing the limitation, an improved BCNN named BitFlow-Net is proposed, which replaces floating-point operations with integer addition in middle layers. First, it is derived that the SF is only effective in back-propagation process, whereas it is counteracted by BatchNorm in inference process. Then, in model running phase, the SF and BatchNorm are fused into an integer addition, named BatchShift. Consequently, the data flow in middle layers is fully binarized during modeling running phase. To verify its potential in industrial applications with multiclass and binary classification tasks, the BitFlow-Net is built based on AlexNet and verified on two large image datasets, i.e., ImageNet and 11K Hands. Experimental results show that the BitFlow-Net can remove all floating-point operations in middle layers of BCNNs and greatly reduce the memory for both cases without affecting the accuracy. Particularly, the BitFlow-Net can achieve the accuracy comparable to that of the full-precision AlexNet network in the binary classification task.