Reviewerly/exHarmony huggingface dataset:
Investigating recurrent neural network memory structures using neuro-evolution

This paper presents a new algorithm, Evolutionary eXploration of Augmenting Memory Models (EXAMM), which is capable of evolving recurrent neural networks (RNNs) using a wide variety of memory structures, such as Δ-RNN, GRU, LSTM, MGU and UGRNN cells. EXAMM evolved RNNs to perform prediction of large-scale, real world time series data from the aviation and power industries. These data sets consist of very long time series (thousands of readings), each with a large number of potentially correlated and dependent parameters. Four different parameters were selected for prediction and EXAMM runs were performed using each memory cell type alone, each cell type and simple neurons, and with all possible memory cell types and simple neurons. Evolved RNN performance was measured using repeated k-fold cross validation, resulting in 2420 EXAMM runs which evolved 4, 840, 000 RNNs in ~24,200 CPU hours on a high performance computing cluster. Generalization of the evolved RNNs was examined statistically, providing findings that can help refine the design of RNN memory cells as well as inform future neuro-evolution algorithms.
Multi-Label Bioinformatics Data Classification With Ensemble Embedded Feature Selection

In bioinformatics, the vast of multi-label type of datasets, including clinical text, gene, and protein data, need to be categorized. Specifically, due to the redundant or irrelevant features in bioinformatics data, the performance of multi-label classifiers will be limited, and therefore, selecting effective features from the feature space is necessary. However, most of the proposed methods, which aimed at dealing with multi-label feature selection problem in the past few years, only adopt a simple and direct strategy that transforms the multi-label feature selection problem into more single-label ones and ignore correlations among different labels. In this paper, a novel algorithm named ensemble embedded feature selection (EEFS) is proposed to handle multi-label bioinformatics data learning problem in a more effective and efficient way. The EEFS does not only explicitly find out the correlations among labels, but it can also adequately utilize the label correlations by multi-label classifiers and evaluation measures. Furthermore, it can reduce the accumulated errors of data itself by employing an ensemble method. The experimental results on five multi-label bioinformatics datasets show that our algorithm achieves significant superiority over the other state-of-the-art algorithms.
HpLapGCN: Hypergraph p-Laplacian graph convolutional networks

Currently, the representation learning of a graph has been proved to be a significant technique to extract graph structured data features. In recent years, many graph representation learning (GRL) algorithms, such as Laplacian Eigenmaps (LE), Node2vec and graph convolutional networks (GCN), have been reported and have achieved great success on node classification tasks. The most representative GCN fuses the feature information and structure information of data, which aims to generalize convolutional neural networks (CNN) to learn data features with arbitrary structure. However, how to exactly express the structure information of data is still an enormous challenge. In this paper, we utilize hypergraph p-Laplacian to preserve the local geometry of samples and then propose an effective variant of GCN, i.e. hypergraph p-Laplacian graph convolutional networks (HpLapGCN). Since hypergraph p-Laplacian is a generalization of the graph Laplacian, HpLapGCN model shows great potential to learn more representative data features. In particular, we simplify and deduce a one-order approximation of spectral hypergraph p-Laplacian convolutions. Thus, we can get a more efficient layer-wise aggregate rule. Extensive experiment results on the Citeseer and Cora datasets prove that our proposed model achieves better performance compare with GCN and p-Laplacian GCN (pLapGCN).
Unsupervised anomaly detection in unmanned aerial vehicles

A real-time anomaly detection solution indicates a continuous stream of operational and labelled data that must satisfy several resources and latency requirements. Traditional solutions to the problem rely heavily on well-defined features and prior supervised knowledge, where most techniques refer to hand-crafted rules derived from known conditions. While successful in controlled situations, these rules assume that good data is available for them to detect anomalies; indicating that these rules will fail to generalise beyond known scenarios. To investigate these issues, current literature is examined for solutions that can be used to detect known and unknown anomalous instances whilst functioning as an out-of-the-box approach for efficient decision-making. The applicability of the isolation forest is discussed for engineering applications using the Aero-Propulsion System Simulation dataset as a benchmark where it is shown to outperform other unsupervised distance-based approaches. Also, the authors have carried out real-time experiments on an unmanned aerial vehicle to highlight further applications of the method. Finally, some conclusions are drawn concerning its simplicity and robustness in handling diagnostic problems.
Interactive Full Image Segmentation by Considering All Regions Jointly

We address interactive full image annotation, where the goal is to accurately segment all object and stuff regions in an image. We propose an interactive, scribble-based annotation framework which operates on the whole image to produce segmentations for all regions. This enables sharing scribble corrections across regions, and allows the annotator to focus on the largest errors made by the machine across the whole image. To realize this, we adapt Mask-RCNN [22] into a fast interactive segmentation framework and introduce an instance-aware loss measured at the pixel-level in the full image canvas, which lets predictions for nearby regions properly compete for space. Finally, we compare to interactive single object segmentation on the COCO panoptic dataset [11, 27, 34]. We demonstrate that our interactive full image segmentation approach leads to a 5% IoU gain, reaching 90% IoU at a budget of four extreme clicks and four corrective scribbles per region.
Recurrent neural network-based semantic variational autoencoder for Sequence-to-sequence learning

Sequence-to-sequence (Seq2seq) models have played an important role in the recent success of various natural language processing methods, such as machine translation, text summarization, and speech recognition. However, current Seq2seq models have trouble preserving global latent information from a long sequence of words. Variational autoencoder (VAE) alleviates this problem by learning a continuous semantic space of the input sentence. However, it does not solve the problem completely. In this paper, we propose a new recurrent neural network (RNN)-based Seq2seq model, RNN semantic variational autoencoder (RNN–SVAE), to better capture the global latent information of a sequence of words. To suitably reflect the meanings of words in a sentence regardless of their position within the sentence, we utilized two approaches: (1) constructing a document information vector based on the attention information between the final state of the encoder and every prior hidden state, and (2) extracting the semantic vector based on the self-attention mechanism. Then, the mean and standard deviation of the continuous semantic space are learned by using this vector to take advantage of the variational method. By using the document information vector and the self-attention mechanism to find the semantic space of the sentence, it becomes possible to better capture the global latent feature of the sentence. Experimental results of three natural language tasks (i.e., language modeling, missing word imputation, paraphrase identification) confirm that the proposed RNN–SVAE yields higher performance than two benchmark models.
Inference in Probabilistic Graphical Models by Graph Neural Networks

A fundamental computation for statistical inference and accurate decision-making is to estimate the marginal probabilities or most probable states of task-relevant variables. Probabilistic graphical models can efficiently represent the structure of such complex data, but performing these inferences is generally difficult. Message-passing algorithms, such as belief propagation, are a natural way to disseminate evidence amongst correlated variables while exploiting the graph structure, but these algorithms can struggle when the conditional dependency graphs contain loops. Here we use Graph Neural Networks (GNNs) to learn a message-passing algorithm that solves these inference tasks. We first show that the architecture of GNNs is well-matched to inference tasks. We then demonstrate the efficacy of this inference approach by training GNNs on a collection of graphical models and showing that they substantially outperform belief propagation on loopy graphs. Our message-passing algorithms generalize out of the training set to larger graphs and graphs with different structure.
Saliency guided deep network for weakly-supervised image segmentation

Weakly-supervised image segmentation is an important task in computer vision. A key problem is how to obtain high-quality objects location from an image-level category. Classification activation mapping is a common method which can be used to generate high-precise object location cues. However, these location cues are generally very sparse and small such that they can not provide adequate information for image segmentation. In this paper, we propose a saliency guided image segmentation network to resolve this problem. We employ a self-attention saliency method to generate subtle saliency maps and render the location cues grow as seeds by seeded region growing method to expand pixel-level labels extent. In the process of seeds growing, we use the saliency values to weight the similarity between pixels to control the growing. Therefore saliency information could help generate discriminative object regions, and the effects of wrong salient pixels can be suppressed efficiently. Experimental results on a common segmentation dataset PASCAL VOC2012 demonstrate the effectiveness of our method.
Learning Multi-Class Segmentations From Single-Class Datasets

Multi-class segmentation has recently achieved significant performance in natural images and videos. This achievement is due primarily to the public availability of large multi-class datasets. However, there are certain domains, such as biomedical images, where obtaining sufficient multi-class annotations is a laborious and often impossible task and only single-class datasets are available. While existing segmentation research in such domains use private multi-class datasets or focus on single-class segmentations, we propose a unified highly efficient framework for robust simultaneous learning of multi-class segmentations by combining single-class datasets and utilizing a novel way of conditioning a convolutional network for the purpose of segmentation. We demonstrate various ways of incorporating the conditional information, perform an extensive evaluation, and show compelling multi-class segmentation performance on biomedical images, which outperforms current state-of-the-art solutions (up to 2.7%). Unlike current solutions, which are meticulously tailored for particular single-class datasets, we utilize datasets from a variety of sources. Furthermore, we show the applicability of our method also to natural images and evaluate it on the Cityscapes dataset. We further discuss other possible applications of our proposed framework.
Adaptive Prediction Models for Data Center Resources Utilization Estimation

Accurate estimation of data center resource utilization is a challenging task due to multi-tenant co-hosted applications having dynamic and time-varying workloads. Accurate estimation of future resources utilization helps in better job scheduling, workload placement, capacity planning, proactive auto-scaling, and load balancing. The inaccurate estimation leads to either under or over-provisioning of data center resources. Most existing estimation methods are based on a single model that often does not appropriately estimate different workload scenarios. To address these problems, we propose a novel method to adaptively and automatically identify the most appropriate model to accurately estimate data center resources utilization. The proposed approach trains a classifier based on statistical features of historical resources usage to decide the appropriate prediction model to use for given resource utilization observations collected during a specific time interval. We evaluated our approach on real datasets and compared the results with multiple baseline methods. The experimental evaluation shows that the proposed approach outperforms the state-of-the-art approaches and delivers 6% to 27% improved resource utilization estimation accuracy compared to baseline methods.
AUC4.5: AUC-Based C4.5 Decision Tree Algorithm for Imbalanced Data Classification

This paper presents a modification of Quinlan’s C4.5 algorithm for imbalanced data classification. While the C4.5 algorithm uses the difference in information entropy to determine the goodness of a split, the proposed method, which is named AUC4.5, examines the difference in the area under the ROC curve (AUC) of a split. It implies that our method attempts to maximize the AUC value of a trained decision tree in order to cope with class imbalance in data. An extensive experimental study was performed on 20 real datasets from the machine learning repository at the University of California at Irvine, Irvine. The proposed AUC4.5 algorithm showed better classification than both the standard and cost-sensitive C4.5 algorithms.
Unsupervised Domain Adaptation for Semantic Segmentation of Urban Scenes

The semantic understanding of urban scenes is one of the key components for an autonomous driving system. Complex deep neural networks for this task require to be trained with a huge amount of labeled data, which is difficult and expensive to acquire. A recently proposed workaround is the usage of synthetic data, however the differences between real world and synthetic scenes limit the performance. We propose an unsupervised domain adaptation strategy to adapt a synthetic supervised training to real world data. The proposed learning strategy exploits three components: a standard supervised learning on synthetic data, an adversarial learning strategy able to exploit both labeled synthetic data and unlabeled real data and finally a self-teaching strategy working on unlabeled data only. The last component is guided by the segmentation confidence, estimated by the fully convolutional discriminator of the adversarial learning module, helping to further reduce the domain shift between synthetic and real data. Furthermore we weighted this loss on the basis of the class frequencies to enhance the performance on less common classes. Experimental results prove the effectiveness of the proposed strategy in adapting a segmentation network trained on synthetic datasets, like GTA5 and SYNTHIA, to a real dataset as Cityscapes.
Practical Image Obfuscation with Provable Privacy

An increasing amount of image data is being generated nowadays, thanks to the popularity of surveillance cameras and camera-equipped personal devices. While such image data can be shared widely to enable research studies, it often contains sensitive information, such as individual identities, location indications, etc. Therefore, the image data must be sanitized before sharing with untrusted parties. Current image privacy-enhancing solutions do not offer provable privacy guarantees, or sacrifice utility to achieve the standard ε-differential privacy. In this study, we propose a novel image obfuscation solution based on metric privacy, a rigorous privacy notion generalized from differential privacy. The key advantage of our solution is that our privacy model allows for higher utility by providing indistinguishability based on image visual similarity, compared to the current method with standard differential privacy. Empirical evaluation with real-world datasets demonstrates that our method provides high utility while providing provable privacy guarantees.
MAPPING MASTER’S STUDENTS’ USE OF EXTERNAL SOURCES IN SOURCE-BASED WRITING IN L1 AND L2

Abstract This article examines how master’s students consult and process sources in source-based writing tasks in L1 and L2. Two hundred eighty master’s students wrote a text in their L1 (Dutch) and L2 (English) at the beginning and end of the academic year. They wrote these texts based on three sources: a report, a web text, and a newspaper article. Their writing processes were registered using the Inputlog keylogging program. This allowed us to determine how much time the students spent reading the sources, when they did so, which sources they consulted most frequently, and how often they switched between the various (types of) sources. The quality of the texts was assessed holistically using pairwise comparisons (D-pac). Confirmative factor analysis showed three components to be relevant to describe source use in L1 and L2 writing: (a) initial reading time, (b) interaction with sources, and (c) the degree of variance in source use throughout the writing process. Individual text quality remained stable in L1 and L2 throughout the academic year. Structural equation modeling showed that the approach in source use, especially source interaction, is correlated with text quality, but in L1 only.
Image Superresolution via Dense Discriminative Network

Deep convolutional neural networks have recently made a considerable achievement in the single-image superresolution (SISR) problem. Most CNN architectures for SISR incorporate long or short connections to integrate features, and treat them equally. However, they neglect the discrimination of features, and consequently, achieving relatively poor performance. To address this problem, in this article, we propose a dense discriminative network that is composed of several aggregation modules (AM). Specifically, the AM merges extraction and integration nodes in a tree structure, which can aggregate features progressively in an efficient way. In particular, we compress and rescale the densely connected information in the aggregation node by modeling the interaction between channels, which shares the same insight with the attention mechanism for improving the discriminative ability of network. Extensive experiments conducted on several publicly available datasets have demonstrated the superiority of our model over state-of-the-art in objective metrics and visual impressions.
Multi-Scale Body-Part Mask Guided Attention for Person Re-Identification

Person re-identification becomes a more and more important task due to its wide applications. In practice, person re-identification still remains challenging due to the variation of person pose, different lighting, occlusion, misalignment, background clutter, etc. In this paper, we propose a multi-scale body-part mask guided attention network (MMGA), which jointly learns whole-body and part-body attention to help extract global and local features simultaneously. In MMGA, body-part masks are used to guide the training of corresponding attention. Experiments show that our proposed method can reduce the negative influence of variation of person pose, misalignment and background clutter. Our method achieves rank-1/mAP of 95.0%/87.2% on the Market1501 dataset, 89.5%/78.1% on the DukeMTMC-reID dataset, outperforming current state-of-the-art methods.
Color Channel Transfer for Image Dehazing

In this letter we introduce a simple but effective concept, Color Channel Transfer (CCT), that is able to substantially improve the performance of various dehazing techniques. CCT is motivated by a key observation: in scattering media the information from at least one color channel presents high attenuation. To compensate for the loss of information in one color channel, CCT employs a color-transfer strategy and operates in a color opponent space that helps to compensate automatically the chromatic loss. The reference is computed by combining the details and saliency of the initial image with uniform gray image that assures a balanced chromatic distribution. The extensive qualitative and quantitative experiments demonstrate the utility of CCT as a preprocessing step for various dehazing problems such as day-time dehazing, night-time dehazing, and underwater image dehazing.
Facing the Truth

The evolution of variant-rich systems is a challenging task. To support developers, the research community has proposed a range of different techniques over the last decades. However, many techniques have not been adopted in practice so far. To advance such techniques and to support their adoption, it is crucial to evaluate them against realistic baselines, ideally in the form of generally accessible benchmarks. To this end, we need to improve our empirical understanding of typical evolution scenarios for variant-rich systems and their relevance for benchmarking. In this paper, we establish eleven evolution scenarios in which benchmarks would be beneficial. Our scenarios cover typical lifecycles of variant-rich system, ranging from clone & own to adopting and evolving a configurable product-line platform. For each scenario, we formulate benchmarking requirements and assess its clarity and relevance via a survey with experts in variant-rich systems and software evolution. We also surveyed the existing benchmarking landscape, identifying synergies and gaps. We observed that most scenarios, despite being perceived as important by experts, are only partially or not at all supported by existing benchmarks-a call to arms for building community benchmarks upon our requirements. We hope that our work raises awareness for benchmarking as a means to advance techniques for evolving variant-rich systems, and that it will lead to a benchmarking initiative in our community.
Sentiment Analysis of Product Reviews in Russian using Convolutional Neural Networks

Nowadays, product reviews on e-commerce sites tend to be a valuable resource in terms of evaluation of customers' behavior, their preferences, and needs. This paper provides an approach for sentiment analysis of product reviews in Russian using convolutional neural networks. We use Word2Vec pre-trained vectors as inputs for neural networks. This approach utilizes no hand-crafted features or sentiment lexicons. The training dataset was collected from reviews on top-ranked goods from the major e-commerce site in Russia, where the user-ranked scores were used as class labels. The system demonstrated the F-measure score up to 75.45% in a three-class classification. The collected training dataset and word embeddings are available to the research community.