Reviewerly/exHarmony huggingface dataset:
Zero-shot Fine-grained Classification by Deep Feature Learning with Semantics

Fine-grained image classification, which aims to distinguish images with subtle distinctions, is a challenging task for two main reasons: lack of sufficient training data for every class and difficulty in learning discriminative features for representation. In this paper, to address the two issues, we propose a two-phase framework for recognizing images from unseen fine-grained classes, i.e., zero-shot fine-grained classification. In the first feature learning phase, we finetune deep convolutional neural networks using hierarchical semantic structure among fine-grained classes to extract discriminative deep visual features. Meanwhile, a domain adaptation structure is induced into deep convolutional neural networks to avoid domain shift from training data to test data. In the second label inference phase, a semantic directed graph is constructed over attributes of fine-grained classes. Based on this graph, we develop a label propagation algorithm to infer the labels of images in the unseen classes. Experimental results on two benchmark datasets demonstrate that our model outperforms the state-of-the-art zero-shot learning models. In addition, the features obtained by our feature learning model also yield significant gains when they are used by other zero-shot learning models, which shows the flexility of our model in zero-shot fine-grained classification.
Camouflage people detection via strong semantic dilation network

Camouflage greatly reduces the probability of the target being discovered, detecting the camouflage object from background is very hard for former object detection methods because they are designed to detect the object in an ideal environment. It brings difficulties to subsequent applications such as battlefield information acquisition, precision guided weapons and object tracking. To deal with the problem, in this paper, a new complete camouflage people detection dataset is firstly constructed, then we propose strong semantic dilation network (SSDN), which is specially designed to detect camouflage people in an end-to-end architecture. SSDN makes full use of semantic information in CNN and dilated convolutions are also added to enlarge the receptive field to find camouflage people. Experiments demonstrate that SSDN perform well in camouflage people detection dataset compared with other object detection methods.
Attentional multilabel learning over graphs: a message passing approach

We address a largely open problem of multilabel classification over graphs. Unlike traditional vector input, a graph has rich variable-size substructures which are related to the labels in some ways. We believe that uncovering these relations might hold the key to classification performance and explainability. We introduce Graph Attention model for Multi-Label learning ( $$\text {GAML}$$ ), a novel graph neural network that can handle this problem effectively. $$\text {GAML}$$ regards labels as auxiliary nodes and models them in conjunction with the input graph. By applying the neural message passing algorithm and attention mechanism to both the label nodes and the input nodes iteratively, $$\text {GAML}$$ can capture the relations between the labels and the input subgraphs at various resolution scales. Moreover, our model can take advantage of explicit label dependencies. It also scales linearly with the number of labels and graph size thanks to our proposed hierarchical attention. We evaluate $$\text {GAML}$$ on an extensive set of experiments with both graph-structured inputs and classical unstructured inputs. The results show that $$\text {GAML}$$ significantly outperforms other competing methods. Importantly, $$\text {GAML}$$ enables intuitive visualizations for better understanding of the label-substructure relations and explanation of the model behaviors.
Analysis and Identification of Dermatological Diseases Using Gaussian Mixture Modeling

Skin diseases are common and are mainly caused by virus, bacteria, fungus, or chemical disturbances.Timely analysis and identification are of utmost importance in order to control the further spread of these diseases.Control of these diseases is even more difficult in rural and resource-poor environments due to a lack of expertise in primary health centers.Hence, there is a need for providing self-assisting and innovative measures for the appropriate diagnosis of skin diseases.Use of mobile applications may provide inexpensive, simple, and efficient solutions for early diagnosis and treatment.This paper investigates the application of the Gaussian mixture model (GMM) based on the analysis and classification of skin diseases from their visual images using a Mahalanobis distance measure.The GMM has been preferred over the convolution neural network (CNN) because of limited resources available within the mobile device.Gray-level co-occurrence matrix (GLCM) parameters contrast, correlation, energy, and homogeneity derived from skin images have been used as the input data for the GMM.The analysis of the results showed that the proposed method is able to predict the classification of skin diseases with satisfactory efficiency.It was also observed that different diseases occupy distinct spatial positions in multidimensional space clustered using the Mahalanobis distance measure.
Automated Change Detection and Reactive Clustering in Multivariate Streaming Data

Many automated systems need the capability of automatic change detection without the given detection threshold. This paper presents an automated change detection algorithm in streaming multivariate data. Two overlapping windows are used to quantify the changes. While a window is used as the reference window from which the clustering is created, the other called the current window captures the newly incoming data points. A newly incoming data point can be considered a change point if it is not a member of any cluster. As our clustering-based change detector does not require detection threshold, it is an automated detector. Based on this change detector, we propose a reactive clustering algorithm for streaming data. Our empirical results show that our clustering-based change detector works well with multivariate streaming data. The detection accuracy depends on the number of clusters in the reference window, and on the window width.
Weighted contrastive divergence

Learning algorithms for energy based Boltzmann architectures that rely on gradient descent are in general computationally prohibitive, typically due to the exponential number of terms involved in computing the partition function. In this way one has to resort to approximation schemes for the evaluation of the gradient. This is the case of Restricted Boltzmann Machines (RBM) and its learning algorithm Contrastive Divergence (CD). It is well-known that CD has a number of shortcomings, and its approximation to the gradient has several drawbacks. Overcoming these defects has been the basis of much research and new algorithms have been devised, such as persistent CD. In this manuscript we propose a new algorithm that we call Weighted CD (WCD), built from small modifications of the negative phase in standard CD. However small these modifications may be, experimental work reported in this paper suggests that WCD provides a significant improvement over standard CD and persistent CD at a small additional computational cost.
Learning from Multiview Correlations in Open-domain Videos

An increasing number of datasets contain multiple views, such as video, sound and automatic captions. A basic challenge in representation learning is how to leverage multiple views to learn better representations. This is further complicated by the existence of a latent alignment between views, such as between speech and its transcription, and by the multitude of choices for the learning objective. We explore an advanced, correlation-based representation learning method on a 4-way parallel, multimodal dataset, and assess the quality of the learned representations on retrieval-based tasks. We show that the proposed approach produces rich representations that capture most of the information shared across views. Our best models for speech and textual modalities achieve retrieval rates from 70.7% to 96.9% on open-domain, user-generated instructional videos. This shows it is possible to learn reliable representations across disparate, unaligned and noisy modalities, and encourages using the proposed approach on larger datasets.
Void Filling of Digital Elevation Models With Deep Generative Models

In recent years, advances in machine learning algorithms, cheap computational resources, and the availability of big data have spurred the deep learning revolution in various application domains. In particular, supervised learning techniques in image analysis have led to superhuman performance in various tasks, such as classification, localization, and segmentation, while unsupervised learning techniques based on increasingly advanced generative models have been applied to generate high-resolution synthetic images indistinguishable from real images. In this paper we consider a state-of-the-art machine learning model for image inpainting, namely a Wasserstein Generative Adversarial Network based on a fully convolutional architecture with a contextual attention mechanism. We show that this model can successfully be transferred to the setting of digital elevation models (DEMs) for the purpose of generating semantically plausible data for filling voids. Training, testing and experimentation is done on GeoTIFF data from various regions in Norway, made openly available by the Norwegian Mapping Authority.
An asymptotic analysis of distributed nonparametric methods

We investigate and compare the fundamental performance of several distributed learning methods that have been proposed recently. We do this in the context of a distributed version of the classical signal-in-Gaussian-white-noise model, which serves as a benchmark model for studying performance in this setting. The results show how the design and tuning of a distributed method can have great impact on convergence rates and validity of uncertainty quantification. Moreover, we highlight the difficulty of designing nonparametric distributed procedures that automatically adapt to smoothness.
Leveraging Historical Associations between Requirements and Source Code to Identify Impacted Classes

As new requirements are introduced and implemented in a software system, developers must identify the set of source code classes which need to be changed. Therefore, past effort has focused on predicting the set of classes impacted by a requirement. In this paper, we introduce and evaluate a new type of information based on the intuition that the set of requirements which are associated with historical changes to a specific class are likely to exhibit semantic similarity to new requirements which impact that class. This new Requirements to Requirements Set (R2RS) family of metrics captures the semantic similarity between a new requirement and the set of existing requirements previously associated with a class. The aim of this paper is to present and evaluate the usefulness of R2RS metrics in predicting the set of classes impacted by a requirement. We consider 18 different R2RS metrics by combining six natural language processing techniques to measure the semantic similarity among texts (e.g., VSM) and three distribution scores to compute overall similarity (e.g., average among similarity scores). We evaluate if R2RS is useful for predicting impacted classes in combination and against four other families of metrics that are based upon temporal locality of changes, direct similarity to code, complexity metrics, and code smells. Our evaluation features five classifiers and 78 releases belonging to four large open-source projects, which result in over 700,000 candidate impacted classes. Experimental results show that leveraging R2RS information increases the accuracy of predicting impacted classes practically by an average of more than 60 percent across the various classifiers and projects.
A Bias Aware News Recommendation System

In this era of fake news and political polarization, it is desirable to have a system to enable users to access balanced news content. Current solutions focus on top down, server based approaches to decide whether a news article is fake or biased, and display only trusted news to the end users. In this paper, we follow a different approach to help the users make informed choices about which news they want to read, making users aware in real time of the bias in news articles they were browsing and recommending news articles from other sources on the same topic with different levels of bias. We use a recent Pew research report to collect news sources that readers with varying political inclinations prefer to read. We then scrape news articles on a variety of topics from these varied news sources. After this, we perform clustering to find similar topics of the articles, as well as calculate a bias score for each article. For a news article the user is currently reading, we display the bias score and also display other articles on the same topic, out of the previously collected articles, from different news sources. This we present to the user. This approach, we hope, would make it possible for users to access more balanced articles on given news topics. We present the implementation details of the system along with some preliminary results on news articles.
Efficient mixture model for clustering of sparse high dimensional binary data

Clustering is one of the fundamental tools for preliminary analysis of data. While most of the clustering methods are designed for continuous data, sparse high-dimensional binary representations became very popular in various domains such as text mining or cheminformatics. The application of classical clustering tools to this type of data usually proves to be very inefficient, both in terms of computational complexity as well as in terms of the utility of the results. In this paper we propose a mixture model, SparseMix, for clustering of sparse high dimensional binary data, which connects model-based with centroid-based clustering. Every group is described by a representative and a probability distribution modeling dispersion from this representative. In contrast to classical mixture models based on the EM algorithm, SparseMix: is specially designed for the processing of sparse data; can be efficiently realized by an on-line Hartigan optimization algorithm; describes every cluster by the most representative vector. We have performed extensive experimental studies on various types of data, which confirmed that SparseMix builds partitions with a higher compatibility with reference grouping than related methods. Moreover, constructed representatives often better reveal the internal structure of data.
Content-Aware Personalised Rate Adaptation for Adaptive Streaming via Deep Video Analysis

Adaptive bitrate (ABR) streaming is the de facto solution for achieving smooth viewing experiences under unstable network conditions. However, most of the existing rate adaptation approaches for ABR are content-agnostic, without considering the semantic information of the video content. Nevertheless, semantic information largely determines the informativeness and interestingness of the video content, and consequently affects the QoE for video streaming. One common case is that the user may expect higher quality for the parts of video content that are more interesting or informative so as to reduce overall subjective quality loss. This creates two main challenges for such a problem: First, how to determine which parts of the video content are more interesting? Second, how to allocate bitrate budgets for different parts of the video content with different significances? To address these challenges, we propose a Content-of-Interest (CoI) based rate adaptation scheme for ABR. We first design a deep learning approach for recognizing the interestingness of the video content, and then design a Deep Q-Network (DQN) approach for rate adaptation by incorporating video interestingness information. The experimental results show that our method can recognize video interestingness precisely, and the bitrate allocation for ABR can be aligned with the interestingness of video content while not compromising the performances on objective QoE metrics.
Model Agnostic Saliency For Weakly Supervised Lesion Detection From Breast DCE-MRI

There is a heated debate on how to interpret the decisions provided by deep learning models (DLM), where the main approaches rely on the visualization of salient regions to interpret the DLM classification process. However, these approaches generally fail to satisfy three conditions for the problem of lesion detection from medical images: 1) for images with lesions, all salient regions should represent lesions, 2) for images containing no lesions, no salient region should be produced, and 3) lesions are generally small with relatively smooth borders. We propose a new model-agnostic paradigm to interpret DLM classification decisions supported by a novel definition of saliency that incorporates the conditions above. Our model-agnostic 1-class saliency detector (MASD) is tested on weakly supervised breast lesion detection from DCE-MRI, achieving state-of-the-art detection accuracy when compared to current visualization methods.
Spatiotemporal CNNs for Pornography Detection in Videos

With the increasing use of social networks and mobile devices, the number of videos posted on the Internet is growing exponentially. Among the inappropriate contents published on the Internet, pornography is one of the most worrying as it can be accessed by teens and children. Two spatiotemporal CNNs, VGG-C3D CNN and ResNet R $$(2+1)$$ D CNN, were assessed for pornography detection in videos in the present study. Experimental results using the Pornography-800 dataset showed that these spatiotemporal CNNs performed better than some state-of-the-art methods based on bag of visual words and are competitive with other CNN-based approaches, reaching accuracy of $$95.1\%$$ .
A hierarchical prototype-based approach for classification

In this paper, a novel hierarchical prototype-based approach for classification is proposed. This approach is able to perceive the data space and derive the multimodal distributions from streaming data at different levels of granularity in an online manner, based on which it further identifies meaningful prototypes to self-organize and self-evolve its hierarchical structure for classification. Thanks to the prototype-based nature, the system structure of the proposed classifier is highly transparent, and its learning process is of "one pass" type and computationally lean. Its decision-making process follows the "nearest prototype" principle and is fully explainable. The proposed approach is capable of presenting the learned knowledge from data in an easy-to-interpret prototype-based hierarchical form to users, and is an attractive tool for solving large-scale, complex real-world problems. Numerical examples based on various benchmark problems justify the validity and effectiveness of the proposed concept and general principles.
New Extension Constructions of Optimal Frequency-Hopping Sequence Sets

In this paper, a general framework of constructing the optimal frequency-hopping sequence (FHS) sets is presented based on the designated direct product. Under this framework, we obtain infinitely many new optimal FHS sets by combining some one-coincide (OC) sequence sets that are newly constructed in this paper with some known optimal FHS sets. Our constructions are also based on extension method. However, our constructions give new and flexible parameters due to the free choice of OC sequence set. As a result, our constructions allow a great flexibility of choosing parameters of FHS sets for a given frequency-hopping spread spectrum system.
Semantic Interoperability Methods for Smart Service Systems: A Survey

Functional and nonfunctional characteristics of software systems are defined by their architecture. Therefore, research streams such as Internet-of-Things (IoT) or component-based software engineering provide researchers and practitioners with construction guidelines for selected architectural characteristics. Current systems can be categorized in delivering services to the user and being engineered in a smart way. For example, services being provided by IoT-Systems must fulfill users’ goals in a highly dynamic and ad-hoc way. Consequently, this survey aims at answering various research questions regarding the methodical composition of system components and services. Furthermore, new research opportunities are sketched that should be tackled to make the scientific progress available to practitioners. Based on a systematic literature review from a software architecture point of view, in this paper we identify 75 primary studies for domain-specific IoT component composition approaches and architectures. Initial results show that current integration approaches mainly focus on performance evaluation of their integration solutions, which may be too narrow for fulfilling user goals by utilizing of IoT architectures.
An Examination of Deep-Learning Based Landmark Detection Methods on Thermal Face Imagery

Thermal-to-visible face recognition is an emerging technology for low-light and nighttime human identification, for which detection of fiducial landmarks is a critical step required for face alignment prior to recognition. However, thermal images with their low contrast, low resolution, and lack of textural information have proven a challenging obstacle for the detection of the fiducial landmarks used for image alignment. This paper analyzes the ability of modern landmark detection algorithms to cope with the adversarial conditions present in the thermal domain by exploring the strengths and weaknesses of three deep-learning based landmark detection architectures originally developed for visible images: the Deep Alignment Network (DAN), Multi-task Convolutional Neural Network (MTCNN), and a Multi-class Patch-based fullyconvolutional neural network (PBC). Our experiments yield a normalized mean squared error of 0.04 at an offset distance of 2.5 meters using the DAN architecture, indicating an ability for cascaded shape regression neural networks to adapt to thermal images. However, we find that even small alignment errors disproportionately reduce correct recognition rates. With images aligned using the best performing model, an 8.2% drop in EER is observed as compared with ground truth alignments, leaving further room for improvement in this area.
Automatic Modulation Classification Using Interacting Multiple Model Kalman Filter for Channel Estimation

A rigorous model for automatic modulation classification (AMC) in cognitive radio systems is proposed in this paper. This is achieved by exploiting the Kalman filter (KF) integrated with an adaptive interacting multiple model (IMM) for resilient estimation of the channel state information. A novel approach is proposed, in adding up the square root singular values of the decomposed channel using the singular value decompositions algorithm. This new scheme, termed Frobenius eigenmode transmission, is chiefly intended to maintain the total power of all individual effective eigenmodes, as opposed to keeping only the dominant one. The analysis is applied over multiple-input multiple-output (MIMO) antennas in combination with a Rayleigh fading channel using a quasi-likelihood ratio test algorithm for AMC. The expectation-maximization is employed for recursive computation of the underlying estimation and classification algorithms. Novel simulations demonstrate the advantages of the combined IMM-KF structure when compared to the perfectly known channel and maximum likelihood estimate, in terms of achieving the targeted optimal performance with the desirable benefit of less computational complexity loads.