Reviewerly/exHarmony huggingface dataset:
Unsupervised Learning with Contrastive Latent Variable Models

In unsupervised learning, dimensionality reduction is an important tool for data exploration and visualization. Because these aims are typically open-ended, it can be useful to frame the problem as looking for patterns that are enriched in one dataset relative to another. These pairs of datasets occur commonly, for instance a population of interest vs. control or signal vs. signal free recordings. However, there are few methods that work on sets of data as opposed to data points or sequences. Here, we present a probabilistic model for dimensionality reduction to discover signal that is enriched in the target dataset relative to the background dataset. The data in these sets do not need to be paired or grouped beyond set membership. By using a probabilistic model where some structure is shared amongst the two datasets and some is unique to the target dataset, we are able to recover interesting structure in the latent space of the target dataset. The method also has the advantages of a probabilistic model, namely that it allows for the incorporation of prior information, handles missing data, and can be generalized to different distributional assumptions. We describe several possible variations of the model and demonstrate the application of the technique to de-noising, feature selection, and subgroup discovery settings.
Robust Bhattacharyya bound linear discriminant analysis through an adaptive algorithm

In this paper, we propose a novel linear discriminant analysis (LDA) criterion via the Bhattacharyya error bound estimation based on a novel L1-norm (L1BLDA) and L2-norm (L2BLDA). Both L1BLDA and L2BLDA maximize the between-class scatters which are measured by the weighted pairwise distances of class means and meanwhile minimize the within-class scatters under the L1-norm and L2-norm, respectively. The proposed models can avoid the small sample size (SSS) problem and have no rank limit that may encounter in LDA. It is worth mentioning that, the employment of L1-norm gives a robust performance of L1BLDA, and L1BLDA is solved through an effective non-greedy alternating direction method of multipliers (ADMM), where all the projection vectors can be obtained once for all. In addition, the weighting constants of L1BLDA and L2BLDA between the between-class and within-class terms are determined by the involved data, which makes our L1BLDA and L2BLDA more adaptive. The experimental results on both benchmark data sets as well as the handwritten digit databases demonstrate the effectiveness of the proposed methods.
Bone Marrow Cells Detection: A Technique for the Microscopic Image Analysis

In the detection of myeloproliferative, the number of cells in each type of bone marrow cells (BMC) is an important parameter for the evaluation. In this study, we propose a new counting method, which consists of three modules including localization, segmentation and classification. The localization of BMC is achieved from a color transformation enhanced BMC sample image and stepwise averaging method. In the nucleus segmentation, both stepwise averaging method and Otsu’s method are applied to obtain a weighted threshold for segmenting the patch into nucleus and non-nucleus. In the cytoplasm segmentation, a color weakening transformation, an improved region growing method and the K-Means algorithm are employed. The connected cells with BMC will be separated by the marker-controlled watershed algorithm. The features will be extracted for the classification after the segmentation. In this study, the BMC are classified using the support vector machine into five classes; namely, neutrophilic split granulocyte, neutrophilic stab granulocyte, metarubricyte, mature lymphocytes and the outlier (all other cells not listed). Experimental results show that the proposed method achieves superior segmentation and classification performance with an average segmentation accuracy of 91.76% and an average recall rate of 87.49%. The comparison shows that the proposed segmentation and classification methods outperform the existing methods.
Verifying Robustness of Gradient Boosted Models

Gradient boosted models are a fundamental machine learning technique. Robustness to small perturbations of the input is an important quality measure for machine learning models, but the literature lacks a method to prove the robustness of gradient boosted models.This work introduces VERIGB, a tool for quantifying the robustness of gradient boosted models. VERIGB encodes the model and the robustness property as an SMT formula, which enables state of the art verification tools to prove the model’s robustness. We extensively evaluate VERIGB on publicly available datasets and demonstrate a capability for verifying large models. Finally, we show that some model configurations tend to be inherently more robust than others.
Learning Instance-wise Sparsity for Accelerating Deep Models

Exploring deep convolutional neural networks of high efficiency and low memory usage is very essential for a wide variety of machine learning tasks. Most of existing approaches used to accelerate deep models by manipulating parameters or filters without data, e.g., pruning and decomposition. In contrast, we study this problem from a different perspective by respecting the difference between data. An instance-wise feature pruning is developed by identifying informative features for different instances. Specifically, by investigating a feature decay regularization, we expect intermediate feature maps of each instance in deep neural networks to be sparse while preserving the overall network performance. During online inference, subtle features of input images extracted by intermediate layers of a well-trained neural network can be eliminated to accelerate the subsequent calculations. We further take coefficient of variation as a measure to select the layers that are appropriate for acceleration. Extensive experiments conducted on benchmark datasets and networks demonstrate the effectiveness of the proposed method.
Terminal brain damage: exposing the graceless degradation in deep neural networks under hardware fault attacks

Deep neural networks (DNNs) have been shown to tolerate brain damage: cumulative changes to the network's parameters (e.g., pruning, numerical perturbations) typically result in a graceful degradation of classification accuracy. However, the limits of this natural resilience are not well understood in the presence of small adversarial changes to the DNN parameters' underlying memory representation, such as bit-flips that may be induced by hardware fault attacks. We study the effects of bitwise corruptions on 19 DNN models--six architectures on three image classification tasks--and we show that most models have at least one parameter that, after a specific bit-flip in their bitwise representation, causes an accuracy loss of over 90%. For large models, we employ simple heuristics to identify the parameters likely to be vulnerable and estimate that 40-50% of the parameters in a model might lead to an accuracy drop greater than 10% when individually subjected to such single-bit perturbations. To demonstrate how an adversary could take advantage of this vulnerability, we study the impact of an exemplary hardware fault attack, Rowhammer, on DNNs. Specifically, we show that a Rowhammer-enabled attacker co-located in the same physical machine can inflict significant accuracy drops (up to 99%) even with single bitflip corruptions and no knowledge of the model. Our results expose the limits of DNNs' resilience against parameter perturbations induced by real-world fault attacks. We conclude by discussing possible mitigations and future research directions towards fault attack-resilient DNNs.
Deep Active Learning with Adaptive Acquisition

Model selection is treated as a standard performance boosting step in many machine learning applications. Once all other properties of a learning problem are fixed, the model is selected by grid search on a held-out validation set. This is strictly inapplicable to active learning. Within the standardized workflow, the acquisition function is chosen among available heuristics a priori, and its success is observed only after the labeling budget is already exhausted. More importantly, none of the earlier studies report a unique consistently successful acquisition heuristic to the extent to stand out as the unique best choice. We present a method to break this vicious circle by defining the acquisition function as a learning predictor and training it by reinforcement feedback collected from each labeling round. As active learning is a scarce data regime, we bootstrap from a well-known heuristic that filters the bulk of data points on which all heuristics would agree, and learn a policy to warp the top portion of this ranking in the most beneficial way for the character of a specific data distribution. Our system consists of a Bayesian neural net, the predictor, a bootstrap acquisition function, a probabilistic state definition, and another Bayesian policy network that can effectively incorporate this input distribution. We observe on three benchmark data sets that our method always manages to either invent a new superior acquisition function or to adapt itself to the a priori unknown best performing heuristic for each specific data set.
Statistical Guarantees for the Robustness of Bayesian Neural Networks

We introduce a probabilistic robustness measure for Bayesian Neural Networks (BNNs), defined as the probability that, given a test point, there exists a point within a bounded set such that the BNN prediction differs between the two. Such a measure can be used, for instance, to quantify the probability of the existence of adversarial examples. Building on statistical verification techniques for probabilistic models, we develop a framework that allows us to estimate probabilistic robustness for a BNN with statistical guarantees, i.e., with a priori error and confidence bounds. We provide experimental comparison for several approximate BNN inference techniques on image classification tasks associated to MNIST and a two-class subset of the GTSRB dataset. Our results enable quantification of uncertainty of BNN predictions in adversarial settings.
GEOCAPSNET: Ground to Aerial View Image Geo-Localization using Capsule Network

The task of cross-view image geo-localization aims to determine the geo-location (GPS coordinates) of a query ground-view image by matching it with the GPS-tagged aerial (satellite) images in a reference dataset. Due to the dramatic changes of viewpoint, matching the cross-view images is challenging. In this paper, we propose the GeoCapsNet based on the capsule network for ground-to-aerial image geo-localization. The network first extracts features from both ground and aerial images via standard convolution layers and the capsule layers further encode the features to model the spatial feature hierarchies and enhance the representation power. Moreover, we introduce a simple and effective weighted soft-margin triplet loss with online batch hard sample mining, which can greatly improve the image retrieval accuracy. Experimental results show that our GeoCapsNet significantly outperforms the state-of-the-art approaches on two benchmark datasets.
Ontology and Rule-Based Recommender System for E-learning Applications

The continuous growth of the internet has given rise to an overwhelming mass of learning materials. Which has increased the demand for a recommendation system to filter information and to deliver the learning materials that fit learners learning context. In this paper, we propose an architecture of a semantic web based recommender system. The proposed architecture is a redesigned architecture of the classical 3-tiers web application architecture with an additional semantic layer. This layer holds two semantic subsystems: an Ontology-based subsystem and SWRL (Semantic Web Rule Language) rules one. The Ontology subsystem is used as a reusable and sharable domain knowledge to model the learning content and context. The SWRL rules are used as a recommendation and filtering technique based on learning object relevance and weighting. These rules are organized into four categories: Learning History Rules (LHR), Learning Performance Rules (LPR), Learning Social Network Rules (LSNR) and Learning Pathway Rules (PR).
Co-attention Memory Network for Multimodal Microblog's Hashtag Recommendation

Hashtags are keywords describing a topic or a theme and are usually chosen by microblogging users. Hence, the hashtags can be used to categorize microblog posts. With the fast development of the social network, the task of recommending suitable hashtags has received considerable attention in recent years. Recently, most neural network methods have treated the task as a multi-class classification problem. In fact, users are constantly introducing new hashtags in a highly dynamic way. Treating the task as a multi-class classification problem with a fixed number of target categories does not allow the method to deal with the new hashtags. To address this problem, the task is reinterpreted as a matching problem and a novel co-attention memory network is proposed to represent the multimodal microblogs and hashtags. We utilize a co-attention mechanism to model the multimodal mircroblogs, and utilize the post history to represent the hashtags. Experimental results on a Twitter-based dataset demonstrated that the proposed method can achieve better performance than the current state-of-the-art methods that treat the task as a multi-class classification problem.
Real-Time Adversarial Attacks

In recent years, many efforts have demonstrated that modern machine learning algorithms are vulnerable to adversarial attacks, where small, but carefully crafted, perturbations on the input can make them fail. While these attack methods are very effective, they only focus on scenarios where the target model takes static input, i.e., an attacker can observe the entire original sample and then add a perturbation at any point of the sample. These attack approaches are not applicable to situations where the target model takes streaming input, i.e., an attacker is only able to observe past data points and add perturbations to the remaining (unobserved) data points of the input. In this paper, we propose a real-time adversarial attack scheme for machine learning models with streaming inputs.
Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning

An increasing number of well-trained deep networks have been released online by researchers and developers, enabling the community to reuse them in a plug-and-play way without accessing the training annotations. However, due to the large number of network variants, such public-available trained models are often of different architectures, each of which being tailored for a specific task or dataset. In this paper, we study a deep-model reusing task, where we are given as input pre-trained networks of heterogeneous architectures specializing in distinct tasks, as teacher models. We aim to learn a multitalented and light-weight student model that is able to grasp the integrated knowledge from all such heterogeneous-structure teachers, again without accessing any human annotation. To this end, we propose a common feature learning scheme, in which the features of all teachers are transformed into a common space and the student is enforced to imitate them all so as to amalgamate the intact knowledge. We test the proposed approach on a list of benchmarks and demonstrate that the learned student is able to achieve very promising performance, superior to those of the teachers in their specialized tasks.
Identity Preserving Generative Adversarial Network for Cross-Domain Person Re-Identification

In this paper, we study the domain adaptive person re-identification(re-ID) problem: train a re-ID model on the labeled source domain and test it on the unlabeled target domain.It's known challenging due to the feature distribution bias between the source domain and target domain.The previous methods directly reduce the bias by image-to-image style translation between the source and the target domain in an unsupervised manner.However, these methods only consider the rough bias between the source domain and the target domain but neglect the detailed bias between the source domain and the target camera domains (divided by camera views), which contain critical factors influencing the testing performance of re-ID model.In this work, we particularly focus on the bias between the source domain and the target camera domains.To overcome this problem, a multi-domain image-to-image translation network, termed Identity Preserving Generative Adversarial Network (IPGAN) is proposed to learn the mapping relationship between the source domain and the target camera domains.IPGAN can translate the styles of images from the source domain to the target camera domains and generate many images with styles of target camera domains.Then the re-ID model is trained with the translated images generated by IPGAN.During the training of the re-ID model, we aim to learn the discriminative feature.We design and train a novel re-ID model, termed IBN-reID, in which Instance and Batch Normalization block (IBN-block) are introduced.Experimental results on Market-1501, DukeMTMC-reID and MSMT17 show that the images generated by IPGAN are more suitable for cross-domain re-ID.Very competitive re-ID accuracy is achieved by our method.
Learning a Deep Vector Quantization Network for Image Compression

Deep convolutional neural network (DCNN) based image codecs, consisting of encoder, quantizer and decoder, have achieved promising image compression results. The major challenge in learning these DCNN models lies in the joint optimization of the encoder, quantizer and decoder, as well as the adaptivity to the input images. In this paper, we proposed a DCNN architecture for image compression, where the encoder, quantizer and decoder are jointly learned. Specifically, a fully convolutional vector quantization network (VQNet) has been proposed to quantize the feature vectors of the image representation, where the representative vectors of the VQNet are jointly optimized with other network parameters through end-to-end training. While most of current DCNN-based methods were only trained once on large-scale datasets, we further perform fine-tuning of the encoder and the codes generated by the VQNet on the input images to further improve the compression performance. Extensive experimental results show that the proposed method achieves state-of-the-art compression results with simple encoder-decoder.
Immersive Gastronomic Experience with Distributed Reality

We have developed an immersive gastronomic experience as a proof of concept of Distributed Reality, a type of Augmented Virtuality which combines a reality transmitted from a remote place, using 360° video, with a local reality, using video see-through. In order to reach fully immersive experience, local objects of interest such as hands and local food are segmented using red chrominance keying. Only those segmented objects are merged with the remote reality, enabling this way to increase self-presence and to allow user interaction. More concretely, the gastronomic experience consists of tasting small pieces of food, while being immersed in a remote place designed to pair with the food, thus creating an innovative concept with potential impact in hospitality and tourism industries. An evaluation performed with 66 users shows that it provides good levels of immersion, local interactivity, and general user satisfaction. The application achieves real time performance and has been developed for a smartphone mounted on a consumer headset, thus being easy to deploy and to reuse in other use cases.
Facial Expression Recognition Based on Fusion Features of Center-Symmetric Local Signal Magnitude Pattern

Local feature descriptors play a fundamental and important role in facial expression recognition. This paper presents a new descriptor, Center-Symmetric Local Signal Magnitude Pattern (CS-LSMP), which is used for extracting texture features from facial images. CS-LSMP operator takes signal and magnitude information of local regions into account compared to conventional LBP-based operators. Additionally, due to the limitation of single feature extraction method and in order to make full advantages of different features, this paper employs CS-LSMP operator to extract features from Orientational Magnitude Feature Maps (OMFMs), Positive-and-Negative Magnitude Feature Maps (PNMFMs), Gabor Feature Maps (GFMs) and facial patches (eyebrows-eyes, mouths) for obtaining fused features. Unlike HOG, which only retains horizontal and vertical magnitudes, our work generates Orientational Magnitude Feature Maps (OMFMs) by expanding multi-orientations. This paper build two distinct feature maps by dividing local magnitudes into two groups, i.e., positive and negative magnitude feature maps. The generated Gabor Feature Maps (GFMs) are also grouped to reduce the computational complexity. Experiments on the JAFFE and CK+ facial expression datasets showed that the proposed framework achieved significant improvement and outperformed some state-of-the-art methods.
STANCY: Stance Classification Based on Consistency Cues

Controversial claims are abundant in online media and discussion forums.A better understanding of such claims requires analyzing them from different perspectives.Stance classification is a necessary step for inferring these perspectives in terms of supporting or opposing the claim.In this work, we present a neural network model for stance classification leveraging BERT representations and augmenting them with a novel consistency constraint.Experiments on the Perspectrum dataset, consisting of claims and users' perspectives from various debate websites, demonstrate the effectiveness of our approach over state-of-the-art baselines.
Learning Programmatic Idioms for Scalable Semantic Parsing

Programmers typically organize executable source code using high-level coding patterns or idiomatic structures such as nested loops, exception handlers and recursive blocks, rather than as individual code tokens.In contrast, state of the art (SOTA) semantic parsers still map natural language instructions to source code by building the code syntax tree one node at a time.In this paper, we introduce an iterative method to extract code idioms from large source code corpora by repeatedly collapsing most-frequent depth-2 subtrees of their syntax trees, and train semantic parsers to apply these idioms during decoding.Applying idiom-based decoding on a recent contextdependent semantic parsing task improves the SOTA by 2.2% BLEU score while reducing training time by more than 50%.This improved speed enables us to scale up the model by training on an extended training set that is 5× larger, to further move up the SOTA by an additional 2.3% BLEU and 0.9% exact match.Finally, idioms also significantly improve accuracy of semantic parsing to SQL on the ATIS-SQL dataset, when training data is limited.