Miracle-dz/UniFAR-FTUs huggingface dataset:
Generic Black-Box End-to-End Attack against RNNs and Other API Calls Based Malware Classifiers

Deep neural networks (DNNs) are used to solve complex classification problems, for which other machine learning classifiers, such as SVM, fall short. Recurrent neural networks (RNNs) have been used for tasks that involves sequential inputs, such as speech to text. In the cyber security domain, RNNs based on API calls have been used effectively to classify previously un-encountered malware. In this paper, we present a blackbox attack against RNNs, focusing on finding adversarial API call sequences that would be misclassified by a RNN without affecting the malware functionality. We also show that this attack is effective against many classifiers, due-to the transferability principle between RNN variants, feed-forward DNNs and traditional machine learning classifiers such as SVM. Finally, we implemented GADGET, a software framework to convert any malware binary to a binary undetected by an API calls based malware classifier, using the proposed attack, without access to the malware source code. We conclude by discussing possible defense mechanisms and countermeasures against the attack.
Allegories: decidability and graph homomorphisms

Allegories were introduced by Freyd and Scedrov; they form a fragment of Tarski's calculus of relations. We show that their equational theory is decidable by characterising it in terms of a specific class of graph homomorphisms. We actually do so for an extension of allegories which we prove to be conservative: allegories with top. This makes it possible to exploit a correspondence between terms and K4-free graphs, for which isomorphisms were known to be finitely axiomatisable.
Hierarchical Metric Learning for Fine Grained Image Classification

This paper deals with the problem of fine-grained image classification and introduces the notion of hierarchical metric learning for the same. It is indeed challenging to categorize fine-grained image classes merely in terms of a single level classifier given the subtle inter-class visual differences. In order to tackle this problem, we propose a two stage framework where i) the image categories are represented hierarchically in terms of a binary tree structure where different subset of classes are present in the non-leaf nodes of the tree. This is accomplished in an automatic fashion considering the available training data in the visual domain, and ii) a (non-leaf) node specific metric learning is further deployed for the categories constituting a given node, thus enforcing better separation between both of its children. Subsequently, we construct (non-leaf) node specific binary classifiers in the learned metric spaces on which testing is henceforth carried out by following the outcomes of the classifiers sequence from root to leaf nodes of the tree. By separately focusing on the semantically similar classes at different levels of the hierarchy, it is expected that the classifiers in the learned metric spaces possess better discriminative capabilities than considering all the classes at a single go. Experimental results obtained on two challenging datasets (Oxford Flowers and Leeds Butterfly) establish the superiority of the proposed framework in comparison to the standard single metric learning based methods convincingly
Fully Context-Aware Video Prediction

This paper proposes a new neural network design for unsupervised learning through video prediction. Current video prediction models based on convolutional networks, recurrent networks, and their combinations often result in blurry predictions. Recent work has attempted to address this issue with techniques like separation of background and foreground modeling, motion flow learning, or adversarial training. We highlight that a contributing factor for this problem is the failure of current architectures to fully capture relevant past information for accurately predicting the future. To address this shortcoming we introduce a fully context-aware architecture, which captures the entire available past context for each pixel using Parallel Multi-Dimensional LSTM units and aggregates it using context blending blocks. Our model is simple, efficient and directly applicable to high resolution video frames. It yields state-of-the-art performance for next step prediction on three challenging real-world video datasets: Human 3.6M, Caltech Pedestrian, and UCF-101 and produces sharp predictions of high visual quality.
Generative Reversible Networks

Generative models with an encoding component such as autoencoders currently receive great interest. However, training of autoencoders is typically complicated by the need to train a separate encoder and decoder model that have to be enforced to be reciprocal to each other. Here, we propose to use the by-design reversible neural networks (RevNets) as a new class of generative models. We investigate the generative performance of RevNets on the CelebA dataset, showing that generative RevNets can generate coherent faces with similar quality as Variational Autoen-coders. This ﬁrst attempt to use RevNets as a generative model slightly underperformed relative to recent advanced generative models using an autoencoder component on CelebA, but this gap may diminish with further optimization of the training setup of generative RevNets. In addition to the experiments on CelebA, we show a proof-of-principle experiment on the MNIST dataset suggesting that adversary-free trained RevNets can discover meaningful latent dimensions without pre-specifying the number of dimensions of the latent sampling distribution. In summary, this study shows that RevNets enable generative applications with an encoding component while over-coming the need to train a separate encoder and decoder model.
Machine vs Machine: Defending Classifiers Against Learning-based Adversarial Attacks

Recently, researchers have discovered that the state-of-the-art object classifiers can be fooled easily by small perturbations in the input unnoticeable to human eyes. Several methods were proposed to craft adversarial examples, as well as methods of robustifying the classifier against such examples. An attacker with the knowledge of the classifier parameters can generate strong adversarial patterns. Conversely, a classifier with the knowledge of such patterns can be trained to be robust to them. The cat-and-mouse game nature of the attacks and the defenses raises the question of the presence of an equilibrium in the dynamic. In this paper, we propose a game framework to formulate the interaction of attacks and defenses and present the natural notion of the best worst-case defense and attack. We propose simple algorithms to numerically find those solutions motivated by sensitivity penalization. In addition, we show the potentials of learning-based attacks, and present the close relationship between the adversarial attack and the privacy attack problems. The results are demonstrated with MNIST and CIFAR-10 datasets.
Beyond Views: Measuring and Predicting Engagement on YouTube Videos

The share of videos in the internet traffic has been growing, therefore understanding how videos capture attention on a global scale is also of growing importance. Most current research focus on modeling the number of views, but we argue that video engagement, or time spent watching is a more appropriate measure for resource allocation problems in attention, networking, and promotion activities. In this paper, we present a first large-scale measurement of video-level aggregate engagement from publicly available data streams, on a collection of 5.3 million YouTube videos published over two months in 2016. We study a set of metrics including time and the average percentage of a video watched. We define a new metric, relative engagement, that is calibrated against video properties and strongly correlate with recognized notions of quality. Moreover, we find that engagement measures of a video are stable over time, thus separating the concerns for modeling engagement and those for popularity -- the latter is known to be unstable over time and driven by external promotions. We also find engagement metrics predictable from a cold-start setup, having most of its variance explained by video context, topics and channel information -- R2=0.77. Our observations imply several prospective uses of engagement metrics -- choosing engaging topics for video production, or promoting engaging videos in recommender systems.
Query-Adaptive R-CNN for Open-Vocabulary Object Detection and Retrieval

We address the problem of open-vocabulary object retrieval and localization, which is to retrieve and localize objects from a very large-scale image database immediately by a textual query (e.g., a word or phrase). We first propose Query-Adaptive R-CNN, a simple yet strong framework for open-vocabulary object detection. Query-Adaptive R-CNN is a simple extension of Faster R-CNN from closed-vocabulary to open-vocabulary object detection: instead of learning a class-specific classifier and regressor, we learn a detector generator that transforms a text into classifier and regressor weights. All of its components can be learned in an end-to-end manner. Even with its simple architecture, it outperforms all state-of-the-art methods in the Flickr30k Entities phrase localization task. In addition, we propose negative phrase augmentation, a generic approach for exploiting hard negatives in the training of open-vocabulary object detection that significantly improves the discriminative ability of the generated classifier. We show that our system can retrieve and localize objects specified by a textual query from one million images in only 0.5 seconds.
Who downloads online content and why?

Online learners sometimes prefer to download course content rather than view it on a course website. These students often miss out on interactive content. Knowing who downloads course materials, and why, can help course creators design courses that fit the needs of their students. In this paper we explore downloading behavior by looking at lecture videos in three online classes. We found that the number of days since a video was posted had the strongest relationship with downloading, and non-technical considerations, such as typical classroom size in a student's home country, matter more than technical issues, such as internet speed. Our findings suggest that more materials will be downloaded when a course will be available for limited time, students are less familiar with the language of instruction, students are used to classrooms with a high student-teacher ratio, or a student's internet speed is slow. Possible reasons for these relationships are discussed.
Universal Denoising Networks : A Novel CNN-based Network Architecture for Image Denoising

We design a novel network architecture for learning discriminative image models that are employed to efficiently tackle the problem of grayscale and color image denoising. Based on the proposed architecture, we introduce two different variants. The first network involves convolutional layers as a core component, while the second one relies instead on non-local filtering layers and thus it is able to exploit the inherent non-local self-similarity property of natural images. As opposed to most of the existing neural networks, which require the training of a specific model for each considered noise level, the proposed networks are able to handle a wide range of different noise levels, while they are very robust when the noise degrading the latent image does not match the statistics of the one used during training. The latter argument is supported by results that we report on publicly available images corrupted by unknown noise and which we compare against solutions obtained by alternative state-of-the-art methods. At the same time the introduced networks achieve excellent results under additive white Gaussian noise (AWGN), which are comparable to the current state-of-the-art network, while they depend on a more shallow architecture with the number of trained parameters being one order of magnitude smaller. These properties make the proposed networks ideal candidates to serve as sub-solvers on restoration methods that deal with general inverse imaging problems such as deblurring, demosaicking, superresolution, etc.
Useful junk?: the effects of visual embellishment on comprehension and memorability of charts

Guidelines for designing information charts (such as bar charts) often state that the presentation should reduce or remove 'chart junk' - visual embellishments that are not essential to understanding the data. In contrast, some popular chart designers wrap the presented data in detailed and elaborate imagery, raising the questions of whether this imagery is really as detrimental to understanding as has been proposed, and whether the visual embellishment may have other benefits. To investigate these issues, we conducted an experiment that compared embellished charts with plain ones, and measured both interpretation accuracy and long-term recall. We found that people's accuracy in describing the embellished charts was no worse than for plain charts, and that their recall after a two-to-three-week gap was significantly better. Although we are cautious about recommending that all charts be produced in this style, our results question some of the premises of the minimalist approach to chart design.
A Deep Generative Model for Disentangled Representations of Sequential Data

We present a VAE architecture for encoding and generating high dimensional sequential data, such as video or audio. Our deep generative model learns a latent representation of the data which is split into a static and dynamic part, allowing us to approximately disentangle latent time-dependent features (dynamics) from features which are preserved over time (content). This architecture gives us partial control over generating content and dynamics by conditioning on either one of these sets of features. In our experiments on artificially generated cartoon video clips and voice recordings, we show that we can convert the content of a given sequence into another one by such content swapping. For audio, this allows us to convert a male speaker into a female speaker and vice versa, while for video we can separately manipulate shapes and dynamics. Furthermore, we give empirical evidence for the hypothesis that stochastic RNNs as latent state models are more efficient at compressing and generating long sequences than deterministic ones, which may be relevant for applications in video compression.
Right Answer for the Wrong Reason: Discovery and Mitigation

Exposing the weaknesses of neural models is crucial for improving their performance and robustness in real-world applications. One common approach is to examine how input perturbations affect the output. Our analysis takes this to an extreme on natural language processing tasks by removing as many words as possible from the input without changing the model prediction. For question answering and natural language inference, this of- ten reduces the inputs to just one or two words, while model confidence remains largely unchanged. This is an undesireable behavior: the model gets the Right Answer for the Wrong Reason (RAWR). We introduce a simple training technique that mitigates this problem while maintaining performance on regular examples.
Diverse and Controllable Image Captioning with Part-of-Speech Guidance

Automatically describing an image is an important capability for virtual assistants. Significant progress has been achieved in recent years on this task of image captioning. However, classical prediction techniques based on maximum likelihood trained LSTM nets don't embrace the inherent ambiguity of image captioning. To address this concern, recent variational auto-encoder and generative adversarial network based methods produce a set of captions by sampling from an abstract latent space. But, this latent space has limited interpretability and therefore, a control mechanism for captioning remains an open problem. This paper proposes a captioning technique conditioned on part-of-speech. Our method provides human interpretable control in form of part-of-speech. Importantly, part-of-speech is a language prior, and conditioning on it provides: (i) more diversity as evaluated by counting n-grams and the novel sentences generated, (ii) achieves high accuracy for the diverse captions on standard captioning metrics.
Unsupervised Hypernym Detection by Distributional Inclusion Vector Embedding

Modeling hypernymy, such as poodle is-a dog, is an important generalization aid to many NLP tasks, such as entailment, relation extraction, and question answering. Supervised learning from labeled hypernym sources, such as WordNet, limit the coverage of these models, which can be addressed by learning hypernyms from unlabeled text. Existing unsupervised methods either do not scale to large vocabularies or yield unacceptably poor accuracy. This paper introduces distributional inclusion vector embedding (DIVE), a simple-to-implement unsupervised method of hypernym discovery via per-word non-negative vector embeddings learned by modeling diversity of word context with specialized negative sampling. In an experimental evaluation more comprehensive than any previous literature of which we are aware - evaluating on 11 datasets using multiple existing as well as newly proposed scoring metrics - we find that our method can provide up to double or triple the precision of previous unsupervised methods, and also sometimes outperforms previous semi-supervised methods, yielding many new state-of-the-art results.
Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning

—There has recently been a surge of work in explanatory artiﬁcial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, providing explanations of their decisions in some level of detail. These explanations are important to ensure algorithmic fairness, identify potential bias/problems in the training data, and to ensure that the algorithms perform as expected. However, explanations produced by these systems is neither standardized nor systematically assessed. In an effort to create best practices and identify open challenges, we provide our deﬁnition of explainability and show how it can be used to classify existing literature. We discuss why current approaches to explanatory methods especially for deep neural networks are insufﬁcient. Finally, based on our survey, we conclude with suggested future research directions for explanatory artiﬁcial intelligence.
Sketch-to-Image Generation Using Deep Contextual Completion

When the input to pix2pix translation is a badly drawn sketch, the output follows the input edges due to the strict alignment imposed by the translation process. In this paper we propose sketch-to-image generation, where the output edges do not necessarily follow the input edges. We address the image generation problem using a novel joint image completion approach, where the sketch provides the image context for completing, or generating the output image. We train a deep generative model to learn the joint distribution of sketch and the corresponding image by using joint images. Our deep contextual completion approach has several advantages. First, the simple joint image representation allows for simple and effective definition of losses in the same joint image-sketch space, which avoids complicated issues in cross-domain learning. Second, while the output is related to its input overall, the generated features exhibit more freedom in appearance and do not strictly align with the input features. Third, from the joint image's point of view, image and sketch are of no difference, thus exactly the same deep joint image completion network can be used for image-to-sketch generation. Experiments evaluated on three different datasets show that the proposed approach can generate more realistic images than the state-ofthe- arts on challenging inputs and generalize well on common categories.
Indoor Scene Understanding in 2.5/3D: A Survey

With the availability of low-cost and compact 2.5/3D visual sensing devices, computer vision community is experiencing a growing interest in visual scene understanding. This survey paper provides a comprehensive background to this research topic. We begin with a historical perspective, followed by popular 3D data representations and a comparative analysis of available datasets. Before delving into the application specific details, this survey provides a succinct introduction to the core technologies that are the underlying methods extensively used in the literature. Afterwards, we review the developed techniques according to a taxonomy based on the scene understanding tasks. This covers holistic indoor scene understanding as well as subtasks such as scene classification, object detection, pose estimation, semantic segmentation, 3D reconstruction, saliency detection, physics-based reasoning and affordance prediction. Later on, we summarize the performance metrics used for evaluation in different tasks and a quantitative comparison among the recent state-of-the-art techniques. We conclude this review with the current challenges and an outlook towards the open research problems requiring further investigation.
Arbitrary Facial Attribute Editing: Only Change What You Want

Facial attribute editing aims to modify either single or multiple attributes on a face image. Recently, the generative adversarial net (GAN) and the encoder-decoder architecture are usually incorporated to handle this task. The attribute editing can then be conducted by decoding the latent representation of the face image conditioned on the specified attributes. A few existing methods attempt to establish attribute-independent latent representation for arbitrarily changing the attributes. However, since the attributes portray the characteristics of the face image, the attribute-independent constraint on the latent representation is excessive. Such constraint may result in information loss and unexpected distortion on the generated images (e.g. over smoothing), especially for those identifiable attributes such as gender, race etc. Instead of imposing the attribute-independent constraint on the latent representation, we introduce an attribute classification constraint on the generated image, just requiring the correct change of the attributes. Meanwhile, reconstruction learning is introduced in order to guarantee the preservation of all other attribute-excluding details on the generated image, and adversarial learning is employed for visually realistic generation. Moreover, our method can be naturally extended to attribute intensity manipulation. Experiments on the CelebA dataset show that our method outperforms the state-of-the-arts on generating realistic attribute editing results with facial details well preserved.