Miracle-dz/UniFAR-FTUs huggingface dataset:
RIOT: a Novel Stochastic Method for Rapidly Configuring Cloud-Based Workflows

Traditional tools for configuring cloud services can run much slower than the workflows they are trying to optimize. For example, in the case studies reported here, we find cases where (using traditional methods) it takes hours to find ways to make a workflow terminate in tens of seconds. Such slow optimizers are a poor choice of tools for reacting to changing operational environmental conditions. Hence, they are unsuited for cloud services that support rapidly changing workflows, e.g., scientific workflows or workflows from the media or telecommunication industries. To solve this problem, this paper presents RIOT (Randomized Instance Order Types), a new configuration tool. RIOT has a very low optimization overhead-- often, less than 10\% of the system runtime, especially for every complex workflow. Instead of simulating many configurations, RIOT uses a novel surrogate sampling method to quickly find promising solutions. As shown by this paper, RIOT achieves comparable results to the other approaches but does so in a fraction of the time.
An Empirical Evaluation of Recurrent Neural Network Rule Extraction

Rule extraction from black-box models is critical in domains that require model validation before implementation, as can be the case in credit scoring and medical diagnosis. Though already a challenging problem in statistical learning in general, the difficulty is even greater when highly non-linear, recursive models, like recurrent neural networks (RNNs), are fit to data. Here, we study the extraction of rules from second order recurrent neural networks (RNNs) trained to recognize the Tomita grammars. We show that production rules can be stably extracted from trained RNNs and that in certain cases the rules outperform the trained RNNs.
RESIDE: A Benchmark for Single Image Dehazing

In this paper, we present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE). RESIDE highlights diverse data sources and image contents, and is divided into five subsets, each serving different training or evaluation purposes. We further provide a rich variety of criteria for dehazing algorithm evaluation, ranging from full-reference metrics, to no-reference metrics, to subjective evaluation and the novel task-driven evaluation. Experiments on RESIDE sheds light on the comparisons and limitations of state-of-the-art dehazing algorithms, and suggest promising future directions.
Fine-grained Video Classification and Captioning

We describe a DNN for fine-grained action classification and video captioning. It gives state-of-the-art performance on the challenging Something-Something dataset, with over 220, 000 videos and 174 fine-grained actions. Classification and captioning on this dataset are challenging because of the subtle differences between actions, the use of thousands of different objects, and the diversity of captions penned by crowd actors. The model architecture shares features for classification and captioning, and is trained end-to-end. It performs much better than the existing classification benchmark for Something-Something, with impressive fine-grained results, and it yields a strong baseline on the new Something-Something captioning task. Our results reveal that there is a strong correlation between the degree of detail in the task and the ability of the learned features to transfer to other tasks.
Deep Unsupervised Intrinsic Image Decomposition by Siamese Training

We harness modern intrinsic decomposition tools based on deep learning to increase their applicability on realworld use cases. Traditional techniques are derived from the Retinex theory: handmade prior assumptions constrain an optimization to yield a unique solution that is qualitatively satisfying on a limited set of examples. Modern techniques based on supervised deep learning leverage largescale databases that are usually synthetic or sparsely annotated. Decomposition quality on images in the wild is therefore arguable. We propose an end-to-end deep learning solution that can be trained without any ground truth supervision, as this is hard to obtain. Time-lapses form an ubiquitous source of data that (under a scene staticity assumption) capture a constant albedo under varying shading conditions. We exploit this natural relationship to train in an unsupervised siamese manner on image pairs. Yet, the trained network applies to single images at inference time. We present a new dataset to demonstrate our siamese training on, and reach results that compete with the state of the art, despite the unsupervised nature of our training scheme. As evaluation is difficult, we rely on extensive experiments to analyze the strengths and weaknesses of our and related methods.
Flex-Convolution (Deep Learning Beyond Grid-Worlds)

The goal of this work is to enable deep neural networks to learn representations for irregular 3D structures -- just like in common approaches for 2D images. Unfortunately, current network primitives such as convolution layers are specifically designed to exploit the natural data representation of images -- a fixed and regular grid structure. This represents a limitation when transferring these techniques to more unstructured data like 3D point clouds or higher dimensional data. In this work, we propose a surprisingly natural generalization flex-convolution of the conventional convolution layer and provide a highly efficient implementation. Compared to very specific neural network architectures for point cloud processing, our more generic approach yields competitive results on the rather small standard benchmark sets using fewer parameters and lower memory consumption. Our design even allows for raw neural networks prediction on several magnitudes larger point clouds, providing superior results compared to previous hand-tuned and well-engineered approaches on the 2D-3D-S dataset.
Coverage-Rate Tradeoff in mmWave Heterogeneous Networks with Generalized User Association

In this paper, we first introduce a generalized modeling and analysis framework to explore the fundamental interactions between user association, coverage probability and link rate in a millimeter wave (mmWave) heterogeneous network (HetNet) in which there are multiple tiers of the ultra-high-frequency (UHF) macrocell and small cell base stations (BSs) and a single tier of mmWave small cell BSs. A generalized user association scheme that can cover many pathloss-based user association schemes is proposed and its related probabilistic properties that facilitate the derivations of the coverage probability and link rate are derived. The derived general expressions of the coverage and link rate not only shed light on how to design user association functions in order to maximizes the coverage and rate but also show that it is impossible to devise a user association scheme that maximizes the coverage and link rate at the same time. Namely, there exists a fundamental tradeoff between coverage and link rate in an mmWave HetNet with distinct bandwidths in the UHF and mmWave bands. We characterize the coverage-optimal and rate-optimal user association schemes and numerically validate their performances and show the tradeoff problem.
Multi-Modal Multi-Task Deep Learning for Autonomous Driving

Several deep learning approaches have been applied to the autonomous driving task, many employing end-to-end deep neural networks. Autonomous driving is complex, utilizing multiple behavioral modalities ranging from lane changing to turning and stopping. However, most existing approaches do not factor in the different behavioral modalities of the driving task into the training strategy. This paper describes a technique for using Multi-Modal Multi-Task Learning that considers multiple behavioral modalities as distinct modes of operation for an end-to-end autonomous deep neural network utilizing the insertion of modal information as secondary input data. Using labeled data from hours of driving our fleet of 1/10th scale model cars, we trained multiple neural networks to imitate the steering angle and driving speed of human control of a car. We show that in each case, our models trained with MTL can match or outperform multiple networks trained on individual tasks, while using a fraction of the parameters and having more distinct modes of operation than a network trained without MTL on the same multi-modal data. These results should encourage Multi-Modal MTL-style training with the insertion of Modal Information for tasks with related behaviors.
Transferring Common-Sense Knowledge for Object Detection

We propose the idea of transferring common-sense knowledge from source categories to target categories for scalable object detection. In our setting, the training data for the source categories have bounding box annotations, while those for the target categories only have image-level annotations. Current state-of-the-art approaches focus on image-level visual or semantic similarity to adapt a detector trained on the source categories to the new target categories. In contrast, our key idea is to (i) use similarity not at image-level, but rather at region-level, as well as (ii) leverage richer common-sense (based on attribute, spatial, etc.,) to guide the algorithm towards learning the correct detections. We acquire such common-sense cues automatically from readily-available knowledge bases without any extra human effort. On the challenging MS COCO dataset, we find that using common-sense knowledge substantially improves detection performance over existing transfer-learning baselines.
Two complete axiomatisations of pure-state qubit quantum computing

Categorical quantum mechanics places finite-dimensional quantum theory in the context of compact closed categories, with an emphasis on diagrammatic reasoning. In this framework, two equational diagrammatic calculi have been proposed for pure-state qubit quantum computing: the ZW calculus, developed by Coecke, Kissinger and the first author for the purpose of qubit entanglement classification, and the ZX calculus, introduced by Coecke and Duncan to give an abstract description of complementary observables. Neither calculus, however, provided a complete axiomatisation of their model. In this paper, we present extended versions of ZW and ZX, and show their completeness for pure-state qubit theory, thus solving two major open problems in categorical quantum mechanics. First, we extend the original ZW calculus to represent states and linear maps with coefficients in an arbitrary commutative ring, and prove completeness by a strategy that rewrites all diagrams into a normal form. We then extend the language and axioms of the original ZX calculus, and show their completeness for pure-state qubit theory through a translation between ZX and ZW specialised to the field of complex numbers. This translation expands the one used by Jeandel, Perdrix, and Vilmart to derive an axiomatisation of the approximately universal Clifford+T fragment; restricting the field of complex numbers to a suitable subring, we obtain an alternative axiomatisation of the same theory.
Do Convolutional Neural Networks act as Compositional Nearest Neighbors?

We present a simple approach based on pixel-wise nearest neighbors to understand and interpret the functioning of state-of-the-art neural networks for pixel-level tasks. We aim to understand and uncover the synthesis/prediction mechanisms of state-of-the-art convolutional neural networks. To this end, we primarily analyze the synthesis process of generative models and the prediction mechanism of discriminative models. The main hypothesis of this work is that convolutional neural networks for pixel-level tasks learn a fast compositional nearest neighbor synthesis/prediction function. Our experiments on semantic segmentation and image-to-image translation show qualitative and quantitative evidence supporting this hypothesis.
Context of Visual Information Seeking

Information seeking is an interactive behavior of the end users with information systems, which occurs in a real environment known as context. Context affects information seeking behavior in many different ways. The aim of this article was to investigate the factors that potentially constitute the context of visual information seeking. We utilized Straussian version of grounded theory, a qualitative approach, to conduct the study. Using a purposive sampling method, twenty-eight subjects participated in the study. The data was analyzed using open, axial and selective coding in MAXQDA software. The contextual factors influencing visual information seeking were classified into seven categories, including: user characteristics, general search features, visual search features, display of results, accessibility and usability of results, task type, and environmental factors. This study contributes to a better understanding of how people conduct searches in and interact with visual search interfaces. Results have important implications for the designers of information retrieval systems.
Learning to parse from a semantic objective: It works. Is it syntax?

Recent work on reinforcement learning and other gradient estimators for latent tree learning has made it possible to train neural networks that learn to both parse a sentence and use the resulting parse to interpret the sentence, all without exposure to ground-truth parse trees at training time. Surprisingly, these models often perform better at sentence understanding tasks than models that use parse trees from conventional parsers. This paper aims to investigate what these latent tree learning models learn. We replicate two such models in a shared codebase and find that (i) they do outperform baselines on sentence classification, but that (ii) their parsing strategies are not especially consistent across random restarts, (iii) the parses they produce tend to be shallower than PTB parses, and (iv) these do not resemble those of PTB or of any other recognizable semantic or syntactic grammar formalism.
OmniDetector: With Neural Networks to Bounding Boxes

We propose a person detector on omnidirectional images, an accurate method to generate minimal enclosing rectangles of persons. The basic idea is to adapt the qualitative detection performance of a convolutional neural network based method, namely YOLOv2 to fish-eye images. The design of our approach picks up the idea of a state-of-the-art object detector and highly overlapping areas of images with their regions of interests. This overlap reduces the number of false negatives. Based on the raw bounding boxes of the detector we fine-tuned overlapping bounding boxes by three approaches. The non-maximum suppression, the soft non-maximum suppression and the soft non-maximum suppression with Gaussian smoothing. The evaluation was done on the PIROPO database, supplemented with bounding boxes on omnidirectional images. We achieve an average precision of 64.4 % with YOLOv2 for the class person. For this purpose we fine-tuned the soft non-maximum suppression with Gaussian smoothing.
Modeling and Evaluation of Synchronous Stochastic Gradient Descent in Distributed Deep Learning on Multiple GPUs

With huge amounts of training data, deep learning has made great breakthroughs in many artificial intelligence (AI) applications. However, such large-scale data sets present computational challenges, requiring training to be distributed on a cluster equipped with accelerators like GPUs. With the fast increase of GPU computing power, the data communications among GPUs have become a potential bottleneck on the overall training performance. In this paper, we first propose a general directed acyclic graph (DAG) model to describe the distributed synchronous stochastic gradient descent (S-SGD) algorithm, which has been widely used in distributed deep learning frameworks. To understand the practical impact of data communications on training performance, we conduct extensive empirical studies on four state-of-the-art distributed deep learning frameworks (i.e., Caffe-MPI, CNTK, MXNet and TensorFlow) over multi-GPU and multi-node environments with different data communication techniques, including PCIe, NVLink, 10GbE, and InfiniBand. Through both analytical and experimental studies, we identify the potential bottlenecks and overheads that could be further optimized. At last, we make the data set of our experimental traces publicly available, which could be used to support simulation-based studies.
Model-Based Action Exploration

Deep reinforcement learning has great stride in solving challenging motion control tasks. Recently there has been a significant amount of work on methods to exploit the data gathered during training, but less work is done on good methods for generating data to learn from. For continuous actions domains, the typical method for generating exploratory actions is by sampling from a Gaussian distribution centred around the mean of a policy. Although these methods can find an optimal policy, in practise, they do not scale well, and solving environments with many actions dimensions becomes impractical. We consider learning a forward dynamics model to predict the result, ($s_{t+1}$), of taking a particular action, ($a$), given a specific observation of the state, ($s_{t}$). With a model such as this we, can perform what comes more naturally to biological systems that have already collect experience, we perform internal predictions of outcomes and endeavour to try actions we believe have a reasonable chance of success. This method greatly reduces the space of exploratory actions, increasing learning speed and enables higher quality solutions to difficult problems, such as robotic locomotion.
Manifold Assumption and Defenses Against Adversarial Perturbations

In the adversarial-perturbation problem of neural networks, an adversary starts with a neural network model $F$ and a point $\bfx$ that $F$ classifies correctly, and applies a \emph{small perturbation} to $\bfx$ to produce another point $\bfx'$ that $F$ classifies \emph{incorrectly}. In this paper, we propose taking into account \emph{the inherent confidence information} produced by models when studying adversarial perturbations, where a natural measure of ``confidence'' is \|F(\bfx)\|_\infty$ (i.e. how confident $F$ is about its prediction? ). Motivated by a thought experiment based on the manifold assumption, we propose a ``goodness property'' of models which states that \emph{confident regions of a good model should be well separated}. We give formalizations of this property and examine existing robust training objectives in view of them. Interestingly, we find that a recent objective by Madry et al. encourages training a model that satisfies well our formal version of the goodness property, but has a weak control of points that are wrong but with low confidence. However, if Madry et al.'s model is indeed a good solution to their objective, then good and bad points are now distinguishable and we can try to embed uncertain points back to the closest confident region to get (hopefully) correct predictions. We thus propose embedding objectives and algorithms, and perform an empirical study using this method. Our experimental results are encouraging: Madry et al.'s model wrapped with our embedding procedure achieves almost perfect success rate in defending against attacks that the base model fails on, while retaining good generalization behavior.
A2-RL: Aesthetics Aware Reinforcement Learning for Automatic Image Cropping

Image cropping aims at improving the aesthetic quality of images by adjusting their composition. Most previous methods rely on the sliding window mechanism. The sliding window mechanism requires fixed aspect ratios and limits the cropping region with arbitrary size. Moreover, the sliding window method usually produces tens of thousands of windows which is very time-consuming. Motivated by these challenges and also inspired by the human's cropping process, we firstly formulate the aesthetic image cropping as a sequential decision-making process and propose an Aesthetics Aware Reinforcement Learning (A2-RL) framework to address this problem. Particularly, the proposed method develops an aesthetics aware reward function which especially benefits image cropping. Similar to human's decision making and to better utilize the historical experience, we use a LSTM based state representation including both the current and historical experience. We train the agent using the actor-critic architecture in an end-to-end manner. The agent is evaluated on several popular unseen cropping databases. Experiment results show that our method achieves the state-of-the-art performance with much fewer candidate windows and much less time compared with previous methods.
Learning a Generative Adversarial Network for High Resolution Artwork Synthesis

This paper proposes a series of new approaches to improve Generative Adversarial Network (GAN) for conditional image synthesis and we name the proposed model as ArtGAN. One of the key innovation of ArtGAN is that, the gradient of the loss function w.r. t. the label (randomly assigned to each generated image) is back-propagated from the categorical discriminator to the generator. With the feedback from the label information, the generator is able to learn more efficiently and generate image with better quality. Inspired by recent works, an autoencoder is incorporated into the categorical discriminator for additional complementary information. Last but not least, we introduce a novel strategy to improve the image quality. In the experiments, we evaluate ArtGAN on CIFAR-10 and STL-10 via ablation studies. The empirical results showed that our proposed model outperforms the state-of-the-art results on CIFAR-10 in terms of Inception score. Qualitatively, we demonstrate that ArtGAN is able to generate plausible-looking images on Oxford-102 and CUB-200, as well as able to draw realistic artworks based on style, artist, and genre. The source code and models are available at: this https URL
Inequity aversion resolves intertemporal social dilemmas

Groups of humans are often able to find ways to cooperate with one another in complex, temporally extended social dilemmas. Models based on behavioral economics are only able to explain this phenomenon for unrealistic stateless matrix games. Recently, multi-agent reinforcement learning has been applied to generalize social dilemma problems to temporally and spatially extended Markov games. However, this has not yet generated an agent that learns to cooperate in social dilemmas as humans do. A key insight is that many, but not all, human individuals have inequity averse social preferences. This promotes a particular resolution of the matrix game social dilemma wherein inequity-averse individuals are personally pro-social and punish defectors. Here we extend this idea to Markov games and show that it promotes cooperation in several types of sequential social dilemma, via a profitable interaction with policy learnability. In particular, we find that inequity aversion improves temporal credit assignment for the important class of intertemporal social dilemmas. These results help explain how large-scale cooperation may emerge and persist.