Reviewerly/exHarmony huggingface dataset:
DRAU: Dual Recurrent Attention Units for Visual Question Answering

Visual Question Answering (VQA) requires AI models to comprehend data in two domains, vision and text. Current state-of-the-art models use learned attention mechanisms to extract relevant information from the input domains to answer a certain question. Thus, robust attention mechanisms are essential for powerful VQA models. In this paper, we propose a recurrent attention mechanism and show its benefits compared to the traditional convolutional approach. We perform two ablation studies to evaluate recurrent attention. First, we introduce a baseline VQA model with visual attention and test the performance difference between convolutional and recurrent attention on the VQA 2.0 dataset. Secondly, we design an architecture for VQA which utilizes dual (textual and visual) Recurrent Attention Units (RAUs). Using this model, we show the effect of all possible combinations of recurrent and convolutional dual attention. Our single model outperforms the first place winner on the VQA 2016 challenge and to the best of our knowledge, it is the second best performing single model on the VQA 1.0 dataset. Furthermore, our model noticeably improves upon the winner of the VQA 2017 challenge. Moreover, we experiment replacing attention mechanisms in state-of-the-art models with our RAUs and show increased performance.
Size-independent sample complexity of neural networks

Abstract We study the sample complexity of learning neural networks by providing new bounds on their Rademacher complexity, assuming norm constraints on the parameter matrix of each layer. Compared to previous work, these complexity bounds have improved dependence on the network depth and, under some additional assumptions, are fully independent of the network size (both depth and width). These results are derived using some novel techniques, which may be of independent interest.
Differentially Private Significance Tests for Regression Coefficients

Many data producers seek to provide users access to confidential data without unduly compromising data subjects' privacy and confidentiality. One general strategy is to require users to do analyses without seeing the confidential data; for example, analysts only get access to synthetic data or query systems that provide disclosure-protected outputs of statistical models. With synthetic data or redacted outputs, the analyst never really knows how much to trust the resulting findings. In particular, if the user did the same analysis on the confidential data, would regression coefficients of interest be statistically significant or not? We present algorithms for assessing this question that satisfy differential privacy. We describe conditions under which the algorithms should give accurate answers about statistical significance. We illustrate the properties of the proposed methods using artificial and genuine data. Supplementary materials for this article are available online.
Robust RGB-D Face Recognition Using Attribute-Aware Loss

n g, Juyo n g a n d D e n g, B ailin 2 0 2 0 .
An Improvement of Spectral Clustering via Message Passing and Density Sensitive Similarity

Spectral clustering transforms the data clustering problem into a graph-partitioning problem and classifies data points by finding the optimal sub-graphs. Traditional spectral clustering algorithms use Gaussian kernel function to construct the similarity matrix, so they are sensitive to the selection of scale parameter. In addition, they need to randomly determine the initial cluster centers at the clustering stage and the clustering performance is not stable. Therefore, this paper presents an algorithm on the basis of message passing, which makes use of a density adaptive similarity measure, describing the relations between data points and obtaining high-quality cluster centers through message passing mechanism in AP clustering. The performance of clustering is optimized by this method. The experiments show that the proposed algorithm can effectively deal with the clustering problem of multi-scale datasets. Moreover, its clustering performance is very stable, and the clustering quality is better than traditional spectral clustering algorithm and k-means algorithm.
Cynthia

It becomes an increasingly popular trend for deep neural networks with large-scale datasets to be trained in a distributed manner in the cloud. However, widely known as resource-intensive and time-consuming, distributed deep neural network (DDNN) training suffers from unpredictable performance in the cloud, due to the intricate factors of resource bottleneck, heterogeneity and the imbalance of computation and communication which eventually cause severe resource under-utilization. In this paper, we propose Cynthia, a cost-efficient cloud resource provisioning framework to provide predictable DDNN training performance and reduce the training budget. To explicitly explore the resource bottleneck and heterogeneity, Cynthia predicts the DDNN training time by leveraging a lightweight analytical performance model based on the resource consumption of workers and parameter servers. With an accurate performance prediction, Cynthia is able to optimally provision the cost-efficient cloud instances to jointly guarantee the training performance and minimize the training budget. We implement Cynthia on top of Kubernetes by launching a 56-docker cluster to train four representative DNN models. Extensive prototype experiments on Amazon EC2 demonstrate that Cynthia can provide predictable training performance while reducing the monetary cost for DDNN workloads by up to 50.6%, in comparison to state-of-the-art resource provisioning strategies, yet with acceptable runtime overhead.
Towards A Scalable DAG-based Distributed Ledger for Smart Communities

In recent years, Distributed Ledger Technology (DLT) has been playing a more and more important role in building trust and security for Internet of Things (IoT). However, the unacceptable performance of the current mainstream DLT systems such as Bitcoin can hardly meet the efficiency and scalability requirements of IoT. In this paper, we propose a scalable transactive smart homes infrastructure by leveraging a Directed Acyclic Graph (DAG) based DLT and following the separation of concerns (SOC) design principle. Based on the proposed solution, an experiment with 40 Home Nodes is conducted to prove the concepts. From the results, we find that our solution provides a high transaction speed and scalability, as well as good performance on security and micropayment which are important in IoT settings. Then, we conduct an analysis and discuss how the new system breaks out the well-known Trilemma, which claims that it is hard for a DLT platform to simultaneously reach decentralization, scalability and security. Finally, we conclude that the proposed DAG-based distributed ledger is an effective solution for building an IoT infrastructure for smart communities.
A Multi-Task Neural Approach for Emotion Attribution, Classification, and Summarization

Emotional content is a crucial ingredient in user-generated videos. However, the sparsity of emotional expressions in the videos poses an obstacle to visual emotion analysis. In this paper, we propose a new neural approach, Bi-stream Emotion Attribution-Classification Network (BEAC-Net), to solve three related emotion analysis tasks: emotion recognition, emotion attribution, and emotion-oriented summarization, in a single integrated framework. BEAC-Net has two major constituents, an attribution network and a classification network. The attribution network extracts the main emotional segment that classification should focus on in order to mitigate the sparsity issue. The classification network utilizes both the extracted segment and the original video in a bi-stream architecture. We contribute a new dataset for the emotion attribution task with human-annotated ground-truth labels for emotion segments. Experiments on two video datasets demonstrate superior performance of the proposed framework and the complementary nature of the dual classification streams.
Defining Image Memorability Using the Visual Memory Schema

Memorability of an image is a characteristic determined by the human observers' ability to remember images they have seen. Yet recent work on image memorability defines it as an intrinsic property that can be obtained independent of the observer. The current study aims to enhance our understanding and prediction of image memorability, improving upon existing approaches by incorporating the properties of cumulative human annotations. We propose a new concept called the Visual Memory Schema (VMS) referring to an organization of image components human observers share when encoding and recognizing images. The concept of VMS is operationalised by asking human observers to define memorable regions of images they were asked to remember during an episodic memory test. We then statistically assess the consistency of VMSs across observers for either correctly or incorrectly recognised images. The associations of the VMSs with eye fixations and saliency are analysed separately as well. Lastly, we adapt various deep learning architectures for the reconstruction and prediction of memorable regions in images and analyse the results when using transfer learning at the outputs of different convolutional network layers.
AI-Oriented Large-Scale Video Management for Smart City: Technologies, Standards, and Beyond

Deep learning has achieved substantial success in intelligent video analysis. To practically facilitate deep neural network models in the large-scale video analysis, there are still unprecedented challenges. Deep feature coding, instead of video coding, provides a practical solution for handling the large-scale video surveillance data. To enable interoperability in the context of deep feature coding, standardization is urgent and important. This paper envisions the future deep feature coding standard for the AI-oriented large-scale video management and discusses existing techniques, standards, and possible solutions for these open problems.
Multiple Fano interferences due to waveguide-mediated phase coupling between atoms

We examine quantum interference effects due to absorption and emission from multiple atoms coupled to a waveguide and highlight the modifications they entail with regard to single-photon transport properties. A prominent upshot of these interference phenomena is the resonant suppression of the reflection amplitude, which leads to the observation of multiple Fano minima in the reflection spectrum. Such minima determine the points at which transparency is induced in the system. By taking recourse to the real-space Hamiltonian framework, we calculate analytically the reflectivity and transmissivity for a one-dimensional waveguide that evanescently couples to a chain of equally spaced quantum emitters. The interemitter spacing relative to the wavelength of the propagating photon, leading to a waveguide-mediated ``phase coupling'' between the atoms, is found to fundamentally affect the existence of Fano minima. For a chain of $N$ atoms, the number of minima can be at most $N\ensuremath{-}1$. However, suitable choices of the phase can suppress the discernibility of the full range of roots in the reflection spectrum. A principal observation for the case of multiple emitters is the emergence of super-Gaussian characteristics close to zero detuning and, consequently, a plateau-shaped broadband spectrum in the region of high reflectivity. For a large chain size, the plateau gets transformed into a flat-topped quasirectangular profile in the frequency domain.
Processor Hardware Security Vulnerabilities and their Detection by Unique Program Execution Checking

Recent discovery of security attacks in advanced processors, known as Spectre and Meltdown, has resulted in high public alertness about security of hardware. The root cause of these attacks is information leakage across covert channels that reveal secret data without any explicit information flow between the secret and the attacker. Many sources believe that such covert channels are intrinsic to highly advanced processor architectures based on speculation and out-of-order execution, suggesting that such security risks can be avoided by staying away from high-end processors. This paper, however, shows that the problem is of wider scope: we present new classes of covert channel attacks which are possible in average-complexity processors with in-order pipelining, as they are mainstream in applications ranging from Internet-of-Things to Autonomous Systems. We present a new approach as a foundation for remedy against covert channels: while all previous attacks were found by clever thinking of human attackers, this paper presents a formal method called Unique Program Execution Checking which detects and locates vulnerabilities to covert channels systematically, including those to covert channels unknown so far.
Recurrent Convolutional Fusion for RGB-D Object Recognition

Providing robots with the ability to recognize objects like humans has always been one of the primary goals of robot vision. The introduction of RGB-D cameras has paved the way for a significant leap forward in this direction thanks to the rich information provided by these sensors. However, the robot vision community still lacks an effective method to synergically use the RGB and depth data to improve object recognition. In order to take a step in this direction, we introduce a novel end-to-end architecture for RGB-D object recognition called recurrent convolutional fusion (RCFusion). Our method generates compact and highly discriminative multi-modal features by combining RGB and depth information representing different levels of abstraction. Extensive experiments on two popular datasets, RGB-D Object Dataset and JHUIT-50, show that RCFusion significantly outperforms state-of-the-art approaches in both the object categorization and instance recognition tasks. In addition, experiments on the more challenging Object Clutter Indoor Dataset confirm the validity of our method in the presence of clutter and occlusion. The code is publicly available at: “ <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/MRLoghmani/rcfusion</uri> .”
On Perfect Clustering of High Dimension, Low Sample Size Data

Popular clustering algorithms based on usual distance functions (e.g., the Euclidean distance) often suffer in high dimension, low sample size (HDLSS) situations, where concentration of pairwise distances and violation of neighborhood structure have adverse effects on their performance. In this article, we use a new data-driven dissimilarity measure, called MADD, which takes care of these problems. MADD uses the distance concentration phenomenon to its advantage, and as a result, clustering algorithms based on MADD usually perform well for high dimensional data. We establish it using theoretical as well as numerical studies. We also address the problem of estimating the number of clusters. This is a challenging problem in cluster analysis, and several algorithms are available for it. We show that many of these existing algorithms have superior performance in high dimensions when they are constructed using MADD. We also construct a new estimator based on a penalized version of the Dunn index and prove its consistency in the HDLSS asymptotic regime. Several simulated and real data sets are analyzed to demonstrate the usefulness of MADD for cluster analysis of high dimensional data.
Oriented Spatial Transformer Network for Pedestrian Detection Using Fish-Eye Camera

Pedestrian detection using fish-eye cameras is a principal research focus in computer vision. Lack of pedestrian datasets of fish-eye images and pedestrian distortion in fish-eye images are two primary challenges. In this paper, two approaches are proposed to deal with these two challenges, respectively. On the one hand, the projective model transformation (PMT) algorithm is proposed, which can transform normal images into fish-eye images. The PMT can be applied to most of the pedestrian datasets and generates corresponding fish-eye image datasets. In this way, enough training data can be provided through the PMT. On the other hand, the oriented spatial transformer network (OSTN) is designed to rectify warped pedestrian features using CNNs, so that pedestrians in fish-eye images are easier for detectors to recognize. The OSTN can be embedded into universal deep learning based detectors easily. Moreover, the new pedestrian detector, where the OSTN is embedded, can be trained end to end. Finally, the OSTN based fish-eye pedestrian detectors can be trained using fish-eye images, which are generated using the PMT. Experiments on ETH, KITTI, Citypersons, and real pedestrian datasets show the effectiveness of the PMT and accuracy improvement of pedestrian detection in fish-eye images using the OSTN.
Denoising in the Dark: Privacy-Preserving Deep Neural Network-Based Image Denoising

Large volumes of images are being exponentially generated today, which poses high demands on the services of storage, processing, and management. To handle the explosive image growth, a natural choice nowadays is cloud computing. However, coming with the cloud-based image services is acute data privacy concerns, which has to be well addressed. In this paper, we present a secure cloud-based image service framework, which allows privacy-preserving and effective image denoising on the cloud side to produce high-quality image content, a key for assuring the quality of various image-centric applications. We resort to state-of-the-art image denoising techniques based on deep neural networks (DNNs), and show how to uniquely bridge cryptographic techniques (like lightweight secret sharing and garbled circuits) and image denoising in depth to support privacy-preserving DNN based image denoising services on the cloud. By design, the image content and the DNN model are all kept private along the whole cloud-based service flow. Our extensive empirical evaluation shows that our security design is able to achieve denoising quality comparable to that in plaintext, with high cost efficiency on the local side and practically affordable cost on the cloud side.
A General Dichotomy of Evolutionary Algorithms on Monotone Functions

It is known that the (1 + 1)-EA with mutation rate c/n optimizes every monotone function efficiently if c <; 1, and needs exponential time on some monotone functions (HOTTOPIC functions) if c ≥ 2.2. We study the same question for a large variety of algorithms, particularly for the (1 + λ)-EA, (μ + 1)-EA, (μ + 1)-GA, their “fast” counterparts, and for the (1 + (λ, λ))GA. We find that all considered mutation-based algorithms show a similar dichotomy for HOTTOPIC functions, or even for all monotone functions. For the (1 + (λ, λ))-GA, this dichotomy is in the parameter cγ, which is the expected number of bit flips in an individual after mutation and crossover, neglecting selection. For the fast algorithms, the dichotomy is in m <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> /m <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , where m <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> and m <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> are the first and second falling moment of the number of bit flips. Surprisingly, the range of efficient parameters is not affected by either population size μ nor by the offspring population size λ. The picture changes completely if crossover is allowed. The genetic algorithms (μ + 1)-GA and (μ+1)-fGA are efficient for arbitrary mutations strengths if μ is large enough.
VUNet: Dynamic Scene View Synthesis for Traversability Estimation Using an RGB Camera

We present VUNet, a novel view(VU) synthesis method for mobile robots in dynamic environments, and its application to the estimation of future traversability. Our method predicts future images for given virtual robot velocity commands using only RGB images at previous and current time steps. The future images result from applying two types of image changes to the previous and current images: first, changes caused by different camera pose. Second, changes due to the motion of the dynamic obstacles. We learn to predict these two types of changes disjointly using two novel network architectures, SNet and DNet. We combine SNet and DNet to synthesize future images that we pass to our previously presented method GONet [N. Hirose, A. Sadeghian, M. Vazquez, P. Goebel, and S. Savarese, “Gonet: A semi-supervised deep learning approach for traversability estimation,” in Proc. IEEE International Conference on Intelligent Robots and Systems, 2018, pp. 3044-3051] to estimate the traversable areas around the robot. Our quantitative and qualitative evaluation indicate that our approach for view synthesis predicts accurate future images in both static and dynamic environments. We also show that these virtual images can be used to estimate future traversability correctly. We apply our view synthesis-based traversability estimation method to two applications for assisted teleoperation.
Predicting Good Configurations for GitHub and Stack Overflow Topic Models

Software repositories contain large amounts of textual data, ranging from source code comments and issue descriptions to questions, answers, and comments on Stack Overflow. To make sense of this textual data, topic modelling is frequently used as a text-mining tool for the discovery of hidden semantic structures in text bodies. Latent Dirichlet allocation (LDA) is a commonly used topic model that aims to explain the structure of a corpus by grouping texts. LDA requires multiple parameters to work well, and there are only rough and sometimes conflicting guidelines available on how these parameters should be set. In this paper, we contribute (i) a broad study of parameters to arrive at good local optima for GitHub and Stack Overflow text corpora, (ii) an a-posteriori characterisation of text corpora related to eight programming languages, and (iii) an analysis of corpus feature importance via per-corpus LDA configuration. We find that (1) popular rules of thumb for topic modelling parameter configuration are not applicable to the corpora used in our experiments, (2) corpora sampled from GitHub and Stack Overflow have different characteristics and require different configurations to achieve good model fit, and (3) we can predict good configurations for unseen corpora reliably. These findings support researchers and practitioners in efficiently determining suitable configurations for topic modelling when analysing textual data contained in software repositories.
Single Image Deraining using a Recurrent Multi-scale Aggregation and Enhancement Network

Single image deraining is an ill-posed inverse problem due to the presence of non-uniform rain shapes, directions, and densities in images. In this paper, we propose a novel progressive single image deraining method named Recurrent Multi-scale Aggregation and Enhancement Network (ReMAEN). Differing from previous methods, ReMAEN contains a symmetric structure where recurrent blocks with shared channel attention are applied to select useful information collaboratively and remove rain streaks stage by stage. In ReMAEN, a Multi-scale Aggregation and Enhancement Block (MAEB) is constructed to detect multi-scale rain details. Moreover, to better leverage the rain details from rainy images, ReMAEN enables a symmetric skipping connection from low level to high level. Extensive experiments on synthetic and real-world datasets demonstrate that our method outperforms the state-of-the-art methods tremendously. The source code is available at https://github.com/nnUyi/ReMAEN.