Reviewerly/exHarmony huggingface dataset:
Practical Deep Raw Image Denoising on Mobile Devices

Deep learning-based image denoising approaches have been extensively studied in recent years, prevailing in many public benchmark datasets. However, the stat-of-the-art networks are computationally too expensive to be directly applied on mobile devices. In this work, we propose a light-weight, efficient neural network-based raw image denoiser that runs smoothly on mainstream mobile devices, and produces high quality denoising results. Our key insights are twofold: (1) by measuring and estimating sensor noise level, a smaller network trained on synthetic sensor-specific data can out-perform larger ones trained on general data; (2) the large noise level variation under different ISO settings can be removed by a novel k-Sigma Transform, allowing a small network to efficiently handle a wide range of noise levels. We conduct extensive experiments to demonstrate the efficiency and accuracy of our approach. Our proposed mobile-friendly denoising model runs at $$\sim $$ 70 ms per megapixel on Qualcomm Snapdragon 855 chipset, and it is the basis of the night shot feature of several flagship smartphones released in 2019.
Design of multirotor aerial vehicles: A taxonomy based on input allocation

This paper reviews the effect of multirotor aerial vehicle designs on their abilities in terms of tasks and system properties. We propose a general taxonomy to characterize and describe multirotor aerial vehicles and their designs, which we apply exhaustively on the vast literature available. Thanks to the systematic characterization of the designs, we exhibit groups of designs having the same abilities in terms of achievable tasks and system properties. In particular, we organize the literature review based on the number of atomic actuation units and we discuss global properties arising from their choice and spatial distribution in the mechanical designs. Finally, we provide a discussion on the common traits of the designs found in the literature and the main open and future problems.
Novel encryption for color images using fractional-order hyperchaotic system

The fractional-order functions show better performance than their corresponding integer-order functions in various image processing applications. In this paper, the authors propose a novel utilization of fractional-order chaotic systems in color image encryption. The 4D hyperchaotic Chen system of fractional-order combined with the Fibonacci Q-matrix. The proposed encryption algorithm consists of three steps: in step#1, the input image decomposed into the primary color channels, R, G, & B. The confusion and diffusion operations are performed for each channel independently. In step#2, the 4D hyperchaotic Chen system of fractional orders generates random numbers to permit pixel positions. In step#3, we split the permitted image into 2×2 blocks where the Fibonacci Q-matrix diffused each of them. Experiments performed where the obtained results ensure the efficiency of the proposed encryption algorithm and its ability to resist attacks.
Dialogue Generation in Character-Based Interactive Storytelling

In recent years, there has been significant progress in developing Interactive Storytelling systems, in particular in terms of the underlying AI techniques. There has been an emerging consensus on the AI approach, in particular the use of planning systems. However, these have concentrated on the generation of a sequence of narrative actions, staged through the behaviour of virtual actors. In order to achieve the long-term objective of implementing interactive media that would reproduce the aesthetic qualities of traditional films, most interactive storytelling systems are still missing the ability to generate dialogues between characters. We describe an extension of our Interactive Storytelling approach which integrates dialogue generation within narrative situations. Our unit of generation is the dialogue act (pair of utterances) and one main objective is to reproduce realistic dialogue phenomena based on implicit forms of expression. The emphasis is also on adopting a unified approach relating narrative representations to the linguistic form of the generated utterances. From a representation of active narrative goals and the emotional affinity between characters, our system is able to derive a set of parameters governing various aspects of the linguistic form to be generated. We illustrate this approach on several implemented examples.
Quaternion Convolutional Neural Network for Color Image Classification and Forensics

The convolutional neural network is widely popular for solving the problems of color image feature extraction. However, in the general network, the interrelationship of the color image channels is neglected. Therefore, a novel quaternion convolutional neural network (QCNN) is proposed in this paper, which always treats color triples as a whole to avoid information loss. The original quaternion convolution operation is presented and constructed to fully mix the information of color channels. The quaternion batch normalization and pooling operations are derived and designed in quaternion domain to further ensure the integrity of color information. Meanwhile, the knowledge of the attention mechanism is incorporated to boost the performance of the proposed QCNN. The experiments demonstrate that the proposed model is more efficient than the traditional convolutional neural network and another QCNN with the same structure, and has better performance in color image classification and color image forensics.
Quantum key distribution with flawed and leaky sources

Abstract In theory, quantum key distribution (QKD) allows secure communications between two parties based on physical laws. However, most of the security proofs of QKD today make unrealistic assumptions and neglect many relevant device imperfections. As a result, they cannot guarantee the security of the practical implementations. Recently, the loss-tolerant protocol (K. Tamaki et al., Phys. Rev. A, 90, 052314, 2014) was proposed to make QKD robust against state preparation flaws. This protocol relies on the emission of qubit systems, which, unfortunately, is difficult to achieve in practice. In this work, we remove such qubit assumption and generalise the loss-tolerant protocol to accommodate multiple optical modes in the emitted signals. These multiple optical modes could arise, e.g., from Trojan horse attacks and/or device imperfections. Our security proof determines some dominant device parameter regimes needed for achieving secure communication and, therefore, it can serve as a guideline to characterise QKD transmitters. Furthermore, we compare our approach with that of H.-K. Lo et al. (Quantum Inf. Comput., 7, 431–458, 2007) and identify which method provides the highest secret key generation rate as a function of the device imperfections. Our work constitutes an important step towards the best practical and secure implementation for QKD.
Large-Scale Representation Learning from Visually Grounded Untranscribed Speech

Systems that can associate images with their spoken audio captions are an important step towards visually grounded language learning. We describe a scalable method to automatically generate diverse audio for image captioning datasets. This supports pretraining deep networks for encoding both audio and images, which we do via a dual encoder that learns to align latent representations from both modalities. We show that a masked margin softmax loss for such models is superior to the standard triplet loss. We fine-tune these models on the Flickr8k Audio Captions Corpus and obtain state-of-the-art results—improving recall in the top 10 from 29.6% to 49.5%. We also obtain human ratings on retrieval outputs to better assess the impact of incidentally matching image-caption pairs that were not associated in the data, finding that automatic evaluation substantially underestimates the quality of the retrieved results.
MMM: Multi-Stage Multi-Task Learning for Multi-Choice Reading Comprehension

Machine Reading Comprehension (MRC) for question answering (QA), which aims to answer a question given the relevant context passages, is an important way to test the ability of intelligence systems to understand human language. Multiple-Choice QA (MCQA) is one of the most difficult tasks in MRC because it often requires more advanced reading comprehension skills such as logical reasoning, summarization, and arithmetic operations, compared to the extractive counterpart where answers are usually spans of text within given passages. Moreover, most existing MCQA datasets are small in size, making the task even harder. We introduce MMM, a Multi-stage Multi-task learning framework for Multi-choice reading comprehension. Our method involves two sequential stages: coarse-tuning stage using out-of-domain datasets and multi-task learning stage using a larger in-domain dataset to help model generalize better with limited data. Furthermore, we propose a novel multi-step attention network (MAN) as the top-level classifier for this task. We demonstrate MMM significantly advances the state-of-the-art on four representative MCQA datasets.
Local distribution-based adaptive minority oversampling for imbalanced data classification

Imbalanced data classification, as a challenging task, has drawn a significant interest in numerous scientific areas. One popular strategy to balance the instance quantities between two classes is oversampling via generating synthetic instances. However, it still suffers from two key issues: where and how many synthetic instances should be generated. In this paper, we propose a Local distribution-based Adaptive Minority Oversampling method (LAMO) to deal with the imbalance classification problem. LAMO first identifies the informative borderline minority instances as sampling seeds according to their neighbors and the corresponding class distribution. Then, LAMO captures the local distribution of each seed according to its Euclidean distances from the nearest majority instance and nearest minority instance.Finally, LAMO generates synthetic instances around seeds via a Gaussian Mixture Model (GMM). For each component of GMM, the mixing coefficient and bandwidth are adaptively set with the aid of seeds’ local distribution. Extensive experiments have been conducted on both simulated and real data sets under varying the imbalance ratio and data size. By comparing with the state-of-the-art oversampling methods, the proposed LAMO obtains promising results in terms of several widely used evaluation metrics.
Reinforcement Learning with Quantum Variational Circuit

The development of quantum computational techniques has advanced greatly in recent years, parallel to the advancements in techniques for deep reinforcement learning. This work explores the potential for quantum computing to facilitate reinforcement learning problems. Quantum computing approaches offer important potential improvements in time and space complexity over traditional algorithms because of its ability to exploit the quantum phenomena of superposition and entanglement. Specifically, we investigate the use of quantum variational circuits, a form of quantum machine learning. We present our techniques for encoding classical data for a quantum variational circuit, we further explore pure and hybrid quantum algorithms for DQN and Double DQN. Our results indicate both hybrid and pure quantum variational circuit have the ability to solve reinforcement learning tasks with a smaller parameter space. These comparison are conducted with two OpenAI Gym environments: CartPole and Blackjack, The success of this work is indicative of a strong future relationship between quantum machine learning and deep reinforcement learning.
Experimental realization of 105-qubit random access quantum memory

Abstract Random access memory is an indispensable device for classical information technology. Analog to this, for quantum information technology, it is desirable to have a random access quantum memory with many memory cells and programmable access to each cell. We report an experiment that realizes a random access quantum memory of 105 qubits carried by 210 memory cells in a macroscopic atomic ensemble. We demonstrate storage of optical qubits into these memory cells and their read out at programmable times by arbitrary orders with fidelities exceeding any classical bound. Experimental realization of a random access quantum memory with many memory cells and programmable control of its write-in and read out makes an important step for its application in quantum communication, networking, and computation.
Cross-modal Memory Networks for Radiology Report Generation

Medical imaging plays a significant role in clinical practice of medical diagnosis, where the text reports of the images are essential in understanding them and facilitating later treatments.By generating the reports automatically, it is beneficial to help lighten the burden of radiologists and significantly promote clinical automation, which already attracts much attention in applying artificial intelligence to medical domain.Previous studies mainly follow the encoder-decoder paradigm and focus on the aspect of text generation, with few studies considering the importance of cross-modal mappings and explicitly exploit such mappings to facilitate radiology report generation.In this paper, we propose a cross-modal memory networks (CMN) to enhance the encoderdecoder framework for radiology report generation, where a shared memory is designed to record the alignment between images and texts so as to facilitate the interaction and generation across modalities.Experimental results illustrate the effectiveness of our proposed model, where state-of-the-art performance is achieved on two widely used benchmark datasets, i.e., IU X-Ray and MIMIC-CXR.Further analyses also prove that our model is able to better align information from radiology images and texts so as to help generating more accurate reports in terms of clinical indicators.
Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction Using Patch-Based Graph Convolutional Networks

Cancer prognostication is a challenging task in computational pathology that requires context-aware representations of histology features to adequately infer patient survival. Despite the advancements made in weakly-supervised deep learning, many approaches are not context-aware and are unable to model important morphological feature interactions between cell identities and tissue types that are prognostic for patient survival. In this work, we present Patch-GCN, a context-aware, spatially-resolved patch-based graph convolutional network that hierarchically aggregates instance-level histology features to model local- and global-level topological structures in the tumor microenvironment. We validate Patch-GCN with 4,370 gigapixel WSIs across five different cancer types from the Cancer Genome Atlas (TCGA), and demonstrate that Patch-GCN outperforms all prior weakly-supervised approaches by 3.58–9.46%. Our code and corresponding models are publicly available at https://github.com/mahmoodlab/Patch-GCN .
Coordinated Multi-Robot Exploration Under Communication Constraints Using Decentralized Markov Decision Processes

Recent works on multi-agent sequential decision making using decentralized partially observable Markov decision processes have been concerned with interaction-oriented resolution techniques and provide promising results. These techniques take advantage of local interactions and coordination. In this paper, we propose an approach based on an interaction-oriented resolution of decentralized decision makers. To this end, distributed value functions (DVF) have been used by decoupling the multi-agent problem into a set of individual agent problems. However existing DVF techniques assume permanent and free communication between the agents. In this paper, we extend the DVF methodology to address full local observability, limited share of information and communication breaks. We apply our new DVF in a real-world application consisting of multi-robot exploration where each robot computes locally a strategy that minimizes the interactions between the robots and maximizes the space coverage of the team even under communication constraints. Our technique has been implemented and evaluated in simulation and in real-world scenarios during a robotic challenge for the exploration and mapping of an unknown environment. Experimental results from real-world scenarios and from the challenge are given where our system was vice-champion.
Batch denoising of ESPI fringe patterns based on convolutional neural network

The denoising of electronic speckle pattern interferometry (ESPI) fringe patterns is a key step in the application of ESPI. In this paper, we propose a method for batch denoising of ESPI fringe patterns based on a convolution neural network (CNN). In the proposed method, the network is first trained by our training dataset, which consists of the noisy ESPI fringe patterns and the corresponding noise-free images. We propose a new computer-simulated method of ESPI fringe patterns to create our training dataset. After training, the other multi-frame ESPI fringe patterns to be processed are fed to the trained network simultaneously, and the corresponding denoising images can be obtained in batches. We demonstrate the performance of the proposed method via application to 50 computer-simulated ESPI fringe patterns and three groups of experimentally obtained ESPI fringe patterns. The experimental results show that our method can obtain desired results even when the quality of ESPI fringe images is considerably low because of variable density, high noise, and low contrast, and our method can denoise multi-frame fringe patterns simultaneously. Moreover, we use the computer-simulated ESPI fringe patterns to train the network; after training, the trained network can be used to denoise either computer-simulated ESPI fringe patterns or the experimentally obtained ESPI fringe patterns. The proposed method is especially suitable for processing a large number of ESPI fringe patterns.
A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation

Recent neural language generation systems often hallucinate contents (i.e., producing irrelevant or contradicted facts), especially when trained on loosely corresponding pairs of the input structure and text.To mitigate this issue, we propose to integrate a language understanding module for data refinement with selftraining iterations to effectively induce strong equivalence between the input data and the paired text.Experiments on the E2E challenge dataset show that our proposed framework can reduce more than 50% relative unaligned noise from the original data-text pairs.A vanilla sequence-to-sequence neural NLG model trained on the refined data has improved on content correctness compared with the current state-of-the-art ensemble generator.
The design and operation of cloudlab

Given the highly empirical nature of research in cloud computing, networked systems, and related fields, testbeds play an important role in the research ecosystem. In this paper, we cover one such facility, CloudLab, which supports systems research by providing raw access to programmable hardware, enabling research at large scales, and creating a shared platform for repeatable research.We present our experiences designing CloudLab and operating it for four years, serving nearly 4,000 users who have run over 79,000 experiments on 2,250 servers, switches, and other pieces of datacenter equipment. From this experience, we draw lessons organized around two themes. The first set comes from analysis of data regarding the use of CloudLab: how users interact with it, what they use it for, and the implications for facility design and operation. Our second set of lessons comes from looking at the ways that algorithms used under the hood, such as resource allocation, have important-- and sometimes unexpected--effects on user experience and behavior. These lessons can be of value to the designers and operators of IaaS facilities in general, systems testbeds in particular, and users who have a stake in understanding how these systems are built.
CounterFactual Regression with Importance Sampling Weights

Perhaps the most pressing concern of a patient diagnosed with cancer is her life expectancy under various treatment options. For a binary-treatment case, this translates into estimating the difference between the outcomes (e.g., survival time) of the two available treatment options – i.e., her Individual Treatment Effect (ITE). This is especially challenging to estimate from observational data, as that data has selection bias: the treatment assigned to a patient depends on that patient's attributes. In this work, we borrow ideas from domain adaptation to address the distributional shift between the source (outcome of the administered treatment, appearing in the observed training data) and target (outcome of the alternative treatment) that exists due to selection bias. We propose a context-aware importance sampling re-weighing scheme, built on top of a representation learning module, for estimating ITEs. Empirical results on two publicly available benchmarks demonstrate that the proposed method significantly outperforms state-of-the-art.