Reviewerly/exHarmony huggingface dataset:
Using Natural Language for Reward Shaping in Reinforcement Learning

Recent reinforcement learning (RL) approaches have shown strong performance in complex domains such as Atari games, but are often highly sample inefficient. A common approach to reduce interaction time with the environment is to use reward shaping, which involves carefully designing reward functions that provide the agent intermediate rewards for progress towards the goal. However, designing appropriate shaping rewards is known to be difficult as well as time-consuming. In this work, we address this problem by using natural language instructions to perform reward shaping. We propose the LanguagE-Action Reward Network (LEARN), a framework that maps free-form natural language instructions to intermediate rewards based on actions taken by the agent. These intermediate language-based rewards can seamlessly be integrated into any standard reinforcement learning algorithm. We experiment with Montezuma's Revenge from the Atari Learning Environment, a popular benchmark in RL. Our experiments on a diverse set of 15 tasks demonstrate that, for the same number of interactions with the environment, language-based rewards lead to successful completion of the task 60% more often on average, compared to learning without language.
Coupled Generative Adversarial Network for Continuous Fine-Grained Action Segmentation

We propose a novel conditional GAN (cGAN) model for continuous fine-grained human action segmentation, that utilises multi-modal data and learned scene context information. The proposed approach utilises two GANs: termed Action GAN and Auxiliary GAN, where the Action GAN is trained to operate over the current RGB frame while the Auxiliary GAN utilises supplementary information such as depth or optical flow. The goal of both GANs is to generate similar 'action codes', a vector representation of the current action. To facilitate this process a context extractor that incorporates data and recent outputs from both modes is used to extract context information to aids recognition performance. The result is a recurrent GAN architecture which learns a task specific loss function from multiple feature modalities. Extensive evaluations on variants of the proposed model to show the importance of utilising different streams of information such as context and auxiliary information in the proposed network; and show that our model is capable of outperforming state-of-the-art methods for three widely used datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities, comprising both static and dynamic camera settings.
Drones' Cryptanalysis - Smashing Cryptography with a Flicker

In an "open skies" era in which drones fly among us, a new question arises: how can we tell whether a passing drone is being used by its operator for a legitimate purpose (e.g., delivering pizza) or an illegitimate purpose (e.g., taking a peek at a person showering in his/her own house)? Over the years, many methods have been suggested to detect the presence of a drone in a specific location, however since populated areas are no longer off limits for drone flights, the previously suggested methods for detecting a privacy invasion attack are irrelevant. In this paper, we present a new method that can detect whether a specific POI (point of interest) is being video streamed by a drone. We show that applying a periodic physical stimulus on a target/victim being video streamed by a drone causes a watermark to be added to the encrypted video traffic that is sent from the drone to its operator and how this watermark can be detected using interception. Based on this method, we present an algorithm for detecting a privacy invasion attack. We analyze the performance of our algorithm using four commercial drones (DJI Mavic Air, Parrot Bebop 2, DJI Spark, and DJI Mavic Pro). We show how our method can be used to (1) determine whether a detected FPV (first-person view) channel is being used to video stream a POI by a drone, and (2) locate a spying drone in space; we also demonstrate how the physical stimulus can be applied covertly. In addition, we present a classification algorithm that differentiates FPV transmissions from other suspicious radio transmissions. We implement this algorithm in a new invasion attack detection system which we evaluate in two use cases (when the victim is inside his/her house and when the victim is being tracked by a drone while driving his/her car); our evaluation shows that a privacy invasion attack can be detected by our system in about 2-3 seconds.
Deep neural-kernel blocks

This paper introduces novel deep architectures using the hybrid neural-kernel core model as the first building block. The proposed models follow a combination of a neural networks based architecture and a kernel based model enriched with pooling layers. In particular, in this context three kernel blocks with average, maxout and convolutional pooling layers are introduced and examined. We start with a simple merging layer which averages the output of the previous representation layers. The maxout layer on the other hand triggers competition among different representations of the input. Thanks to this pooling layer, not only the dimensionality of the output of multi-scale representations is reduced but also multiple sub-networks are formed within the same model. In the same context, the pointwise convolutional layer is also employed with the aim of projecting the multi-scale representations onto a new space. Experimental results show an improvement over the core deep hybrid model as well as kernel based models on several real-life datasets.
Creation of Synthetic Data with Conditional Generative Adversarial Networks

The generation of synthetic data is becoming a fundamental task in the daily life of any organization due to new protection data laws that are emerging. Generative Adversarial Networks (GANs) and its variants have attracted many researchers in their research work due to its elegant theoretical basis and its great performance in the generation of new data [19]. The goal of synthetic data generation is to create data that will perform similarly to the original dataset for many analysis tasks, such as classification. The problem of GANs is that in a classification problem, GANs do not take class labels into account when generating new data, they treat it as another attribute. This research work has focused on the creation of new synthetic data from the “Default of Credit Card Clients” dataset with a Conditional Generative Adversarial Network (CGAN). CGANs are an extension of GANs where the class label is taken into account when the new data is generated. The performance of our results has been measured by comparing the results obtained with classification algorithms, both in the original dataset and in the data generated.
Functional Variational Bayesian Neural Networks

Variational Bayesian neural networks (BNNs) perform variational inference over weights, but it is difficult to specify meaningful priors and approximate posteriors in a high-dimensional weight space. We introduce functional variational Bayesian neural networks (fBNNs), which maximize an Evidence Lower BOund (ELBO) defined directly on stochastic processes, i.e. distributions over functions. We prove that the KL divergence between stochastic processes equals the supremum of marginal KL divergences over all finite sets of inputs. Based on this, we introduce a practical training objective which approximates the functional ELBO using finite measurement sets and the spectral Stein gradient estimator. With fBNNs, we can specify priors entailing rich structures, including Gaussian processes and implicit stochastic processes. Empirically, we find fBNNs extrapolate well using various structured priors, provide reliable uncertainty estimates, and scale to large datasets.
Collaborative Recommendations using Hierarchical Clustering based on K-d Trees and Quadtrees

Majority of the e-commerce sites implement Recommender Systems (RS) to help users navigate through the large search space and assist their decision making process by suggesting products that the user may like. Collaborative Filtering (CF) is the most successful and widely used algorithm in the domain of RS. However, due to the exponential growth of the web in terms of both content and number of users, CF based RS face serious scalability issues. To alleviate this problem, we propose a clustering based CF approach using two hierarchical space partitioning data structures — K-d tree and Quadtree. We cluster or partition the users’ space of the system on the basis of user location and then use the resultant clusters for predicting ratings of a target user. Since the CF based recommendation algorithm is applied separately to the clusters and not on the entire rating data, it helps in bringing down the runtime of the algorithm substantially. We further measure spatial autocorrelation indices in the clusters to justify our clustering method. However, our objective is not only to reduce the runtime but also to maintain an acceptable recommendation quality. This requirement is rightly addressed by the proposed method which assures scalability, by processing very large datasets using the same computing resource. Moreover our proposed clustering scheme is oblivious of the underlying CF algorithm. Results from the extensive experiments conducted, show that our hierarchical clustering based recommendation approach reduces runtime of the standard CF algorithms by about 88%, 82%, 79% and 85% for MovieLens-100K, MovieLens-1M, Book-Crossing and TripAdvisor data respectively, while maintaining good recommendation quality.
Temporal Data Representation and Querying Based on RDF

With the explosive growth of temporal data, how to query and manage temporal data has become an important research issue.Resource description framework (RDF), as the standard data and knowledge description language of the semantic web, has been widely used to represent various domain data.Aiming at the representation and querying of temporal data, this paper proposes a temporal data representation model based on RDF and its corresponding querying method.First, a representation model called RDFt is proposed, which can represent temporal data with both the time information and the update count information, and the syntax and semantics of the RDFt model are given.Then, we propose a query language called SPARQL[t] for RDFt, and we give the query syntax and operations of SPARQL[t] in detail.In addition, a querying transformation algorithm from SPARQL[t] to SPARQL is proposed, in order to achieve compatibility with the existing RDF query engines.Finally, we implemented a prototype system that can support RDFt temporal data representation and querying, and the case studies and experimental results verify the feasibility of the proposed approach.
Beyond Integrated Development Environments: Adding Context to Software Development

Software developers create amazing software that is constantly changing the world in which we live: Navigation systems make it easy to find hard to find locations, mobile phones help diagnose health conditions and communication with almost anyone anywhere is virtually effortless. To create these amazing systems, software developers use tooling that is stuck in the past. Integrated development environments enable tools to work together more seamlessly, but remain oriented around the static structure (i.e., files, classes, etc.) of software. This paper proposes that a focus on the study and implementation of context could enable software tooling to take a substantial step forward, helping software developers to work more effectively. We delve into initial ideas of what context in software development might be and how context might support creating tools that augment human intelligence, allowing developers to better focus on the complex problems they face as they build amazing software.
Engaging consumers in mobile instant messaging: the role of cute branded emoticons

Purpose With the rapid surge of mobile marketing, an increasing number of brands have launched branded emoticons in an attempt to build brand relationships with consumers. Despite the apparent promise of branded emoticon usage, there is only limited academic research on branded emoticons. This paper aims to build on impression management theory and the conceptualization of cuteness to investigate how the effect of cuteness in branded emoticon design influences perceived playfulness in mobile instant messaging (MIM) interaction and the creation of brand engagement in self-concept. Design/methodology/approach Consumers with usage experience of branded emoticons in MIM apps were recruited to complete an online survey. Partial least squares structural equation modelling was used to analyze the data. Findings Two facets of cuteness – kindchenschema cuteness and whimsical cuteness – can project a favourable social image to consumers that facilitates playfulness in social interaction and enhances brand engagement in self-concept, which leads to their willingness to purchase the brand and stickiness to the MIM apps. Originality/value The popularity of branded emoticons represents a new form of social interaction and an innovative way to build brand relationships. The present study is the first to examine the design aspects of branded emoticons and highlights that the cuteness of a branded emoticon may be a crucial factor in engaging consumers in MIM.
Opportunities for examining the natural nanogeochemical environment using recent advances in nanoparticle analysis

Current approaches provide a view of natural nanomaterials while emerging technologies open the door to further studies of nanogeoscience.
Closing Gaps of a Quantum Advantage with Short-Time Hamiltonian Dynamics

Demonstrating a quantum computational speed-up is a crucial milestone for near-term quantum technology. Recently, sampling protocols for quantum simulators have been proposed that have the potential to show such a quantum advantage, based on commonly made assumptions. The key challenge in the theoretical analysis of this scheme---as of other comparable schemes such as boson sampling---is to lessen the assumptions and close the theoretical loopholes, replacing them by rigorous arguments. In this work, we prove two open conjectures for a simple sampling protocol that is based on the continuous time evolution of a translation-invariant Ising Hamiltonian: anticoncentration of the generated probability distributions and average-case hardness of exactly evaluating those probabilities. The latter is proven building upon recently developed techniques for random circuit sampling. For the former, we exploit the insight that approximate 2-designs for the unitary group admit anticoncentration. We then develop new techniques to prove that the 2D time evolution of the protocol gives rise to approximate 2-designs. Our work provides the strongest theoretical evidence to date that Hamiltonian quantum simulators are classically intractable.
Random Path Selection for Continual Learning

Incremental life-long learning is a main challenge towards the long-standing goal of Artificial General Intelligence. In real-life settings, learning tasks arrive in a sequence and machine learning models must continually learn to increment already acquired knowledge. The existing incremental learning approaches fall well below the state-of-the-art cumulative models that use all training classes at once. In this paper, we propose a random path selection algorithm, called RPS-Net, that progressively chooses optimal paths for the new tasks while encouraging parameter sharing and reuse. Our approach avoids the overhead introduced by computationally expensive evolutionary and reinforcement learning based path selection strategies while achieving considerable performance gains. As an added novelty, the proposed model integrates knowledge distillation and retrospection along with the path selection strategy to overcome catastrophic forgetting. In order to maintain an equilibrium between previous and newly acquired knowledge, we propose a simple controller to dynamically balance the model plasticity. Through extensive experiments, we demonstrate that the proposed method surpasses the state-of-the-art performance on incremental learning and by utilizing parallel computation this method can run in constant time with nearly the same efficiency as a conventional deep convolutional neural network.
Metalearned Neural Memory

We augment recurrent neural networks with an external memory mechanism that builds upon recent progress in metalearning. We conceptualize this memory as a rapidly adaptable function that we parameterize as a deep neural network. Reading from the neural memory function amounts to pushing an input (the key vector) through the function to produce an output (the value vector). Writing to memory means changing the function; specifically, updating the parameters of the neural network to encode desired information. We leverage training and algorithmic techniques from metalearning to update the neural memory function in one shot. The proposed memory-augmented model achieves strong performance on a variety of learning problems, from supervised question answering to reinforcement learning.
Light Polarization-Controlled Conversion of Ultrafast Coherent–Incoherent Exciton Dynamics in Few-Layer ReS<sub>2</sub>

Coherent light-matter interaction can transiently modulate the quantum states of matter under nonresonant laser excitation. This phenomenon, called the optical Stark effect, is one of the promising candidates for realizing ultrafast optical switches. However, the ultrafast modulations induced by the coherent light-matter interactions usually involve unwanted incoherent responses, significantly reducing the overall operation speed. Here, by using ultrafast pump-probe spectroscopy, we suppress the incoherent response and modulate the coherent-to-incoherent ratio in the two-dimensional semiconductor ReS2. We selectively convert the coherent and incoherent responses of an anisotropic exciton state by solely using photon polarizations, improving the control ratio by 3 orders of magnitude. The efficient modulation was enabled by transient superpositions of differential spectra from two nondegenerate exciton states due to the light polarization dependencies. This work provides a valuable contribution toward realizing ideal ultrafast optical switches.
Cold Case: The Lost MNIST Digits

Although the popular MNIST dataset \citep{mnist} is derived from the NIST database \citep{nist-sd19}, precise processing steps of this derivation have been lost to time. We propose a reconstruction that is accurate enough to serve as a replacement for the MNIST dataset, with insignificant changes in accuracy. We trace each MNIST digit to its NIST source and its rich metadata such as writer identifier, partition identifier, etc. We also reconstruct the complete MNIST test set with 60,000 samples instead of the usual 10,000. Since the balance 50,000 were never distributed, they enable us to investigate the impact of twenty-five years of MNIST experiments on the reported testing performances. Our results unambiguously confirm the trends observed by \citet{recht2018cifar,recht2019imagenet}: although the misclassification rates are slightly off, classifier ordering and model selection remain broadly reliable. We attribute this phenomenon to the pairing benefits of comparing classifiers on the same digits.
An Overview of FPGA Based Deep Learning Accelerators: Challenges and Opportunities

With the rapid development of in-depth learning, neural network and deep learning algorithms have been widely used in various fields, e.g., image, video and voice processing. However, the neural network model is getting larger and larger, which is expressed in the calculation of model parameters. Although a wealth of existing efforts on GPU platforms currently used by researchers for improving computing performance, dedicated hardware solutions are essential and emerging to provide advantages over pure software solutions. In this paper, we systematically investigate the neural network accelerator based on Field-Programmable Gate Array(FPGA). Specifically, we respectively review the accelerators designed for specific problems, specific algorithms, algorithm features, and general templates. We also compared the design and implementation of the accelerator based on FPGA under different devices and network models and compared it with the versions of CPU and GPU. Finally, we present to discuss the advantages and disadvantages of accelerators on FPGA platforms and to further explore the opportunities for future research.
A deep learning virtual instrument for monitoring extreme UV solar spectral irradiance

Measurements of the extreme ultraviolet (EUV) solar spectral irradiance (SSI) are essential for understanding drivers of space weather effects, such as radio blackouts, and aerodynamic drag on satellites during periods of enhanced solar activity. In this paper, we show how to learn a mapping from EUV narrowband images to spectral irradiance measurements using data from NASA's Solar Dynamics Observatory obtained between 2010 to 2014. We describe a protocol and baselines for measuring the performance of models. Our best performing machine learning (ML) model based on convolutional neural networks (CNNs) outperforms other ML models, and a differential emission measure (DEM) based approach, yielding average relative errors of under 4.6% (maximum error over emission lines) and more typically 1.6% (median). We also provide evidence that the proposed method is solving this mapping in a way that makes physical sense and by paying attention to magnetic structures known to drive EUV SSI variability.
Semi-supervised mp-MRI data synthesis with StitchLayer and auxiliary distance maximization

The availability of a large amount of annotated data is critical for many medical image analysis applications, in particular for those relying on deep learning methods which are known to be data-hungry. However, annotated medical data, especially multimodal data, is often scarce and costly to obtain. In this paper, we address the problem of synthesizing multi-parameter magnetic resonance imaging data (i.e. mp-MRI), which typically consists of Apparent Diffusion Coefficient (ADC) and T2-weighted (T2w) images, containing clinically significant (CS) prostate cancer (PCa) via semi-supervised learning and adversarial learning. Specifically, our synthesizer generates mp-MRI data in a sequential manner: first utilizing a decoder to generate an ADC map from a 128-d latent vector, followed by translating the ADC to the T2w image via U-Net. The synthesizer is trained in a semi-supervised manner. In the supervised training process, a limited amount of paired ADC-T2w images and the corresponding ADC encodings are provided and the synthesizer learns the paired relationship by explicitly minimizing the reconstruction losses between synthetic and real images. To avoid overfitting limited ADC encodings, an unlimited amount of random latent vectors and unpaired ADC-T2w Images are utilized in the unsupervised training process for learning the marginal image distributions of real images. To improve the robustness for training the synthesizer, we decompose the difficult task of generating full-size images into several simpler tasks which generate sub-images only. A StitchLayer is then employed to seamlessly fuse sub-images together in an interlaced manner into a full-size image. In addition, to enforce the synthetic images to indeed contain distinguishable CS PCa lesions, we propose to also maximize an auxiliary distance of Jensen-Shannon divergence (JSD) between CS and nonCS images. Experimental results show that our method can effectively synthesize a large variety of mp-MRI images which contain meaningful CS PCa lesions, display a good visual quality and have the correct paired relationship between the two modalities of a pair. Compared to the state-of-the-art methods based on adversarial learning (Liu and Tuzel, 2016; Costa et al., 2017), our method achieves a significant improvement in terms of both visual quality and several popular quantitative evaluation metrics.