Reviewerly/exHarmony huggingface dataset:
EasyAug: An Automatic Textual Data Augmentation Platform for Classification Tasks

Imbalanced data is a perennial problem that impedes the learning abilities of current machine learning-based classification models. One approach to address it is to leverage data augmentation to expand the training set. For image data, there are a number of suitable augmentation techniques that have proven effective in previous work. For textual data, however, due to the discrete units inherent in natural language, techniques that randomly perturb the signal may be ineffective. Additionally, due to the substantial discrepancy between different textual datasets (e.g., different domains), an augmentation approach that facilitates the classification on one dataset may be detrimental on another dataset. For practitioners, comparing different data augmentation techniques is non-trivial, as the corresponding methods might need to be incorporated into different system architectures, and the implementation of some approaches, such as generative models, is laborious. To address these challenges, we develop EasyAug, a data augmentation platform that provides several augmentation approaches. Users can conveniently compare the classification results and can easily choose the most suitable one for their own dataset. In addition, the system is extensible and can incorporate further augmentation approaches, such that with minimal effort a new method can comprehensively be compared with the baselines.
Similarity Search for Encrypted Images in Secure Cloud Computing

With the emergence of intelligent terminals, the Content-Based Image Retrieval (CBIR) technique has attracted much attention from many areas (i.e., cloud computing, social networking services, etc.). Although existing privacy-preserving CBIR schemes can guarantee image privacy while supporting image retrieval, these schemes still have inherent defects (i.e., low search accuracy, low search efficiency, key leakage, etc.). To address these challenging issues, in this article we provide a similarity Search for Encrypted Images in secure cloud computing (called SEI). First, the feature descriptors extracted by the Convolutional Neural Network (CNN) model are used to improve search accuracy. Next, an encrypted hierarchical index tree by using <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -means clustering based on Affinity Propagation (AP) clustering is devised, which can improve search efficiency. Then, a limited key-leakage k-Nearest Neighbor (kNN) algorithm is proposed to protect key from being completely leaked to untrusted image users. Finally, SEI is extended to further prevent image users’ search information from being exposed to the cloud server. Our formal security analysis proves that SEI can protect image privacy as well as key privacy. Our empirical experiments using a real-world dataset illustrate the higher search accuracy and efficiency of SEI.
Object Detection Based on Global-Local Saliency Constraint in Aerial Images

Different from object detection in natural image, optical remote sensing object detection is a challenging task, due to the diverse meteorological conditions, complex background, varied orientations, scale variations, etc. In this paper, to address this issue, we propose a novel object detection network (the global-local saliency constraint network, GLS-Net) that can make full use of the global semantic information and achieve more accurate oriented bounding boxes. More precisely, to improve the quality of the region proposals and bounding boxes, we first propose a saliency pyramid which combines a saliency algorithm with a feature pyramid network, to reduce the impact of complex background. Based on the saliency pyramid, we then propose a global attention module branch to enhance the semantic connection between the target and the global scenario. A fast feature fusion strategy is also used to combine the local object information based on the saliency pyramid with the global semantic information optimized by the attention mechanism. Finally, we use an angle-sensitive intersection over union (IoU) method to obtain a more accurate five-parameter representation of the oriented bounding boxes. Experiments with a publicly available object detection dataset for aerial images demonstrate that the proposed GLS-Net achieves a state-of-the-art detection performance.
Benchmark probabilistic solar forecasts: Characteristics and recommendations

We illustrate and compare commonly used benchmark, or reference, methods for probabilistic solar forecasting that researchers use to measure the performance of their proposed techniques. A thorough review of the literature indicates wide variation in the benchmarks implemented in probabilistic solar forecast studies. To promote consistent and sensible methodological comparisons, we implement and compare ten variants from six common benchmark classes at two temporal scales: intra-hourly forecasts and hourly resolution forecasts. Using open-source Surface Radiation Budget Network (SURFRAD) data from 2018, these benchmark methods are compared using proper probabilistic metrics and common diagnostic tools. Practical implementation issues, such as the impact of missing data and applicability for operational forecasting, are also discussed. We make recommendations for practitioners on the appropriate selection of benchmark methods to properly showcase state-of-the-art improvements in forecast reliability and sharpness. All code and open-source data are available on Github for reproducibility and for other researchers to apply the same benchmark methods to their own data.
Spatio-Temporal Point Process for Multiple Object Tracking

Multiple Object Tracking (MOT) focuses on modeling the relationship of detected objects among consecutive frames and merge them into different trajectories. MOT remains a challenging task as noisy and confusing detection results often hinder the final performance. Furthermore, most existing research are focusing on improving detection algorithms and association strategies. As such, we propose a novel framework that can effectively predict and mask-out the noisy and confusing detection results before associating the objects into trajectories. In particular, we formulate such "bad" detection results as a sequence of events and adopt the spatio-temporal point process}to model such events. Traditionally, the occurrence rate in a point process is characterized by an explicitly defined intensity function, which depends on the prior knowledge of some specific tasks. Thus, designing a proper model is expensive and time-consuming, with also limited ability to generalize well. To tackle this problem, we adopt the convolutional recurrent neural network (conv-RNN) to instantiate the point process, where its intensity function is automatically modeled by the training data. Furthermore, we show that our method captures both temporal and spatial evolution, which is essential in modeling events for MOT. Experimental results demonstrate notable improvements in addressing noisy and confusing detection results in MOT datasets. An improved state-of-the-art performance is achieved by incorporating our baseline MOT algorithm with the spatio-temporal point process model.
IDA-3D: Instance-Depth-Aware 3D Object Detection From Stereo Vision for Autonomous Driving

3D object detection is an important scene understanding task in autonomous driving and virtual reality. Approaches based on LiDAR technology have high performance, but LiDAR is expensive. Considering more general scenes, where there is no LiDAR data in the 3D datasets, we propose a 3D object detection approach from stereo vision which does not rely on LiDAR data either as input or as supervision in training, but solely takes RGB images with corresponding annotated 3D bounding boxes as training data. As depth estimation of object is the key factor affecting the performance of 3D object detection, we introduce an Instance-DepthAware (IDA) module which accurately predicts the depth of the 3D bounding box’s center by instance-depth awareness, disparity adaptation and matching cost reweighting. Moreover, our model is an end-to-end learning framework which does not require multiple stages or postprocessing algorithm. We provide detailed experiments on KITTI benchmark and achieve impressive improvements compared with the existing image-based methods. Our code is available at https://github.com/swords123/IDA-3D.
Adaptive Deep Metric Learning for Affective Image Retrieval and Classification

An image is worth a thousand words. Many researchers have conducted extensive studies to understand visual emotions since an increasing number of users express emotions via images and videos online. However, most existing methods based on convolutional neural networks aim to retrieve and classify affective images in a discrete label space while ignoring both the hierarchical and complex nature of emotions. On the one hand, different from concrete and isolated object concepts ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., cat and dog), a hierarchical relationship exists among emotions. On the other hand, most widely used deep methods depend on the representation from fully connected layers, which lacks the essential texture information for recognizing emotions. In this work, we address the above problems via adaptive deep metric learning. Specifically, we design an adaptive sentiment similarity loss, which is able to embed affective images considering the emotion polarity and adaptively adjust the margin between different image pairs. To effectively distinguish affective images, we further propose the sentiment vector that captures the texture information extracted from multiple convolutional layers. Finally, we develop a unified multi-task deep framework to simultaneously optimize both retrieval and classification goals. Extensive and thorough evaluations on four benchmark datasets demonstrate that the proposed framework performs favorably against the state-of-the-art methods.
GradNet Image Denoising

High-frequency regions like edges compromise the image denoising performance. In traditional hand-crafted systems, image edges/textures were regularly used to restore the frequencies in these regions. However, this practice seems to be left forgotten in the deep learning era. In this paper, we revisit this idea of using the image gradient and introduce the GradNet. Our major contribution is fusing the image gradient in the network. Specifically, the image gradient is computed from the denoised network input and is subsequently concatenated with the feature maps extracted from the shallow layers. In this step, we argue that image gradient shares intrinsically similar nature with features from the shallow layers, and thus that our fusion strategy is superior. One minor contribution in this work is proposing a gradient consistency regularization, which enforces the gradient difference of the denoised image and the clean ground-truth to be minimized. Putting the two techniques together, the proposed GradNet allows us to achieve competitive denoising accuracy on three synthetic datasets and three real-world datasets. We show through ablation studies that the two techniques are indispensable. Moreover, we verify that our system is particularly capable of removing noise from textured regions.
Learning on the Job: Online Lifelong and Continual Learning

One of the hallmarks of the human intelligence is the ability to learn continuously, accumulate the knowledge learned in the past and use the knowledge to help learn more and learn better. It is hard to imagine a truly intelligent system without this capability. This type of learning differs significantly than the classic machine learning (ML) paradigm of isolated single-task learning. Although there is already research on learning a sequence of tasks incrementally under the names of lifelong learning or continual learning, they still follow the traditional two-phase separate training and testing paradigm in learning each task. The tasks are also given by the user. This paper adds on-the-job learning to the mix to emphasize the need to learn during application (thus online) after the model has been deployed, which traditional ML cannot do. It aims to leverage the learned knowledge to discover new tasks, interact with humans and the environment, make inferences, and incrementally learn the new tasks on the fly during applications in a self-supervised and interactive manner. This is analogous to human on-the-job learning after formal training. We use chatbots and self-driving cars as examples to discuss the need, some initial work, and key challenges and opportunities in building this capability.
Symbolic Plans as High-Level Instructions for Reinforcement Learning

Reinforcement learning (RL) agents seek to maximize the cumulative reward obtained when interacting with their environment. Users define tasks or goals for RL agents by designing specialized reward functions such that maximization aligns with task satisfaction. This work explores the use of high-level symbolic action models as a framework for defining final-state goal tasks and automatically producing their corresponding reward functions. We also show how automated planning can be used to synthesize high-level plans that can guide hierarchical RL (HRL) techniques towards efficiently learning adequate policies. We provide a formal characterization of taskable RL environments and describe sufficient conditions that guarantee we can satisfy various notions of optimality (e.g., minimize total cost, maximize probability of reaching the goal). In addition, we do an empirical evaluation that shows that our approach converges to near-optimal solutions faster than standard RL and HRL methods and that it provides an effective framework for transferring learned skills across multiple tasks in a given environment.
Empirical Analysis of Multi-Task Learning for Reducing Identity Bias in Toxic Comment Detection

With the recent rise of toxicity in online conversations on social media platforms, using modern machine learning algorithms for toxic comment detection has become a central focus of many online applications. Researchers and companies have developed a variety of models to identify toxicity in online conversations, reviews, or comments with mixed successes. However, many existing approaches have learned to incorrectly associate non-toxic comments that have certain trigger-words (e.g. gay, lesbian, black, muslim) as a potential source of toxicity. In this paper, we evaluate several state-of-the-art models with the specific focus of reducing model bias towards these commonly-attacked identity groups. We propose a multi-task learning model with an attention layer that jointly learns to predict the toxicity of a comment as well as the identities present in the comments in order to reduce this bias. We then compare our model to an array of shallow and deep-learning models using metrics designed especially to test for unintended model bias within these identity groups.
Design of a Face Recognition System based on Convolutional Neural Network (CNN)

Face recognition is an important function of video surveillance systems, enabling verification and identification of people who appear in a scene often captured by a distributed network of cameras. The recognition of people from the faces in images arouses great interest in the scientific community, partly because of the application interests but also because of the challenge that this represents for artificial vision algorithms. They must be able to cope with the great variability of the aspects of the faces themselves as well as the variations of the shooting parameters (pose, lighting, haircut, expression, background, etc.). This paper aims to develop a face recognition application for a biometric system based on Convolutional Neural Networks. It proposes a structure of a Deep Learning model which allows improving the existing state-of-the-art precision and processing time.
Solving quantum statistical mechanics with variational autoregressive networks and quantum circuits

Abstract We extend the ability of an unitary quantum circuit by interfacing it with a classical autoregressive neural network. The combined model parametrizes a variational density matrix as a classical mixture of quantum pure states, where the autoregressive network generates bitstring samples as input states to the quantum circuit. We devise an efficient variational algorithm to jointly optimize the classical neural network and the quantum circuit to solve quantum statistical mechanics problems. One can obtain thermal observables such as the variational free energy, entropy, and specific heat. As a byproduct, the algorithm also gives access to low energy excitation states. We demonstrate applications of the approach to thermal properties and excitation spectra of the quantum Ising model with resources that are feasible on near-term quantum computers.
Better Document-Level Machine Translation with Bayes’ Rule

We show that Bayes’ rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents a compelling benefit because parallel documents are not always available. In our formulation, the posterior probability of a candidate translation is the product of the unconditional (prior) probability of the candidate output document and the “reverse translation probability” of translating the candidate output back into the source language. Our proposed model uses a powerful autoregressive language model as the prior on target language documents, but it assumes that each sentence is translated independently from the target to the source language. Crucially, at test time, when a source document is observed, the document language model prior induces dependencies between the translations of the source sentences in the posterior. The model’s independence assumption not only enables efficient use of available data, but it additionally admits a practical left-to-right beam-search algorithm for carrying out inference. Experiments show that our model benefits from using cross-sentence context in the language model, and it outperforms existing document translation approaches.
A Report on the 2020 Sarcasm Detection Shared Task

Detecting sarcasm and verbal irony is critical for understanding people's actual sentiments and beliefs.Thus, the field of sarcasm analysis has become a popular research problem in natural language processing.As the community working on computational approaches for sarcasm detection is growing, it is imperative to conduct benchmarking studies to analyze the current state-of-the-art, facilitating progress in this area.We report on the shared task on sarcasm detection we conducted as a part of the 2nd Workshop on Figurative Language Processing (FigLang 2020) at ACL 2020.
Geryon: Accelerating Distributed CNN Training by Network-Level Flow Scheduling

Increasingly rich data sets and complicated models make distributed machine learning more and more important. However, the cost of extensive and frequent parameter synchronizations can easily diminish the benefits of distributed training across multiple machines. In this paper, we present Geryon, a network-level flow scheduling scheme to accelerate distributed Convolutional Neural Network (CNN) training. Geryon leverages multiple flows with different priorities to transfer parameters of different urgency levels, which can naturally coordinate multiple parameter servers and prioritize the urgent parameter transfers in the entire network fabric. Geryon requires no modification in CNN models and does not affect the training accuracy. Based on the experimental results of four representative CNN models on a testbed of 8 GPU servers, Geryon achieves up to 95.7% scaling efficiency even with 10GbE bandwidth. In contrast, for most models, the scaling efficiency of vanilla TensorFlow is no more than 37% and that of TensorFlow with parameter partition and slicing is around 80%. In terms of training throughput, Geryon enhanced with parameter partition and slicing achieves up to 4.37x speedup, where the flow scheduling algorithm itself achieves up to 1.2x speedup over parameter partition and slicing.
Deep Network With Approximation Error Being Reciprocal of Width to Power of Square Root of Depth

A new network with super-approximation power is introduced. This network is built with Floor (⌊x⌋) or ReLU (max{0,x}) activation function in each neuron; hence, we call such networks Floor-ReLU networks. For any hyperparameters N∈N+ and L∈N+, we show that Floor-ReLU networks with width max{d,5N+13} and depth 64dL+3 can uniformly approximate a Hölder function f on [0,1]d with an approximation error 3λdα/2N-αL, where α∈(0,1] and λ are the Hölder order and constant, respectively. More generally for an arbitrary continuous function f on [0,1]d with a modulus of continuity ωf(·), the constructive approximation rate is ωf(dN-L)+2ωf(d)N-L. As a consequence, this new class of networks overcomes the curse of dimensionality in approximation power when the variation of ωf(r) as r→0 is moderate (e.g., ωf(r)≲rα for Hölder continuous functions), since the major term to be considered in our approximation rate is essentially d times a function of N and L independent of d within the modulus of continuity.
Automatic Modulation Classification Scheme Based on LSTM With Random Erasing and Attention Mechanism

Automatic modulation classification (AMC) is a key technology of cognitive radio used in non-cooperative communication. Recently, deep learning has been applied to AMC tasks. In this paper, an AMC scheme based on deep learning is proposed, which combines random erasing and attention mechanism to achieve high classification accuracy. Firstly, we propose two data augmentation methods, random erasing at sample level and random erasing at amplitude/phase (AP) channel level. The former replaces training samples with noise information, while the latter replaces AP channel information of training samples with noise information. Erased data segments are randomly stitched to enable training data expansion. Training data of different qualities enables deep learning model to have stronger generalization capability and higher robustness. Then, we propose a single-layer Long Short-Term Memory (LSTM) model based on attention mechanism. In the first part of this model, we propose the signal embedding, which enables the input to contain modulation information more comprehensively and accurately. Then hidden state output by LSTM is input into the attention module, and weighting is applied to the hidden state to help the LSTM model capture the temporal features of modulated signals. Compared to a model without attention mechanism, this model has faster convergence speed and better classification performance. Lastly, we propose a random erasing-based test time augmentation (RE-TTA) method. Test data is randomly erased for multiple times and classification results are comprehensively evaluated, in order to further improve classification accuracy. Experimental results on dataset RML2016.10a show that classification accuracy of the proposed scheme is competitive compared with the state-of-the-art methods.