Reviewerly/exHarmony huggingface dataset:
Quantifying the separability of data classes in neural networks

Deep neural networks typically outperform more traditional machine learning models in their ability to classify complex data, and yet is not clear how the individual hidden layers of a deep network contribute to the overall classification performance. We thus introduce a Generalized Discrimination Value (GDV) that measures, in a non-invasive manner, how well different data classes separate in each given network layer. The GDV can be used for the automatic tuning of hyper-parameters, such as the width profile and the total depth of a network. Moreover, the layer-dependent GDV(L) provides new insights into the data transformations that self-organize during training: In the case of multi-layer perceptrons trained with error backpropagation, we find that classification of highly complex data sets requires a temporal {\em reduction} of class separability, marked by a characteristic 'energy barrier' in the initial part of the GDV(L) curve. Even more surprisingly, for a given data set, the GDV(L) is running through a fixed 'master curve', independently from the total number of network layers. Furthermore, applying the GDV to Deep Belief Networks reveals that also unsupervised training with the Contrastive Divergence method can systematically increase class separability over tens of layers, even though the system does not 'know' the desired class labels. These results indicate that the GDV may become a useful tool to open the black box of deep learning.
Cryptocurrency volatility forecasting: What can we learn from the first wave of the COVID-19 outbreak?

This study aims to examine the issue of cryptocurrency volatility modelling and forecasting based on high-frequency data. More specifically, this study assesses whether crisis periods, particularly the coronavirus disease pandemic, influence the dynamic of cryptocurrency volatility. We investigate the four main cryptocurrency markets (Bitcoin, Ethereum Classic, Ethereum, and Ripple) from April 2018 to June 2020. The realized volatility measure is computed and decomposed to various components (continuous versus discontinuous, positive and negative semi-variances, and signed jumps). A variety of heterogeneous autoregressive (HAR) models are developed including these components, thereby enabling assessment of different assumptions (including persistence and asymmetric dynamic) of modelling and volatility forecasting based on in-sample and out-of-sample forecasting strategies, respectively. Our results reveal three main findings. First, the extended HAR model that includes the positive and negative jumps appears to be the best model for predicting future volatility for both crisis and non-crisis periods. Second, during the crisis period, only the negative jump component is statistically significant. Third, in terms of volatility forecasting, the results show that the extended HAR model that includes positive and negative semi-variances outperform the other models.
Text-based Person Search via Multi-Granularity Embedding Learning

Most existing text-based person search methods highly depend on exploring the corresponding relations between the regions of the image and the words in the sentence. However, these methods correlated image regions and words in the same semantic granularity. It 1) results in irrelevant corresponding relations between image and text, 2) causes an ambiguity embedding problem. In this study, we propose a novel multi-granularity embedding learning model for text-based person search. It generates multi-granularity embeddings of partial person bodies in a coarse-to-fine manner by revisiting the person image at different spatial scales. Specifically, we distill the partial knowledge from image scrips to guide the model to select the semantically relevant words from the text description. It can learn discriminative and modality-invariant visual-textual embeddings. In addition, we integrate the partial embeddings at each granularity and perform multi-granularity image-text matching. Extensive experiments validate the effectiveness of our method, which can achieve new state-of-the-art performance by the learned discriminative partial embeddings.
Learning software configuration spaces: A systematic literature review

Most modern software systems (operating systems like Linux or Android, Web browsers like Firefox or Chrome, video encoders like ffmpeg, x264 or VLC, mobile and cloud applications, etc.) are highly configurable. Hundreds of configuration options, features, or plugins can be combined, each potentially with distinct functionality and effects on execution time, security, energy consumption, etc. Due to the combinatorial explosion and the cost of executing software, it is quickly impossible to exhaustively explore the whole configuration space. Hence, numerous works have investigated the idea of learning it from a small sample of configurations’ measurements. The pattern “sampling, measuring, learning” has emerged in the literature, with several practical interests for both software developers and end-users of configurable systems. In this systematic literature review, we report on the different application objectives (e.g., performance prediction, configuration optimization, constraint mining), use-cases, targeted software systems, and application domains. We review the various strategies employed to gather a representative and cost-effective sample. We describe automated software techniques used to measure functional and non-functional properties of configurations. We classify machine learning algorithms and how they relate to the pursued application. Finally, we also describe how researchers evaluate the quality of the learning process. The findings from this systematic review show that the potential application objective is important; there are a vast number of case studies reported in the literature related to particular domains or software systems. Yet, the huge variant space of configurable systems is still challenging and calls to further investigate the synergies between artificial intelligence and software engineering.
Efficient Conformer: Progressive Downsampling and Grouped Attention for Automatic Speech Recognition

The recently proposed Conformer architecture has shown state-of-the-art performances in Automatic Speech Recog-nition by combining convolution with attention to model both local and global dependencies. In this paper, we study how to reduce the Conformer architecture complexity with a limited computing budget, leading to a more efficient archi-tecture design that we call Efficient Conformer. We introduce progressive downsampling to the Conformer encoder and propose a novel attention mechanism named grouped attention, allowing us to reduce attention complexity from <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$O(n^{2}d)$</tex> to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$O(n^{2}d/g)$</tex> for sequence length <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex> , hidden dimension <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$d$</tex> and group size parameter <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$g$</tex> . We also experiment the use of strided multi-head self-attention as a global downsampling operation. Our experiments are performed on the LibriSpeech dataset with CTC and RNN-Transducer losses. We show that within the same computing budget, the proposed architecture achieves better performances with faster training and decoding compared to the Conformer. Our 13M parameters CTC model achieves competitive WERs of 3.6%/9.0% without using a language model and 2.7%/6.7% with an external n-gram language model on the test-clean/test-other sets while being 29% <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Inference time on a single Intel Core i9-9940X 3.3GHz CPU thread faster than our CTC Conformer baseline at inference and 36% faster to train.
A bioinspired neural architecture search based convolutional neural network for breast cancer detection using histopathology images

The design of neural architecture to address the challenge of detecting abnormalities in histopathology images can leverage the gains made in the field of neural architecture search (NAS). The NAS model consists of a search space, search strategy and evaluation strategy. The approach supports the automation of deep learning (DL) based networks such as convolutional neural networks (CNN). Automating the process of CNN architecture engineering using this approach allows for finding the best performing network for learning classification problems in specific domains and datasets. However, the engineering process of NAS is often limited by the potential solutions in search space and the search strategy. This problem often narrows the possibility of obtaining best performing networks for challenging tasks such as the classification of breast cancer in digital histopathological samples. This study proposes a NAS model with a novel search space initialization algorithm and a new search strategy. We designed a block-based stochastic categorical-to-binary (BSCB) algorithm for generating potential CNN solutions into the search space. Also, we applied and investigated the performance of a new bioinspired optimization algorithm, namely the Ebola optimization search algorithm (EOSA), for the search strategy. The evaluation strategy was achieved through computation of loss function, architectural latency and accuracy. The results obtained using images from the BACH and BreakHis databases showed that our approach obtained best performing architectures with the top-5 of the architectures yielding a significant detection rate. The top-1 CNN architecture demonstrated a state-of-the-art performance of base on classification accuracy. The NAS strategy applied in this study and the resulting candidate architecture provides researchers with the most appropriate or suitable network configuration for using digital histopathology.
Attend and Guide (AG-Net): A Keypoints-Driven Attention-Based Deep Network for Image Recognition

This paper presents a novel keypoints-based attention mechanism for visual recognition in still images. Deep Convolutional Neural Networks (CNNs) for recognizing images with distinctive classes have shown great success, but their performance in discriminating fine-grained changes is not at the same level. We address this by proposing an end-to-end CNN model, which learns meaningful features linking fine-grained changes using our novel attention mechanism. It captures the spatial structures in images by identifying semantic regions (SRs) and their spatial distributions, and is proved to be the key to modelling subtle changes in images. We automatically identify these SRs by grouping the detected keypoints in a given image. The ``usefulness'' of these SRs for image recognition is measured using our innovative attentional mechanism focusing on parts of the image that are most relevant to a given task. This framework applies to traditional and fine-grained image recognition tasks and does not require manually annotated regions (e.g. bounding-box of body parts, objects, etc.) for learning and prediction. Moreover, the proposed keypoints-driven attention mechanism can be easily integrated into the existing CNN models. The framework is evaluated on six diverse benchmark datasets. The model outperforms the state-of-the-art approaches by a considerable margin using Distracted Driver V1 (Acc: 3.39%), Distracted Driver V2 (Acc: 6.58%), Stanford-40 Actions (mAP: 2.15%), People Playing Musical Instruments (mAP: 16.05%), Food-101 (Acc: 6.30%) and Caltech-256 (Acc: 2.59%) datasets.
Detection of Breast Cancer Using Histopathological Image Classification Dataset with Deep Learning Techniques

Cancer is one of the top causes of mortality, and it arises when cells in the body grow abnormally, like in the case of breast cancer. For people all around the world, it has now become a huge issue and a threat to their safety and wellbeing. Breast cancer is one of the major causes of death among females all over the globe, and it is particularly prevalent in the United States. It is possible to diagnose breast cancer using a variety of imaging modalities including mammography, computerized tomography (CT), magnetic resonance imaging (MRI), ultrasound, and biopsies, among others. To analyze the picture, a histopathology study (biopsy) is often performed, which assists in the diagnosis of breast cancer. The goal of this study is to develop improved strategies for various CAD phases that will play a critical role in minimizing the variability gap between and among observers. It created an automatic segmentation approach that is then followed by self-driven post-processing activities to successfully identify the Fourier Transform based Segmentation in the CAD system to improve its performance. When compared to existing techniques, the proposed segmentation technique has several advantages: spatial information is incorporated, there is no need to set any initial parameters beforehand, it is independent of magnification, it automatically determines the inputs for morphological operations to enhance segmented images so that pathologists can analyze the image with greater clarity, and it is fast. Extensive tests were conducted to determine the most effective feature extraction techniques and to investigate how textural, morphological, and graph characteristics impact the accuracy of categorization classification. In addition, a classification strategy for breast cancer detection has been developed that is based on weighted feature selection and uses an upgraded version of the Genetic Algorithm in conjunction with a Convolutional Neural Network Classifier. The practical application of the suggested improved segmentation and classification algorithms for the CAD framework may reduce the number of incorrect diagnoses and increase the accuracy of classification. So, it may serve as a second opinion tool for pathologists and aid in the early detection of diseases.
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation

Despite recent progress, state-of-the-art question answering models remain vulnerable to a variety of adversarial attacks.While dynamic adversarial data collection, in which a human annotator tries to write examples that fool a model-in-the-loop, can improve model robustness, this process is expensive which limits the scale of the collected data.In this work, we are the first to use synthetic adversarial data generation to make question answering models more robust to human adversaries.We develop a data generation pipeline that selects source passages, identifies candidate answers, generates questions, then finally filters or relabels them to improve quality.Using this approach, we amplify a smaller human-written adversarial dataset to a much larger set of synthetic question-answer pairs.By incorporating our synthetic data, we improve the state-of-theart on the AdversarialQA dataset by 3.7F 1 and improve model generalisation on nine of the twelve MRQA datasets.We further conduct a novel human-in-the-loop evaluation and show that our models are considerably more robust to new human-written adversarial examples: crowdworkers can fool our model only 8.8% of the time on average, compared to 17.6% for a model trained without synthetic data.
Deep learning for laboratory earthquake prediction and autoregressive forecasting of fault zone stress

Earthquake forecasting and prediction have long and in some cases sordid histories but recent work has rekindled interest based on advances in early warning, hazard assessment for induced seismicity and successful prediction of laboratory earthquakes. In the lab, frictional stick-slip events provide an analog for earthquakes and the seismic cycle. Labquakes are ideal targets for machine learning (ML) because they can be produced in long sequences under controlled conditions. Recent works show that ML can predict several aspects of labquakes using fault zone acoustic emissions. Here, we generalize these results and explore deep learning (DL) methods for labquake prediction and autoregressive (AR) forecasting. DL improves existing ML methods of labquake prediction. AR methods allow forecasting at future horizons via iterative predictions. We demonstrate that DL models based on Long-Short Term Memory (LSTM) and Convolution Neural Networks predict labquakes under several conditions, and that fault zone stress can be predicted with fidelity, confirming that acoustic energy is a fingerprint of fault zone stress. We predict also time to start of failure (TTsF) and time to the end of Failure (TTeF) for labquakes. Interestingly, TTeF is successfully predicted in all seismic cycles, while the TTsF prediction varies with the amount of preseismic fault creep. We report AR methods to forecast the evolution of fault stress using three sequence modeling frameworks: LSTM, Temporal Convolution Network and Transformer Network. AR forecasting is distinct from existing predictive models, which predict only a target variable at a specific time. The results for forecasting beyond a single seismic cycle are limited but encouraging. Our ML/DL models outperform the state-of-the-art and our autoregressive model represents a novel framework that could enhance current methods of earthquake forecasting.
Tempered particle filtering

The accuracy of particle filters for nonlinear state-space models crucially depends on the proposal distribution that mutates time t−1 particle values into time t values. In the widely-used bootstrap particle filter, this distribution is generated by the state-transition equation. While straightforward to implement, the practical performance is often poor. We develop a self-tuning particle filter in which the proposal distribution is constructed adaptively through a sequence of Monte Carlo steps. Intuitively, we start from a measurement error distribution with an inflated variance, and then gradually reduce the variance to its nominal level in a sequence of tempering steps. We show that the filter generates an unbiased and consistent approximation of the likelihood function. Holding the run time fixed, our filter is substantially more accurate in two DSGE model applications than the bootstrap particle filter.
Agent-Based Approaches for Intelligent Intercloud Resource Allocation

Whereas an Intercloud is an interconnected global “cloud of clouds” that enables each cloud to tap into resources of other clouds, interactions among Intercloud stakeholders are complex because Intercloud resources are distributed and controlled by different clouds. “Agent-based cloud computing” involves the construction of agents for bolstering discovery, matching, selection, composition, negotiation, scheduling, workflow, and monitoring of Intercloud resources. An agent is a computer system that is capable of making decisions independently and interacting with other agents through cooperation, coordination, and negotiation. Using an agent-based approach, characteristics associated with intelligent behaviors of agents such as interacting socially through cooperation, coordination, and negotiation can be built into clouds. This survey 1) discusses the significance and advantages of using an agent paradigm for Intercloud resource allocation, 2) reviews representative models of agent-based Intercloud resource allocation and provides a comparison among these models, 3) compares agent-based and non-agent-based approaches for task executions in multiple clouds, and 4) provides pointers to future directions.
Analysis of time-frequency scattering transforms

In this paper we address the problem of constructing a feature extractor which combines Mallat's scattering transform framework with time-frequency (Gabor) representations. To do this, we introduce a class of frames, called uniform covering frames, which includes a variety of semi-discrete Gabor systems. Incorporating a uniform covering frame with a neural network structure yields the Fourier scattering transform $\mathcal{S}_\mathcal{F}$ and the truncated Fourier scattering transform. We prove that $\mathcal{S}_\mathcal{F}$ propagates energy along frequency decreasing paths and its energy decays exponentially as a function of the depth. These quantitative estimates are fundamental in showing that $\mathcal{S}_\mathcal{F}$ satisfies the typical scattering transform properties, and in controlling the information loss due to width and depth truncation. We introduce the fast Fourier scattering transform algorithm, and illustrate the algorithm's performance. The time-frequency covering techniques developed in this paper are flexible and give insight into the analysis of scattering transforms.
On Completely Regular Codes

This work is a survey on completely regular codes. Known properties, relations with other combinatorial structures, and construction methods are considered. The existence problem is also discussed, and known results for some particular cases are established. In addition, we present several new results on completely regular codes with covering radius ρ = 2 and on extended completely regular codes.
Achieving Privacy-Friendly Storage and Secure Statistics for Smart Meter Data on Outsourced Clouds

Smart meters have already been widely used for electric utilities to provide reliable power service. Since those meters keep reporting customer's energy consumption data in minute-level or even second-level, Terabyte-level big data has to be stored and analyzed for the companies. To relieve the storage and computation pressure, some companies attempt to outsource their data on the cloud. However, this exposes customer's privacy at risk, because customer's activities can be inferred from analyzing the meter readings. In this paper, we propose a privacy-friendly cloud storage (PCS) scheme and three secure cloud statistic (SCS) schemes for smart meter data on outsourced clouds. Putting these schemes together achieves three queries from the electric companies. Next, we provably analyze the privacy and the security for these schemes. Finally, we design MapReduce algorithms to show the performance for the cloud statistic.
Fast and Stable Multivariate Kernel Density Estimation by Fast Sum Updating

Kernel density estimation and kernel regression are powerful but computationally expensive techniques: a direct evaluation of kernel density estimates at $M$ evaluation points given $N$ input sample points requires a quadratic $\mathcal{O}(MN)$ operations, which is prohibitive for large scale problems. For this reason, approximate methods such as binning with Fast Fourier Transform or the Fast Gauss Transform have been proposed to speed up kernel density estimation. Among these fast methods, the Fast Sum Updating approach is an attractive alternative, as it is an exact method and its speed is independent of the input sample and the bandwidth. Unfortunately, this method, based on data sorting, has for the most part been limited to the univariate case. In this paper, we revisit the fast sum updating approach and extend it in several ways. Our main contribution is to extend it to the general multivariate case for general input data and rectilinear evaluation grid. Other contributions include its extension to a wider class of kernels, including the triangular, cosine and Silverman kernels, its combination with parsimonious additive multivariate kernels, and its combination with a fast approximate k-nearest-neighbors bandwidth for multivariate datasets. Our numerical tests of multivariate regression and density estimation confirm the speed, accuracy and stability of the method. We hope this paper will renew interest for the fast sum updating approach and help solve large-scale practical density estimation and regression problems.