Reviewerly/exHarmony huggingface dataset:
Trace-Norm Adversarial Examples

White box adversarial perturbations are sought via iterative optimization algorithms most often minimizing an adversarial loss on a $l_p$ neighborhood of the original image, the so-called distortion set. Constraining the adversarial search with different norms results in disparately structured adversarial examples. Here we explore several distortion sets with structure-enhancing algorithms. These new structures for adversarial examples, yet pervasive in optimization, are for instance a challenge for adversarial theoretical certification which again provides only $l_p$ certificates. Because adversarial robustness is still an empirical field, defense mechanisms should also reasonably be evaluated against differently structured attacks. Besides, these structured adversarial perturbations may allow for larger distortions size than their $l_p$ counter-part while remaining imperceptible or perceptible as natural slight distortions of the image. Finally, they allow some control on the generation of the adversarial perturbation, like (localized) bluriness.
Not All Unlabeled Data are Equal: Learning to Weight Data in  Semi-supervised Learning

Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for every unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.
Robust ambiguity for contact tracing

A known drawback of `decentralised' contact tracing architectures is that users who have been in contact with an infected person are able to precisely identify the relevant contact, and thereby perhaps identify the infected person. In their proposal, the PACT team discuss a simple DH-based protocol to mitigate this problem, but dismiss it because it is vulnerable to a malicious user who may deviate from the specified behaviour. This note presents a modified protocol which achieves robustness against a fully malicious user, and establishes some simple security properties.
Addressing the interpretability problem for deep learning using many valued quantum logic

Deep learning models are widely used for various industrial and scientific applications. Even though these models have achieved considerable success in recent years, there exists a lack of understanding of the rationale behind decisions made by such systems in the machine learning community. This problem of interpretability is further aggravated by the increasing complexity of such models. This paper utilizes concepts from machine learning, quantum computation and quantum field theory to demonstrate how a many valued quantum logic system naturally arises in a specific class of generative deep learning models called Convolutional Deep Belief Networks. It provides a robust theoretical framework for constructing deep learning models equipped with the interpretability of many valued quantum logic systems without compromising their computing efficiency.
Application Research of Petroleum Basic Data Mining System Based on Intelligent Computing and Decision Tree Algorithm

Recent improvements in data mining technologies, besides the IoT, enable the implementation of a strategy for boosting oil output from oil wells. As a regularly employed improved oil recovery technology, steam flood injection takes use of thermodynamic and gravitational capabilities to deploy and neutralize oil on-site to raise oil output. Instead of relying on conventional physics to model steam floods, this research proposes using a combination of a chimp optimization algorithm (ChOA) and a decision tree to better represent steam flood performance. We present a method for dealing with a particular type of petroleum time series data using ChOA in conjunction with decision trees and IoT. It is shown that the method is useful in predicting oil production in steam floods. Even more impressive is the 4.02 percent increase in oil output that may be achieved via the use of a new optimization system that offers the best possible steam allocation plan. Our objective has been to develop a cloud-based minimum viable product capable of data collection and storage and also training and deployment of a cloud ChOA model. Predictive maintenance, for example, might benefit from this workflow’s ability to analyze time series data.
Development of Support System to Check Engine Room Machinery and Its Applications to Other Fields

We have developed a "support system to check engine room machinery" (hereinafter, it is called "support system") that can conduct easy and reliable checks on the engine room, the results of which can be utilized effectively for inspection and maintenance. As the support system is applied not only to other onboard operations that should be carried out in accordance with operation procedure manuals, but also to on-site operations in other fields, we introduced additional functions to the support system. Moreover, we investigated and developed a function necessary to assist newcomers who learn practical lessons on-site.
INFLATION FORECASTS IN AMBON USING NEURAL NETWORK APPLICATIONS BACKPROPAGATION

An artificial Neural Network is the processing of information systems on certain characteristics which are artificial representations based on human neural networks. Artificial Neural Networks can be applied to various fields in human life, one of which is the economic field. In this study, the Artificial Neural Network is used to predict the inflation rate using the Backpropagation method. The data used in this study is 144 data, with 100 data as training data and 44 data as test data taken from the Central Statistics Agency of Maluku Province from 2008-2019. The best prediction accuracy level is obtained by using learning rate (a) = 0.1, Target Error = 0.000001, Maximum epoch = 500, network architecture 11-1, and 70% training data sharing scheme and 30% test data. The average absolute error percentage (MAPE) is 85.21%.
Iterative Paraphrastic Augmentation with Discriminative Span Alignment

We introduce a novel paraphrastic augmentation strategy based on sentence-level lexically constrained paraphrasing and discriminative span alignment. Our approach allows for the large-scale expansion of existing resources, or the rapid creation of new resources from a small, manually-produced seed corpus. We illustrate our framework on the Berkeley FrameNet Project, a large-scale language understanding effort spanning more than two decades of human labor. Based on roughly four days of collecting training data for the alignment model and approximately one day of parallel compute, we automatically generate 495,300 unique (Frame, Trigger) combinations annotated in context, a roughly 50x expansion atop FrameNet v1.7.
Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey

As we seek to deploy machine learning models beyond virtual and controlled domains, it is critical to analyze not only the accuracy or the fact that it works most of the time, but if such a model is truly robust and reliable. This paper studies strategies to implement adversary robustly trained algorithms towards guaranteeing safety in machine learning algorithms. We provide a taxonomy to classify adversarial attacks and defenses, formulate the Robust Optimization problem in a min-max setting and divide it into 3 subcategories, namely: Adversarial (re)Training, Regularization Approach, and Certified Defenses. We survey the most recent and important results in adversarial example generation, defense mechanisms with adversarial (re)Training as their main defense against perturbations. We also survey mothods that add regularization terms that change the behavior of the gradient, making it harder for attackers to achieve their objective. Alternatively, we've surveyed methods which formally derive certificates of robustness by exactly solving the optimization problem or by approximations using upper or lower bounds. In addition, we discuss the challenges faced by most of the recent algorithms presenting future research perspectives.
Single Shot Structured Pruning Before Training

We introduce a method to speed up training by 2x and inference by 3x in deep neural networks using structured pruning applied before training. Unlike previous works on pruning before training which prune individual weights, our work develops a methodology to remove entire channels and hidden units with the explicit aim of speeding up training and inference. We introduce a compute-aware scoring mechanism which enables pruning in units of sensitivity per FLOP removed, allowing even greater speed ups. Our method is fast, easy to implement, and needs just one forward/backward pass on a single batch of data to complete pruning before training begins.
Developing cooperative policies for multi-stage tasks

This paper proposes the Cooperative Soft Actor Critic (CSAC) method of enabling consecutive reinforcement learning agents to cooperatively solve a long time horizon multi-stage task. This method is achieved by modifying the policy of each agent to maximise both the current and next agent's critic. Cooperatively maximising each agent's critic allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain, the cooperative policies were able to outperform both uncooperative policies as well as a single agent trained across the entire domain. CSAC achieved a success rate of at least 20\% higher than the uncooperative policies, and converged on a solution at least 4 times faster than the single agent.
SE3M: A Model for Software Effort Estimation Using Pre-trained Embedding Models

Estimating effort based on requirement texts presents many challenges, especially in obtaining viable features to infer effort. Aiming to explore a more effective technique for representing textual requirements to infer effort estimates by analogy, this paper proposes to evaluate the effectiveness of pre-trained embeddings models. For this, two embeddings approach, context-less and contextualized models are used. Generic pre-trained models for both approaches went through a fine-tuning process. The generated models were used as input in the applied deep learning architecture, with linear output. The results were very promising, realizing that pre-trained incorporation models can be used to estimate software effort based only on requirements texts. We highlight the results obtained to apply the pre-trained BERT model with fine-tuning in a single project repository, whose value is the Mean Absolute Error (MAE) is 4.25 and the standard deviation of only 0.17, which represents a result very positive when compared to similar works. The main advantages of the proposed estimation method are reliability, the possibility of generalization, speed, and low computational cost provided by the fine-tuning process, and the possibility to infer new or existing requirements.
Using Human Psychophysics to Evaluate Generalization in Scene Text Recognition Models

Scene text recognition models have advanced greatly in recent years. Inspired by human reading we characterize two important scene text recognition models by measuring their domains i.e. the range of stimulus images that they can read. The domain specifies the ability of readers to generalize to different word lengths, fonts, and amounts of occlusion. These metrics identify strengths and weaknesses of existing models. Relative to the attention-based (Attn) model, we discover that the connectionist temporal classification (CTC) model is more robust to noise and occlusion, and better at generalizing to different word lengths. Further, we show that in both models, adding noise to training images yields better generalization to occlusion. These results demonstrate the value of testing models till they break, complementing the traditional data science focus on optimizing performance.
ARPEGE Cloud Cover Forecast Post-Processing with Convolutional Neural Network

Cloud cover is crucial information for many applications such as planning land observation missions from space. It remains however a challenging variable to forecast, and Numerical Weather Prediction (NWP) models suffer from significant biases, hence justifying the use of statistical post-processing techniques. In this study, ARPEGE (M\'et\'eo-France global NWP) cloud cover is post-processed using a convolutional neural network (CNN). CNN is the most popular machine learning tool to deal with images. In our case, CNN allows the integration of spatial information contained in NWP outputs. We use a gridded cloud cover product derived from satellite observations over Europe as ground truth, and predictors are spatial fields of various variables produced by ARPEGE at the corresponding lead time. We show that a simple U-Net architecture produces significant improvements over Europe. Moreover, the U-Net outclasses more traditional machine learning methods used operationally such as a random forest and a logistic quantile regression. We introduced a weighting predictor layer prior to the traditional U-Net architecture which produces a ranking of predictors by importance, facilitating the interpretation of the results. Using $N$ predictors, only $N$ additional weights are trained which does not impact the computational time, representing a huge advantage compared to traditional methods of ranking (permutation importance, sequential selection, ...).
PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning

To build a high-quality open-domain chatbot, we introduce the effective training process of PLATO-2 via curriculum learning. There are two stages involved in the learning process. In the first stage, a coarse-grained generation model is trained to learn response generation under the simplified framework of one-to-one mapping. In the second stage, a fine-grained generative model augmented with latent variables and an evaluation model are further trained to generate diverse responses and to select the best response, respectively. PLATO-2 was trained on both Chinese and English data, whose effectiveness and superiority are verified through comprehensive evaluations, achieving new state-of-the-art results.
Social Distancing 2.0 with Privacy-Preserving Contact Tracing to Avoid a  Second Wave of COVID-19

How to avoid a second wave of COVID-19 after reopening the economy is a pressing question. The extremely high basic reproductive number $R_0$ (5.7 to 6.4, shown in new studies) of SARS-CoV-2 further complicates the challenge. Here we assess effects of Social distancing 2.0, i.e. proximity alert (to maintain inter-personal distance) plus privacy-preserving contact tracing. To solve the dual task, we developed an open source mobile app. The app uses a Bluetooth-based, decentralized contact tracing platform over which the anonymous user ID cannot be linked by the government or a third party. Modelling results show that a 50\% adoption rate of Social distancing 2.0, with privacy-preserving contact tracing, would suffice to decrease the $R_0$ to less than 1 and prevent the resurgence of COVID-19 epidemic.
Ultra2Speech -- A Deep Learning Framework for Formant Frequency Estimation and Tracking from Ultrasound Tongue Images

Thousands of individuals need surgical removal of their larynx due to critical diseases every year and therefore, require an alternative form of communication to articulate speech sounds after the loss of their voice box. This work addresses the articulatory-to-acoustic mapping problem based on ultrasound (US) tongue images for the development of a silent-speech interface (SSI) that can provide them with an assistance in their daily interactions. Our approach targets automatically extracting tongue movement information by selecting an optimal feature set from US images and mapping these features to the acoustic space. We use a novel deep learning architecture to map US tongue images from the US probe placed beneath a subject's chin to formants that we call, Ultrasound2Formant (U2F) Net. It uses hybrid spatio-temporal 3D convolutions followed by feature shuffling, for the estimation and tracking of vowel formants from US images. The formant values are then utilized to synthesize continuous time-varying vowel trajectories, via Klatt Synthesizer. Our best model achieves R-squared (R^2) measure of 99.96% for the regression task. Our network lays the foundation for an SSI as it successfully tracks the tongue contour automatically as an internal representation without any explicit annotation.
Neural Mutual Information Estimation for Channel Coding: State-of-the-Art Estimators, Analysis, and Performance Comparison

Deep learning based physical layer design, i.e., using dense neural networks as encoders and decoders, has received considerable interest recently. However, while such an approach is naturally training data-driven, actions of the wireless channel are mimicked using standard channel models, which only partially reflect the physical ground truth. Very recently, neural network based mutual information (MI) estimators have been proposed that directly extract channel actions from the input-output measurements and feed these outputs into the channel encoder. This is a promising direction as such a new design paradigm is fully adaptive and training data-based. This paper implements further recent improvements of such MI estimators, analyzes theoretically their suitability for the channel coding problem, and compares their performance. To this end, a new MI estimator using a \emph{``reverse Jensen''} approach is proposed.
Constructing a Chain Event Graph from a Staged Tree

Chain Event Graphs (CEGs) are a recent family of probabilistic graphical models - a generalisation of Bayesian Networks - providing an explicit representation of structural zeros, structural missing values and context-specific conditional independences within their graph topology. A CEG is constructed from an event tree through a sequence of transformations beginning with the colouring of the vertices of the event tree to identify one-step transition symmetries. This coloured event tree, also known as a staged tree, is the output of the learning algorithms used for this family. Surprisingly, no general algorithm has yet been devised that automatically transforms any staged tree into a CEG representation. In this paper we provide a simple iterative backward algorithm for this transformation. Additionally, we show that no information is lost from transforming a staged tree into a CEG. Finally, we demonstrate that with an optimal stopping criterion, our algorithm is more efficient than the generalisation of a special case presented in Silander and Leong (2013). We also provide Python code using this algorithm to obtain a CEG from any staged tree along with the functionality to add edges with sampling zeros.
Pattern Masking for Dictionary Matching

In the Pattern Masking for Dictionary Matching (PMDM) problem, we are given a dictionary $\mathcal{D}$ of $d$ strings, each of length $\ell$, a query string $q$ of length $\ell$, and a positive integer $z$, and we are asked to compute a smallest set $K\subseteq\{1,\ldots,\ell\}$, so that if $q[i]$, for all $i\in K$, is replaced by a wildcard, then $q$ matches at least $z$ strings from $\mathcal{D}$. The PMDM problem lies at the heart of two important applications featured in large-scale real-world systems: record linkage of databases that contain sensitive information, and query term dropping. In both applications, solving PMDM allows for providing data utility guarantees as opposed to existing approaches. We first show, through a reduction from the well-known $k$-Clique problem, that a decision version of the PMDM problem is NP-complete, even for strings over a binary alphabet. We present a data structure for PMDM that answers queries over $\mathcal{D}$ in time $\mathcal{O}(2^{\ell/2}(2^{\ell/2}+\tau)\ell)$ and requires space $\mathcal{O}(2^{\ell}d^2/\tau^2+2^{\ell/2}d)$, for any parameter $\tau\in[1,d]$. We also approach the problem from a more practical perspective. We show an $\mathcal{O}((d\ell)^{k/3}+d\ell)$-time and $\mathcal{O}(d\ell)$-space algorithm for PMDM if $k=|K|=\mathcal{O}(1)$. We generalize our exact algorithm to mask multiple query strings simultaneously. We complement our results by showing a two-way polynomial-time reduction between PMDM and the Minimum Union problem [Chlamt\'{a}\v{c} et al., SODA 2017]. This gives a polynomial-time $\mathcal{O}(d^{1/4+\epsilon})$-approximation algorithm for PMDM, which is tight under plausible complexity conjectures.
Statistical-Query Lower Bounds via Functional Gradients

We give the first statistical-query lower bounds for agnostically learning any non-polynomial activation with respect to Gaussian marginals (e.g., ReLU, sigmoid, sign). For the specific problem of ReLU regression (equivalently, agnostically learning a ReLU), we show that any statistical-query algorithm with tolerance $n^{-(1/\epsilon)^b}$ must use at least $2^{n^c} \epsilon$ queries for some constant $b, c > 0$, where $n$ is the dimension and $\epsilon$ is the accuracy parameter. Our results rule out general (as opposed to correlational) SQ learning algorithms, which is unusual for real-valued learning problems. Our techniques involve a gradient boosting procedure for "amplifying" recent lower bounds due to Diakonikolas et al. (COLT 2020) and Goel et al. (ICML 2020) on the SQ dimension of functions computed by two-layer neural networks. The crucial new ingredient is the use of a nonstandard convex functional during the boosting procedure. This also yields a best-possible reduction between two commonly studied models of learning: agnostic learning and probabilistic concepts.
A note on cyclotomic polynomials and Linear Feedback Shift Registers

Linear Feedback Shift Registers (LFRS) are tools commonly used in cryptography in many different context, for example as pseudo-random numbers generators. In this paper we characterize LFRS with certain symmetry properties. Related to this question we also classify polynomials f of degree n satisfying the property that if a is a root of f then $f(a^n)=0$. The classification heavily depends on the choice of the fields of coefficients of the polynomial; we consider the cases $K=\mathbb{F}_p$ and $K=\mathbb{Q}$.