OpenRaiser/Intern-Atlas huggingface dataset:
Quantifying identifiability to choose and audit $ε$ in differentially private deep learning
Venue: arXiv cs.CR 2021
Year: 2021

Differential privacy allows bounding the influence that training data records have on a machine learning model. To use differential privacy in machine learning, data scientists must choose privacy parameters $(ε,δ)$. Choosing meaningful privacy parameters is key, since models trained with weak privacy parameters might result in excessive privacy leakage, while strong privacy parameters might overly degrade model utility. However, privacy parameter values are difficult to choose for two main reasons. First, the theoretical upper bound on privacy loss $(ε,δ)$ might be loose, depending on the chosen sensitivity and data distribution of practical datasets. Second, legal requirements and societal norms for anonymization often refer to individual identifiability, to which $(ε,δ)$ are only indirectly related. We transform $(ε,δ)$ to a bound on the Bayesian posterior belief of the adversary assumed by differential privacy concerning the presence of any record in the training dataset. The bound holds for multidimensional queries under composition, and we show that it can be tight in practice. Furthermore, we derive an identifiability bound, which relates the adversary assumed in differential privacy to previous work on membership inference adversaries. We formulate an implementation of this differential privacy adversary that allows data scientists to audit model training and compute empirical identifiability scores and empirical $(ε,δ)$.
R-Learning Based Admission Control for Service Federation in Multi-domain 5G Networks
Venue: arXiv cs.NI 2021
Year: 2021

Service federation in 5G/B5G networks enables service providers to orchestrate network services across multiple domains where admission control is a key issue. For each demand, without knowing the future ones, the admission controller either determines the domain to deploy the demand or rejects it in order to maximize the long-term average profit. In this paper, at first, under the assumption of knowing the arrival and departure rates of demands, we obtain the optimal admission control policy by formulating the problem as a Markov decision process that is solved by the policy iteration method. As a practical solution, where the rates are not known, we apply the Q-Learning and R-Learning algorithms to approximate the optimal policy. The extensive simulation results show the learning approaches outperform the greedy policy, and while the performance of Q-Learning depends on the discount factor, the optimality gap of the R-Learning algorithm is at most 3-5% independent of the system configuration.
Online-Extractability in the Quantum Random-Oracle Model
Venue: arXiv cs.CR 2021
Year: 2021

We show the following generic result. Whenever a quantum query algorithm in the quantum random-oracle model outputs a classical value $t$ that is promised to be in some tight relation with $H(x)$ for some $x$, then $x$ can be efficiently extracted with almost certainty. The extraction is by means of a suitable simulation of the random oracle and works online, meaning that it is straightline, i.e., without rewinding, and on-the-fly, i.e., during the protocol execution and without disturbing it. The technical core of our result is a new commutator bound that bounds the operator norm of the commutator of the unitary operator that describes the evolution of the compressed oracle (which is used to simulate the random oracle above) and of the measurement that extracts $x$. We show two applications of our generic online extractability result. We show tight online extractability of commit-and-open $Σ$-protocols in the quantum setting, and we offer the first non-asymptotic post-quantum security proof of the textbook Fujisaki-Okamoto transformation, i.e, without adjustments to facilitate the proof.
An empirical analysis of phrase-based and neural machine translation
Venue: arXiv cs.CL 2021
Year: 2021

Two popular types of machine translation (MT) are phrase-based and neural machine translation systems. Both of these types of systems are composed of multiple complex models or layers. Each of these models and layers learns different linguistic aspects of the source language. However, for some of these models and layers, it is not clear which linguistic phenomena are learned or how this information is learned. For phrase-based MT systems, it is often clear what information is learned by each model, and the question is rather how this information is learned, especially for its phrase reordering model. For neural machine translation systems, the situation is even more complex, since for many cases it is not exactly clear what information is learned and how it is learned. To shed light on what linguistic phenomena are captured by MT systems, we analyze the behavior of important models in both phrase-based and neural MT systems. We consider phrase reordering models from phrase-based MT systems to investigate which words from inside of a phrase have the biggest impact on defining the phrase reordering behavior. Additionally, to contribute to the interpretability of neural MT systems we study the behavior of the attention model, which is a key component in neural MT systems and the closest model in functionality to phrase reordering models in phrase-based systems. The attention model together with the encoder hidden state representations form the main components to encode source side linguistic information in neural MT. To this end, we also analyze the information captured in the encoder hidden state representations of a neural MT system. We investigate the extent to which syntactic and lexical-semantic information from the source side is captured by hidden state representations of different neural MT architectures.
Visual diagnosis of the Varroa destructor parasitic mite in honeybees using object detector techniques
Venue: arXiv cs.CV 2021
Year: 2021

The Varroa destructor mite is one of the most dangerous Honey Bee (Apis mellifera) parasites worldwide and the bee colonies have to be regularly monitored in order to control its spread. Here we present an object detector based method for health state monitoring of bee colonies. This method has the potential for online measurement and processing. In our experiment, we compare the YOLO and SSD object detectors along with the Deep SVDD anomaly detector. Based on the custom dataset with 600 ground-truth images of healthy and infected bees in various scenes, the detectors reached a high F1 score up to 0.874 in the infected bee detection and up to 0.727 in the detection of the Varroa Destructor mite itself. The results demonstrate the potential of this approach, which will be later used in the real-time computer vision based honey bee inspection system. To the best of our knowledge, this study is the first one using object detectors for this purpose. We expect that performance of those object detectors will enable us to inspect the health status of the honey bee colonies.
Requirement Analyses and Evaluations of Blockchain Platforms per Possible Use Cases
Venue: arXiv cs.CR 2021
Year: 2021

It is said that blockchain will contribute to the digital transformation of society in a wide range of ways, from the management of public and private documents to the traceability in various industries, as well as digital currencies. A number of so-called blockchain platforms have been developed, and experiments and applications have been carried out on them. But are these platforms really conducive to practical use of the blockchain concept? To answer the question, we need to better understand what the technology called blockchain really is. We need to sort out the confusion we see in understanding what blockchain was invented for and what it means. We also need to clarify the structure of its applications. This document provides a generic model of understanding blockchain and its applications. We introduce design patterns to classify the platforms. We categorize possible use cases by identifying the structure among applications, and organize the functional, performance, operational and legal requirements for each such case. Based on the categorization and criteria, we evaluated and compared the following platforms: Hyperledger Fabric, Hyperledger Iroha, Hyperledger Indy, Ethereum, Quorum/Hyperledger Besu, Ethereum 2.0, Polkadot, Corda and BBc-1. We have tried to be fair in our evaluations and comparisons, but we also expect to provoke discussion. The intended readers for this document is anyone involved in development of application systems who wants to understand blockchain and their platforms, including non-engineers and non-technologists. The assessments in this document will allow readers to understand the technological requirements for the blockchain platforms, to question existing technologies, and to choose the appropriate platforms for the applications they envision. The comparisons hopefully will also be useful as a guide for designing new technologies.
A Deep Learning Approach to Mapping Irrigation: IrrMapper-U-Net
Venue: arXiv cs.CV 2021
Year: 2021

Accurate maps of irrigation are essential for understanding and managing water resources. We present a new method of mapping irrigation and demonstrate its accuracy for the state of Montana from years 2000-2019. The method is based off of an ensemble of convolutional neural networks that use reflectance information from Landsat imagery to classify irrigated pixels, that we call IrrMapper-U-Net. The methodology does not rely on extensive feature engineering and does not condition the classification with land use information from existing geospatial datasets. The ensemble does not need exhaustive hyperparameter tuning and the analysis pipeline is lightweight enough to be implemented on a personal computer. Furthermore, the proposed methodology provides an estimate of the uncertainty associated with classification. We evaluated our methodology and the resulting irrigation maps using a highly accurate novel spatially-explicit ground truth data set, using county-scale USDA surveys of irrigation extent, and using cadastral surveys. We found that that our method outperforms other methods of mapping irrigation in Montana in terms of overall accuracy and precision. We found that our method agrees better statewide with the USDA National Agricultural Statistics Survey estimates of irrigated area compared to other methods, and has far fewer errors of commission in rainfed agriculture areas. The method learns to mask clouds and ignore Landsat 7 scan-line failures without supervision, reducing the need for preprocessing data. This methodology has the potential to be applied across the entire United States and for the complete Landsat record.
An Almost Optimal Edit Distance Oracle
Venue: arXiv cs.DS 2021
Year: 2021

We consider the problem of preprocessing two strings $S$ and $T$, of lengths $m$ and $n$, respectively, in order to be able to efficiently answer the following queries: Given positions $i,j$ in $S$ and positions $a,b$ in $T$, return the optimal alignment of $S[i \mathinner{.\,.} j]$ and $T[a \mathinner{.\,.} b]$. Let $N=mn$. We present an oracle with preprocessing time $N^{1+o(1)}$ and space $N^{1+o(1)}$ that answers queries in $\log^{2+o(1)}N$ time. In other words, we show that we can query the alignment of every two substrings in almost the same time it takes to compute just the alignment of $S$ and $T$. Our oracle uses ideas from our distance oracle for planar graphs [STOC 2019] and exploits the special structure of the alignment graph. Conditioned on popular hardness conjectures, this result is optimal up to subpolynomial factors. Our results apply to both edit distance and longest common subsequence (LCS). The best previously known oracle with construction time and size $\mathcal{O}(N)$ has slow $Ω(\sqrt{N})$ query time [Sakai, TCS 2019], and the one with size $N^{1+o(1)}$ and query time $\log^{2+o(1)}N$ (using a planar graph distance oracle) has slow $Ω(N^{3/2})$ construction time [Long & Pettie, SODA 2021]. We improve both approaches by roughly a $\sqrt N$ factor.
Ensembles of Random SHAPs
Venue: arXiv cs.LG 2021
Year: 2021

Ensemble-based modifications of the well-known SHapley Additive exPlanations (SHAP) method for the local explanation of a black-box model are proposed. The modifications aim to simplify SHAP which is computationally expensive when there is a large number of features. The main idea behind the proposed modifications is to approximate SHAP by an ensemble of SHAPs with a smaller number of features. According to the first modification, called ER-SHAP, several features are randomly selected many times from the feature set, and Shapley values for the features are computed by means of "small" SHAPs. The explanation results are averaged to get the final Shapley values. According to the second modification, called ERW-SHAP, several points are generated around the explained instance for diversity purposes, and results of their explanation are combined with weights depending on distances between points and the explained instance. The third modification, called ER-SHAP-RF, uses the random forest for preliminary explanation of instances and determining a feature probability distribution which is applied to selection of features in the ensemble-based procedure of ER-SHAP. Many numerical experiments illustrating the proposed modifications demonstrate their efficiency and properties for local explanation.
So you want to be a Super Researcher?
Venue: arXiv cs.CY 2021
Year: 2021

Publishing original scientific research is inherent to the work of a researcher. However, the pressure to maintain productivity and scientific impact can lead to research group publishing excessively, negatively affecting the mental health of a researcher. Ph.D. students and early career researchers are particularly susceptible to this pressure due to the inherent vulnerability of their positions. At present, there are no resources that concisely summarise the publication culture of a research group to help the researcher make an informed decision before joining. In this article, we present the 'Super Researcher' app, an R Shiny application(app) with a user-friendly interface. Using text-mining methodology to extract publicly available author data from Scopus, this pilot app has four fundamental functions to provide snapshot information that will help researchers grasp the publication culture of a research group within minutes. The 'Super Researcher' app provides information on: 1) institution data, 2) author's publication, 3) co-author network plots and 4) publication journals. The 'Super Researcher' app is built on R shiny which provides an interactive interface to users. This app utilizes the Big Data framework Apache Spark to mine relevant information from a huge author information database. The author's information is stored and manipulated using both SQL(SQLite) and NoSQL(HBase) databases. Hbase is used for local data storage and manipulation while SQLite feeds data to the R Shiny interface. In this paper, we introduce these functionalities and illustrate how this information can help guide a researcher to select a new Principle Investigator (PI) with better compatibility in terms of publication attitude using a case study. Available: https://researchmind.co.uk/super-researcher/
Rover Relocalization for Mars Sample Return by Virtual Template Synthesis and Matching
Venue: arXiv cs.RO 2021
Year: 2021

We consider the problem of rover relocalization in the context of the notional Mars Sample Return campaign. In this campaign, a rover (R1) needs to be capable of autonomously navigating and localizing itself within an area of approximately 50 x 50 m using reference images collected years earlier by another rover (R0). We propose a visual localizer that exhibits robustness to the relatively barren terrain that we expect to find in relevant areas, and to large lighting and viewpoint differences between R0 and R1. The localizer synthesizes partial renderings of a mesh built from reference R0 images and matches those to R1 images. We evaluate our method on a dataset totaling 2160 images covering the range of expected environmental conditions (terrain, lighting, approach angle). Experimental results show the effectiveness of our approach. This work informs the Mars Sample Return campaign on the choice of a site where Perseverance (R0) will place a set of sample tubes for future retrieval by another rover (R1).
Learning in Matrix Games can be Arbitrarily Complex
Venue: arXiv cs.LG 2021
Year: 2021

A growing number of machine learning architectures, such as Generative Adversarial Networks, rely on the design of games which implement a desired functionality via a Nash equilibrium. In practice these games have an implicit complexity (e.g. from underlying datasets and the deep networks used) that makes directly computing a Nash equilibrium impractical or impossible. For this reason, numerous learning algorithms have been developed with the goal of iteratively converging to a Nash equilibrium. Unfortunately, the dynamics generated by the learning process can be very intricate and instances of training failure hard to interpret. In this paper we show that, in a strong sense, this dynamic complexity is inherent to games. Specifically, we prove that replicator dynamics, the continuous-time analogue of Multiplicative Weights Update, even when applied in a very restricted class of games -- known as finite matrix games -- is rich enough to be able to approximate arbitrary dynamical systems. Our results are positive in the sense that they show the nearly boundless dynamic modelling capabilities of current machine learning practices, but also negative in implying that these capabilities may come at the cost of interpretability. As a concrete example, we show how replicator dynamics can effectively reproduce the well-known strange attractor of Lonrenz dynamics (the "butterfly effect") while achieving no regret.
Measuring Model Biases in the Absence of Ground Truth
Venue: arXiv cs.CV 2021
Year: 2021

The measurement of bias in machine learning often focuses on model performance across identity subgroups (such as man and woman) with respect to groundtruth labels. However, these methods do not directly measure the associations that a model may have learned, for example between labels and identity subgroups. Further, measuring a model's bias requires a fully annotated evaluation dataset which may not be easily available in practice. We present an elegant mathematical solution that tackles both issues simultaneously, using image classification as a working example. By treating a classification model's predictions for a given image as a set of labels analogous to a bag of words, we rank the biases that a model has learned with respect to different identity labels. We use (man, woman) as a concrete example of an identity label set (although this set need not be binary), and present rankings for the labels that are most biased towards one identity or the other. We demonstrate how the statistical properties of different association metrics can lead to different rankings of the most "gender biased" labels, and conclude that normalized pointwise mutual information (nPMI) is most useful in practice. Finally, we announce an open-sourced nPMI visualization tool using TensorBoard.
Constrained Contrastive Distribution Learning for Unsupervised Anomaly Detection and Localisation in Medical Images
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised anomaly detection (UAD) learns one-class classifiers exclusively with normal (i.e., healthy) images to detect any abnormal (i.e., unhealthy) samples that do not conform to the expected normal patterns. UAD has two main advantages over its fully supervised counterpart. Firstly, it is able to directly leverage large datasets available from health screening programs that contain mostly normal image samples, avoiding the costly manual labelling of abnormal samples and the subsequent issues involved in training with extremely class-imbalanced data. Further, UAD approaches can potentially detect and localise any type of lesions that deviate from the normal patterns. One significant challenge faced by UAD methods is how to learn effective low-dimensional image representations to detect and localise subtle abnormalities, generally consisting of small lesions. To address this challenge, we propose a novel self-supervised representation learning method, called Constrained Contrastive Distribution learning for anomaly detection (CCD), which learns fine-grained feature representations by simultaneously predicting the distribution of augmented data and image contexts using contrastive learning with pretext constraints. The learned representations can be leveraged to train more anomaly-sensitive detection models. Extensive experiment results show that our method outperforms current state-of-the-art UAD approaches on three different colonoscopy and fundus screening datasets. Our code is available at https://github.com/tianyu0207/CCD.
Human-Understandable Decision Making for Visual Recognition
Venue: arXiv cs.AI 2021
Year: 2021

The widespread use of deep neural networks has achieved substantial success in many tasks. However, there still exists a huge gap between the operating mechanism of deep learning models and human-understandable decision making, so that humans cannot fully trust the predictions made by these models. To date, little work has been done on how to align the behaviors of deep learning models with human perception in order to train a human-understandable model. To fill this gap, we propose a new framework to train a deep neural network by incorporating the prior of human perception into the model learning process. Our proposed model mimics the process of perceiving conceptual parts from images and assessing their relative contributions towards the final recognition. The effectiveness of our proposed model is evaluated on two classical visual recognition tasks. The experimental results and analysis confirm our model is able to provide interpretable explanations for its predictions, but also maintain competitive recognition accuracy.
Towards a standardised strategy to collect and distribute application software artifacts
Venue: arXiv cs.SE 2021
Year: 2021

Reference sets contain known content that are used to identify relevant or filter irrelevant content. Application profiles are a type of reference set that contain digital artifacts associated with application software. An application profile can be compared against a target data set to identify relevant evidence of application usage in a variety of investigation scenarios. The research objective is to design and implement a standardised strategy to collect and distribute application software artifacts using application profiles. An advanced technique for creating application profiles was designed using a formalised differential analysis strategy. The design was implemented in a live differential forensic analysis tool, LiveDiff, to automate and simplify data collection. A storage mechanism was designed based on a previously standardised forensic data abstraction. The design was implemented in a new data abstraction, Application Profile XML (APXML), to provide storage, distribution and automated processing of collected artifacts.