OpenRaiser/Intern-Atlas huggingface dataset:
Machine Learning Fairness in Justice Systems: Base Rates, False Positives, and False Negatives
Venue: arXiv cs.CY 2020
Year: 2020

Machine learning best practice statements have proliferated, but there is a lack of consensus on what the standards should be. For fairness standards in particular, there is little guidance on how fairness might be achieved in practice. Specifically, fairness in errors (both false negatives and false positives) can pose a problem of how to set weights, how to make unavoidable tradeoffs, and how to judge models that present different kinds of errors across racial groups. This paper considers the consequences of having higher rates of false positives for one racial group and higher rates of false negatives for another racial group. The paper examines how different errors in justice settings can present problems for machine learning applications, the limits of computation for resolving tradeoffs, and how solutions might have to be crafted through courageous conversations with leadership, line workers, stakeholders, and impacted communities.
Value Entropy Model: Metric Method of Service Ecosystem Evolution
Venue: arXiv cs.OH 2020
Year: 2020

With the development of cloud computing, service computing, IoT(Internet of Things) and mobile Internet, the diversity and sociality of services are increasingly apparent. To meet the customized user demands, service ecosystems begins to emerge with the formation of various IT services collaboration network. However, service ecosystem is a complex social-technology system with the characteristics of natural ecosystems, economic systems and complex networks. Hence, how to realize the multi-dimensional evaluation of service ecosystem is of great significance to promote its sound development. Based on this, this paper proposes a value entropy model to analyze the performance of service ecosystem, which is conducive to integrate evaluation indicators of different dimensions. In addition, a computational experiment system is constructed to verify the effectiveness of value entropy model. The result shows that our model can provide new means and ideas for the analysis of service ecosystem.
Reinforcement Learning-driven Information Seeking: A Quantum Probabilistic Approach
Venue: arXiv cs.IR 2020
Year: 2020

Understanding an information forager's actions during interaction is very important for the study of interactive information retrieval. Although information spread in uncertain information space is substantially complex due to the high entanglement of users interacting with information objects~(text, image, etc.). However, an information forager, in general, accompanies a piece of information (information diet) while searching (or foraging) alternative contents, typically subject to decisive uncertainty. Such types of uncertainty are analogous to measurements in quantum mechanics which follow the uncertainty principle. In this paper, we discuss information seeking as a reinforcement learning task. We then present a reinforcement learning-based framework to model forager exploration that treats the information forager as an agent to guide their behaviour. Also, our framework incorporates the inherent uncertainty of the foragers' action using the mathematical formalism of quantum mechanics.
ClipUp: A Simple and Powerful Optimizer for Distribution-based Policy Evolution
Venue: arXiv cs.NE 2020
Year: 2020

Distribution-based search algorithms are an effective approach for evolutionary reinforcement learning of neural network controllers. In these algorithms, gradients of the total reward with respect to the policy parameters are estimated using a population of solutions drawn from a search distribution, and then used for policy optimization with stochastic gradient ascent. A common choice in the community is to use the Adam optimization algorithm for obtaining an adaptive behavior during gradient ascent, due to its success in a variety of supervised learning settings. As an alternative to Adam, we propose to enhance classical momentum-based gradient ascent with two simple techniques: gradient normalization and update clipping. We argue that the resulting optimizer called ClipUp (short for "clipped updates") is a better choice for distribution-based policy evolution because its working principles are simple and easy to understand and its hyperparameters can be tuned more intuitively in practice. Moreover, it removes the need to re-tune hyperparameters if the reward scale changes. Experiments show that ClipUp is competitive with Adam despite its simplicity and is effective on challenging continuous control benchmarks, including the Humanoid control task based on the Bullet physics simulator.
Salvage Reusable Samples from Noisy Data for Robust Learning
Venue: arXiv cs.CV 2020
Year: 2020

Due to the existence of label noise in web images and the high memorization capacity of deep neural networks, training deep fine-grained (FG) models directly through web images tends to have an inferior recognition ability. In the literature, to alleviate this issue, loss correction methods try to estimate the noise transition matrix, but the inevitable false correction would cause severe accumulated errors. Sample selection methods identify clean ("easy") samples based on the fact that small losses can alleviate the accumulated errors. However, "hard" and mislabeled examples that can both boost the robustness of FG models are also dropped. To this end, we propose a certainty-based reusable sample selection and correction approach, termed as CRSSC, for coping with label noise in training deep FG models with web images. Our key idea is to additionally identify and correct reusable samples, and then leverage them together with clean examples to update the networks. We demonstrate the superiority of the proposed approach from both theoretical and experimental perspectives.
Training DNN Model with Secret Key for Model Protection
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a model protection method by using block-wise pixel shuffling with a secret key as a preprocessing technique to input images for the first time. The protected model is built by training with such preprocessed images. Experiment results show that the performance of the protected model is close to that of non-protected models when the key is correct, while the accuracy is severely dropped when an incorrect key is given, and the proposed model protection is robust against not only brute-force attacks but also fine-tuning attacks, while maintaining almost the same performance accuracy as that of using a non-protected model.
Gathering GitHub OSS Requirements from Q&A Community: an Empirical Study
Venue: arXiv cs.SE 2020
Year: 2020

Cross-community collaboration can exploit the expertise and knowledges of crowds in different communities. Recently increasing users in open source software (OSS) community like GitHub attempt to gather software requirements from question and answer (Q&A) communities such as Stack Overflow (SO). In order to investigate this emerging crosscommunity collaboration phenomenon, the paper presents an exploratory study on cross-community requirements gathering of OSS projects in GitHub. We manually sample 3266 practice cases and quantitatively analyze the popularity of the phenomenon, the characteristics of the gathered requirements, and collaboration behaviors of cross-community. Some important findings are obtained: more than half of the requirements gathered from SO are enhancements and the majority of the gathered requirements arenon-functionalrequirements.Inaddition,OSSdeveloperscan directlyobtainrelatedsolutionsandcontributionsofthegathered requirements from SO in the gathering process.
Leaderless State-Machine Replication: Specification, Properties, Limits (Extended Version)
Venue: arXiv cs.DC 2020
Year: 2020

Modern Internet services commonly replicate critical data across several geographical locations using state-machine replication (SMR). Due to their reliance on a leader replica, classical SMR protocols offer limited scalability and availability in this setting. To solve this problem, recent protocols follow instead a leaderless approach, in which each replica is able to make progress using a quorum of its peers. In this paper, we study this new emerging class of SMR protocols and states some of their limits. We first propose a framework that captures the essence of leaderless state-machine replication (Leaderless SMR). Then, we introduce a set of desirable properties for these protocols: (R)eliability, (O)ptimal (L)atency and (L)oad Balancing. We show that protocols matching all of the ROLL properties are subject to a trade-off between performance and reliability. We also establish a lower bound on the message delay to execute a command in protocols optimal for the ROLL properties. This lower bound explains the persistent chaining effect observed in experimental results.
Learning Sampling in Financial Statement Audits using Vector Quantised Autoencoder Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

The audit of financial statements is designed to collect reasonable assurance that an issued statement is free from material misstatement 'true and fair presentation'. International audit standards require the assessment of a statements' underlying accounting relevant transactions referred to as 'journal entries' to detect potential misstatements. To efficiently audit the increasing quantities of such entries, auditors regularly conduct a sample-based assessment referred to as 'audit sampling'. However, the task of audit sampling is often conducted early in the overall audit process. Often at a stage, in which an auditor might be unaware of all generative factors and their dynamics that resulted in the journal entries in-scope of the audit. To overcome this challenge, we propose the application of Vector Quantised-Variational Autoencoder (VQ-VAE) neural networks. We demonstrate, based on two real-world city payment datasets, that such artificial neural networks are capable of learning a quantised representation of accounting data. We show that the learned quantisation uncovers (i) the latent factors of variation and (ii) can be utilised as a highly representative audit sample in financial statement audits.
IIIT-AR-13K: A New Dataset for Graphical Object Detection in Documents
Venue: arXiv cs.CV 2020
Year: 2020

We introduce a new dataset for graphical object detection in business documents, more specifically annual reports. This dataset, IIIT-AR-13k, is created by manually annotating the bounding boxes of graphical or page objects in publicly available annual reports. This dataset contains a total of 13k annotated page images with objects in five different popular categories - table, figure, natural image, logo, and signature. It is the largest manually annotated dataset for graphical object detection. Annual reports created in multiple languages for several years from various companies bring high diversity into this dataset. We benchmark IIIT-AR-13K dataset with two state of the art graphical object detection techniques using Faster R-CNN [20] and Mask R-CNN [11] and establish high baselines for further research. Our dataset is highly effective as training data for developing practical solutions for graphical object detection in both business documents and technical articles. By training with IIIT-AR-13K, we demonstrate the feasibility of a single solution that can report superior performance compared to the equivalent ones trained with a much larger amount of data, for table detection. We hope that our dataset helps in advancing the research for detecting various types of graphical objects in business documents.
On the relationship between (secure) multi-party computation and (secure) federated learning
Venue: arXiv cs.CR 2020
Year: 2020

The contribution of this short note, contains the following two parts: in the first part, we are able to show that the federate learning (FL) procedure presented by Kairouz et al. \cite{Kairouz1901}, is a random processing. Namely, an $m$-ary functionality for the FL procedure can be defined in the context of multi-party computation (MPC); Furthermore, an instance of FL protocol along Kairouz et al.'s definition can be viewed as an implementation of the defined $m$-ary functionality. As such, an instance of FL procedure is also an instance of MPC protocol. In short, FL is a subset of MPC. To privately computing the defined FL (m-ary) functionality, various techniques such as homomorphic encryption (HE), secure multi-party computation (SMPC) and differential privacy (DP) have been deployed. In the second part, we are able to show that if the underlying FL instance privately computes the defined $m$-ary functionality in the simulation-based framework, then the simulation-based FL solution is also an instance of SMPC. Consequently, SFL is a subset of SMPC.
Improving the Accuracy of Global Forecasting Models using Time Series Data Augmentation
Venue: arXiv cs.LG 2020
Year: 2020

Forecasting models that are trained across sets of many time series, known as Global Forecasting Models (GFM), have shown recently promising results in forecasting competitions and real-world applications, outperforming many state-of-the-art univariate forecasting techniques. In most cases, GFMs are implemented using deep neural networks, and in particular Recurrent Neural Networks (RNN), which require a sufficient amount of time series to estimate their numerous model parameters. However, many time series databases have only a limited number of time series. In this study, we propose a novel, data augmentation based forecasting framework that is capable of improving the baseline accuracy of the GFM models in less data-abundant settings. We use three time series augmentation techniques: GRATIS, moving block bootstrap (MBB), and dynamic time warping barycentric averaging (DBA) to synthetically generate a collection of time series. The knowledge acquired from these augmented time series is then transferred to the original dataset using two different approaches: the pooled approach and the transfer learning approach. When building GFMs, in the pooled approach, we train a model on the augmented time series alongside the original time series dataset, whereas in the transfer learning approach, we adapt a pre-trained model to the new dataset. In our evaluation on competition and real-world time series datasets, our proposed variants can significantly improve the baseline accuracy of GFM models and outperform state-of-the-art univariate forecasting methods.
Personalised Visual Art Recommendation by Learning Latent Semantic Representations
Venue: arXiv cs.IR 2020
Year: 2020

In Recommender systems, data representation techniques play a great role as they have the power to entangle, hide and reveal explanatory factors embedded within datasets. Hence, they influence the quality of recommendations. Specifically, in Visual Art (VA) recommendations the complexity of the concepts embodied within paintings, makes the task of capturing semantics by machines far from trivial. In VA recommendation, prominent works commonly use manually curated metadata to drive recommendations. Recent works in this domain aim at leveraging visual features extracted using Deep Neural Networks (DNN). However, such data representation approaches are resource demanding and do not have a direct interpretation, hindering user acceptance. To address these limitations, we introduce an approach for Personalised Recommendation of Visual arts based on learning latent semantic representation of paintings. Specifically, we trained a Latent Dirichlet Allocation (LDA) model on textual descriptions of paintings. Our LDA model manages to successfully uncover non-obvious semantic relationships between paintings whilst being able to offer explainable recommendations. Experimental evaluations demonstrate that our method tends to perform better than exploiting visual features extracted using pre-trained Deep Neural Networks.
A logic-based decision support system for the diagnosis of headache disorders according to the ICHD-3 international classification
Venue: arXiv cs.AI 2020
Year: 2020

Decision support systems play an important role in medical fields as they can augment clinicians to deal more efficiently and effectively with complex decision-making processes. In the diagnosis of headache disorders, however, existing approaches and tools are still not optimal. On the one hand, to support the diagnosis of this complex and vast spectrum of disorders, the International Headache Society released in 1988 the International Classification of Headache Disorders (ICHD), now in its 3rd edition: a 200 pages document classifying more than 300 different kinds of headaches, where each is identified via a collection of specific nontrivial diagnostic criteria. On the other hand, the high number of headache disorders and their complex criteria make the medical history process inaccurate and not exhaustive both for clinicians and existing automatic tools. To fill this gap, we present HEAD-ASP, a novel decision support system for the diagnosis of headache disorders. Through a REST Web Service, HEAD-ASP implements a dynamic questionnaire that complies with ICHD-3 by exploiting two logical modules to reach a complete diagnosis while trying to minimize the total number of questions being posed to patients. Finally, HEAD-ASP is freely available on-line and it is receiving very positive feedback from the group of neurologists that is testing it.
On the Accuracy of CRNNs for Line-Based OCR: A Multi-Parameter Evaluation
Venue: arXiv cs.CV 2020
Year: 2020

We investigate how to train a high quality optical character recognition (OCR) model for difficult historical typefaces on degraded paper. Through extensive grid searches, we obtain a neural network architecture and a set of optimal data augmentation settings. We discuss the influence of factors such as binarization, input line height, network width, network depth, and other network training parameters such as dropout. Implementing these findings into a practical model, we are able to obtain a 0.44% character error rate (CER) model from only 10,000 lines of training data, outperforming currently available pretrained models that were trained on more than 20 times the amount of data. We show ablations for all components of our training pipeline, which relies on the open source framework Calamari.
Modelos dinâmicos aplicados à aprendizagem de valores em inteligência artificial
Venue: arXiv cs.CY 2020
Year: 2020

Experts in Artificial Intelligence (AI) development predict that advances in the development of intelligent systems and agents will reshape vital areas in our society. Nevertheless, if such an advance is not made prudently and critically, reflexively, it can result in negative outcomes for humanity. For this reason, several researchers in the area have developed a robust, beneficial, and safe concept of AI for the preservation of humanity and the environment. Currently, several of the open problems in the field of AI research arise from the difficulty of avoiding unwanted behaviors of intelligent agents and systems, and at the same time specifying what we really want such systems to do, especially when we look for the possibility of intelligent agents acting in several domains over the long term. It is of utmost importance that artificial intelligent agents have their values aligned with human values, given the fact that we cannot expect an AI to develop human moral values simply because of its intelligence, as discussed in the Orthogonality Thesis. Perhaps this difficulty comes from the way we are addressing the problem of expressing objectives, values, and ends, using representational cognitive methods. A solution to this problem would be the dynamic approach proposed by Dreyfus, whose phenomenological philosophy shows that the human experience of being-in-the-world in several aspects is not well represented by the symbolic or connectionist cognitive method, especially in regards to the question of learning values. A possible approach to this problem would be to use theoretical models such as SED (situated embodied dynamics) to address the values learning problem in AI.
Assisted Perception: Optimizing Observations to Communicate State
Venue: arXiv cs.LG 2020
Year: 2020

We aim to help users estimate the state of the world in tasks like robotic teleoperation and navigation with visual impairments, where users may have systematic biases that lead to suboptimal behavior: they might struggle to process observations from multiple sensors simultaneously, receive delayed observations, or overestimate distances to obstacles. While we cannot directly change the user's internal beliefs or their internal state estimation process, our insight is that we can still assist them by modifying the user's observations. Instead of showing the user their true observations, we synthesize new observations that lead to more accurate internal state estimates when processed by the user. We refer to this method as assistive state estimation (ASE): an automated assistant uses the true observations to infer the state of the world, then generates a modified observation for the user to consume (e.g., through an augmented reality interface), and optimizes the modification to induce the user's new beliefs to match the assistant's current beliefs. We evaluate ASE in a user study with 12 participants who each perform four tasks: two tasks with known user biases -- bandwidth-limited image classification and a driving video game with observation delay -- and two with unknown biases that our method has to learn -- guided 2D navigation and a lunar lander teleoperation video game. A different assistance strategy emerges in each domain, such as quickly revealing informative pixels to speed up image classification, using a dynamics model to undo observation delay in driving, identifying nearby landmarks for navigation, and exaggerating a visual indicator of tilt in the lander game. The results show that ASE substantially improves the task performance of users with bandwidth constraints, observation delay, and other unknown biases.
A no-phone/no-app contact tracing hardware token
Venue: arXiv cs.CY 2020
Year: 2020

We report the development of an open-source, hardware-based contact tracer, made from readily available parts, costing less than $20USD. This work was motivated by the need for a technology-assisted contact tracer that avoids privacy issues found with those involving a mobile phone. Contact tracing is done here without the use of a mobile phone or an app at all. Instead, contact tracing is implemented using Bluetooth Low Energy on an ESP32 micro-controller. The ESP32 is used to both advertise and receive health information to others in close proximity, forming a strictly peer-to-peer contact tracer. The contact tracer can be assembled by an individual and configured use within minutes.
A Multilingual Neural Machine Translation Model for Biomedical Data
Venue: arXiv cs.CL 2020
Year: 2020

We release a multilingual neural machine translation model, which can be used to translate text in the biomedical domain. The model can translate from 5 languages (French, German, Italian, Korean and Spanish) into English. It is trained with large amounts of generic and biomedical data, using domain tags. Our benchmarks show that it performs near state-of-the-art both on news (generic domain) and biomedical test sets, and that it outperforms the existing publicly released models. We believe that this release will help the large-scale multilingual analysis of the digital content of the COVID-19 crisis and of its effects on society, economy, and healthcare policies. We also release a test set of biomedical text for Korean-English. It consists of 758 sentences from official guidelines and recent papers, all about COVID-19.