OpenRaiser/Intern-Atlas huggingface dataset:
Triaging moderate COVID-19 and other viral pneumonias from routine blood tests
Venue: arXiv cs.LG 2020
Year: 2020

The COVID-19 is sweeping the world with deadly consequences. Its contagious nature and clinical similarity to other pneumonias make separating subjects contracted with COVID-19 and non-COVID-19 viral pneumonia a priority and a challenge. However, COVID-19 testing has been greatly limited by the availability and cost of existing methods, even in developed countries like the US. Intrigued by the wide availability of routine blood tests, we propose to leverage them for COVID-19 testing using the power of machine learning. Two proven-robust machine learning model families, random forests (RFs) and support vector machines (SVMs), are employed to tackle the challenge. Trained on blood data from 208 moderate COVID-19 subjects and 86 subjects with non-COVID-19 moderate viral pneumonia, the best result is obtained in an SVM-based classifier with an accuracy of 84%, a sensitivity of 88%, a specificity of 80%, and a precision of 92%. The results are found explainable from both machine learning and medical perspectives. A privacy-protected web portal is set up to help medical personnel in their practice and the trained models are released for developers to further build other applications. We hope our results can help the world fight this pandemic and welcome clinical verification of our approach on larger populations.
Arabic Dialect Identification in the Wild
Venue: arXiv cs.CL 2020
Year: 2020

We present QADI, an automatically collected dataset of tweets belonging to a wide range of country-level Arabic dialects -covering 18 different countries in the Middle East and North Africa region. Our method for building this dataset relies on applying multiple filters to identify users who belong to different countries based on their account descriptions and to eliminate tweets that are either written in Modern Standard Arabic or contain inappropriate language. The resultant dataset contains 540k tweets from 2,525 users who are evenly distributed across 18 Arab countries. Using intrinsic evaluation, we show that the labels of a set of randomly selected tweets are 91.5% accurate. For extrinsic evaluation, we are able to build effective country-level dialect identification on tweets with a macro-averaged F1-score of 60.6% across 18 classes.
Performance Analysis of the Hybrid IoT Security Model of MQTT and UMA
Venue: arXiv cs.NI 2020
Year: 2020

IoT applications are promising for future daily activities; therefore, the number of IoT connected devices is expected to reach billions in the coming few years. However, IoT has different application frameworks. Furthermore, IoT applications require higher security standards. In this work, an IoT application framework is presented with a security embedded structure using the integration between message queue telemetry transport (MQTT) and user managed access (UMA). The performance analysis of the model is presented. Comparing the model with existing models and different design structures shows that the model presented in this work is promising for a functioning IoT design model with security. The security in the model is a built-in feature in its structure. The model is built on recommended frameworks; therefore, it is ready for integration with other web standards for data sharing, which will help in making IoT application integrated from different developing parties.
Federated Recommendation System via Differential Privacy
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we are interested in what we term the federated private bandits framework, that combines differential privacy with multi-agent bandit learning. We explore how differential privacy based Upper Confidence Bound (UCB) methods can be applied to multi-agent environments, and in particular to federated learning environments both in `master-worker' and `fully decentralized' settings. We provide a theoretical analysis on the privacy and regret performance of the proposed methods and explore the tradeoffs between these two.
Deep Hierarchical Classification for Category Prediction in E-commerce System
Venue: arXiv cs.IR 2020
Year: 2020

In e-commerce system, category prediction is to automatically predict categories of given texts. Different from traditional classification where there are no relations between classes, category prediction is reckoned as a standard hierarchical classification problem since categories are usually organized as a hierarchical tree. In this paper, we address hierarchical category prediction. We propose a Deep Hierarchical Classification framework, which incorporates the multi-scale hierarchical information in neural networks and introduces a representation sharing strategy according to the category tree. We also define a novel combined loss function to punish hierarchical prediction losses. The evaluation shows that the proposed approach outperforms existing approaches in accuracy.
OD-SGD: One-step Delay Stochastic Gradient Descent for Distributed Training
Venue: arXiv cs.LG 2020
Year: 2020

The training of modern deep learning neural network calls for large amounts of computation, which is often provided by GPUs or other specific accelerators. To scale out to achieve faster training speed, two update algorithms are mainly applied in the distributed training process, i.e. the Synchronous SGD algorithm (SSGD) and Asynchronous SGD algorithm (ASGD). SSGD obtains good convergence point while the training speed is slowed down by the synchronous barrier. ASGD has faster training speed but the convergence point is lower when compared to SSGD. To sufficiently utilize the advantages of SSGD and ASGD, we propose a novel technology named One-step Delay SGD (OD-SGD) to combine their strengths in the training process. Therefore, we can achieve similar convergence point and training speed as SSGD and ASGD separately. To the best of our knowledge, we make the first attempt to combine the features of SSGD and ASGD to improve distributed training performance. Each iteration of OD-SGD contains a global update in the parameter server node and local updates in the worker nodes, the local update is introduced to update and compensate the delayed local weights. We evaluate our proposed algorithm on MNIST, CIFAR-10 and ImageNet datasets. Experimental results show that OD-SGD can obtain similar or even slightly better accuracy than SSGD, while its training speed is much faster, which even exceeds the training speed of ASGD.
Algorithmic Techniques for Necessary and Possible Winners
Venue: arXiv cs.GT 2020
Year: 2020

We investigate the practical aspects of computing the necessary and possible winners in elections over incomplete voter preferences. In the case of the necessary winners, we show how to implement and accelerate the polynomial-time algorithm of Xia and Conitzer. In the case of the possible winners, where the problem is NP-hard, we give a natural reduction to Integer Linear Programming (ILP) for all positional scoring rules and implement it in a leading commercial optimization solver. Further, we devise optimization techniques to minimize the number of ILP executions and, oftentimes, avoid them altogether. We conduct a thorough experimental study that includes the construction of a rich benchmark of election data based on real and synthetic data. Our findings suggest that, the worst-case intractability of the possible winners notwithstanding, the algorithmic techniques presented here scale well and can be used to compute the possible winners in realistic scenarios.
Industrial Federated Learning -- Requirements and System Design
Venue: arXiv cs.AI 2020
Year: 2020

Federated Learning (FL) is a very promising approach for improving decentralized Machine Learning (ML) models by exchanging knowledge between participating clients without revealing private data. Nevertheless, FL is still not tailored to the industrial context as strong data similarity is assumed for all FL tasks. This is rarely the case in industrial machine data with variations in machine type, operational- and environmental conditions. Therefore, we introduce an Industrial Federated Learning (IFL) system supporting knowledge exchange in continuously evaluated and updated FL cohorts of learning tasks with sufficient data similarity. This enables optimal collaboration of business partners in common ML problems, prevents negative knowledge transfer, and ensures resource optimization of involved edge devices.
Manthan: A Data Driven Approach for Boolean Function Synthesis
Venue: arXiv cs.AI 2020
Year: 2020

Boolean functional synthesis is a fundamental problem in computer science with wide-ranging applications and has witnessed a surge of interest resulting in progressively improved techniques over the past decade. Despite intense algorithmic development, a large number of problems remain beyond the reach of the state of the art techniques. Motivated by the progress in machine learning, we propose Manthan, a novel data-driven approach to Boolean functional synthesis. Manthan views functional synthesis as a classification problem, relying on advances in constrained sampling for data generation, and advances in automated reasoning for a novel proof-guided refinement and provable verification. On an extensive and rigorous evaluation over 609 benchmarks, we demonstrate that Manthan significantly improves upon the current state of the art, solving 356 benchmarks in comparison to 280, which is the most solved by a state of the art technique; thereby, we demonstrate an increase of 76 benchmarks over the current state of the art. Furthermore, Manthan solves 60 benchmarks that none of the current state of the art techniques could solve. The significant performance improvements, along with our detailed analysis, highlights several interesting avenues of future work at the intersection of machine learning, constrained sampling, and automated reasoning.
Simultaneous imputation and disease classification in incomplete medical datasets using Multigraph Geometric Matrix Completion (MGMC)
Venue: arXiv cs.LG 2020
Year: 2020

Large-scale population-based studies in medicine are a key resource towards better diagnosis, monitoring, and treatment of diseases. They also serve as enablers of clinical decision support systems, in particular Computer Aided Diagnosis (CADx) using machine learning (ML). Numerous ML approaches for CADx have been proposed in literature. However, these approaches assume full data availability, which is not always feasible in clinical data. To account for missing data, incomplete data samples are either removed or imputed, which could lead to data bias and may negatively affect classification performance. As a solution, we propose an end-to-end learning of imputation and disease prediction of incomplete medical datasets via Multigraph Geometric Matrix Completion (MGMC). MGMC uses multiple recurrent graph convolutional networks, where each graph represents an independent population model based on a key clinical meta-feature like age, sex, or cognitive function. Graph signal aggregation from local patient neighborhoods, combined with multigraph signal fusion via self-attention, has a regularizing effect on both matrix reconstruction and classification performance. Our proposed approach is able to impute class relevant features as well as perform accurate classification on two publicly available medical datasets. We empirically show the superiority of our proposed approach in terms of classification and imputation performance when compared with state-of-the-art approaches. MGMC enables disease prediction in multimodal and incomplete medical datasets. These findings could serve as baseline for future CADx approaches which utilize incomplete datasets.
APPCorp: A Corpus for Android Privacy Policy Document Structure Analysis
Venue: arXiv cs.CY 2020
Year: 2020

With the increasing popularity of mobile devices and the wide adoption of mobile Apps, an increasing concern of privacy issues is raised. Privacy policy is identified as a proper medium to indicate the legal terms, such as GDPR, and to bind legal agreement between service providers and users. However, privacy policies are usually long and vague for end users to read and understand. It is thus important to be able to automatically analyze the document structures of privacy policies to assist user understanding. In this work we create a manually labelled corpus containing $167$ privacy policies (of more than $447$K words and $5,276$ annotated paragraphs). We report the annotation process and details of the annotated corpus. We also benchmark our data corpus with $4$ document classification models, thoroughly analyze the results and discuss challenges and opportunities for the research committee to use the corpus. We release our labelled corpus as well as the classification models for public access.
4chan & 8chan embeddings
Venue: arXiv cs.CL 2020
Year: 2020

We have collected over 30M messages from the publicly available /pol/ message boards on 4chan and 8chan, and compiled them into a model of toxic language use. The trained word embeddings (0.4GB) are released for free and may be useful for further study on toxic discourse or to boost hate speech detection systems: https://textgain.com/8chan.
The universality of skipping behaviours on music streaming platforms
Venue: arXiv cs.SI 2020
Year: 2020

A recent study of skipping behaviour on music streaming platforms has shown that the skip profile for a given song -- i.e. the measure of the skipping rate as a function of the time in the song -- can be seen as some intrinsic characteristic of the song, in the sense that it is both very specific and highly stable over time and geographical regions. In this paper, we take this analysis one step further by introducing a simple model of skip behaviours, in which the skip profile for a given song is viewed as the response to a small number of events that happen within it. In particular, it allows us to identify accurately the timing of the events that trigger skip responses, as well as the fraction of users who skip following each these events. Strikingly, the responses triggered by individual events appears to follow a temporal profile that is consistent across songs, genres, devices and listening contexts, suggesting that people react to musical surprises in a universal way.
DjangoChecker: Applying Extended Taint Tracking and Server Side Parsing for Detection of Context-Sensitive XSS Flaws
Venue: arXiv cs.CR 2020
Year: 2020

Cross-site scripting (XSS) flaws are a class of security flaws that permit the injection of malicious code into a web application. In simple situations, these flaws can be caused by missing input sanitizations. Sometimes, however, all application inputs are sanitized, but the sanitizations are not appropriate for the browser contexts of the sanitized values. Using an incorrect sanitizer can make the application look protected, when it is in fact vulnerable as if no sanitization was used, creating a context-sensitive XSS flaw. To discover context-sensitive XSS flaws, we introduce DjangoChecker. DjangoChecker combines extended dynamic taint tracking with a model browser for context analysis. We demonstrate the practical application of DjangoChecker on eight mature web applications based on Django, discovering previously unknown flaws in seven of the eight applications, including highly severe flaws that allow arbitrary JavaScript execution in the seven flawed applications.
Sparse recovery in bounded Riesz systems with applications to numerical methods for PDEs
Venue: arXiv cs.IT 2020
Year: 2020

We study sparse recovery with structured random measurement matrices having independent, identically distributed, and uniformly bounded rows and with a nontrivial covariance structure. This class of matrices arises from random sampling of bounded Riesz systems and generalizes random partial Fourier matrices. Our main result improves the currently available results for the null space and restricted isometry properties of such random matrices. The main novelty of our analysis is a new upper bound for the expectation of the supremum of a Bernoulli process associated with a restricted isometry constant. We apply our result to prove new performance guarantees for the CORSING method, a recently introduced numerical approximation technique for partial differential equations (PDEs) based on compressive sensing.
Plane-Activated Mapped Microstructure
Venue: arXiv cs.GR 2020
Year: 2020

Querying and interacting with models of massive material micro-structure requires localized on-demand generation of the micro-structure since the full-scale storing and retrieving is cost prohibitive. When the micro-structure is efficiently represented as the image of a canonical structure under a non-linear space deformation to allow it to conform to curved shape, the additional challenge is to relate the query of the mapped micro-structure back to its canonical structure. This paper presents an efficient algorithm to pull back a mapped micro-structure to a partition of the canonical domain structure into boxes and only activates boxes whose image is likely intersected by a plane. The active boxes are organized into a forest whose trees are traversed depth first to generate mapped micro-structure only of the active boxes. The traversal supports, for example, 3D print slice generation in additive manufacturing.
Thermodynamically Stable DNA Code Design using a Similarity Significance Model
Venue: arXiv cs.IT 2020
Year: 2020

DNA code design aims to generate a set of DNA sequences (codewords) with minimum likelihood of undesired hybridizations among sequences and their reverse-complement (RC) pairs (cross-hybridization). Inspired by the distinct hybridization affinities (or stabilities) of perfect double helix constructed by individual single-stranded DNA (ssDNA) and its RC pair, we propose a novel similarity significance (SS) model to measure the similarity between DNA sequences. Particularly, instead of directly measuring the similarity of two sequences by any metric/approach, the proposed SS works in a way to evaluate how more likely will the undesirable hybridizations occur over the desirable hybridizations in the presence of the two measured sequences and their RC pairs. With this SS model, we construct thermodynamically stable DNA codes subject to several combinatorial constraints using a sorting-based algorithm. The proposed scheme results in DNA codes with larger code sizes and wider free energy gaps (hence better cross-hybridization performance) compared to the existing methods.
Simulation-Based Inference for Global Health Decisions
Venue: arXiv cs.LG 2020
Year: 2020

The COVID-19 pandemic has highlighted the importance of in-silico epidemiological modelling in predicting the dynamics of infectious diseases to inform health policy and decision makers about suitable prevention and containment strategies. Work in this setting involves solving challenging inference and control problems in individual-based models of ever increasing complexity. Here we discuss recent breakthroughs in machine learning, specifically in simulation-based inference, and explore its potential as a novel venue for model calibration to support the design and evaluation of public health interventions. To further stimulate research, we are developing software interfaces that turn two cornerstone COVID-19 and malaria epidemiology models COVID-sim, (https://github.com/mrc-ide/covid-sim/) and OpenMalaria (https://github.com/SwissTPH/openmalaria) into probabilistic programs, enabling efficient interpretable Bayesian inference within those simulators.
Probabilistic Guarantees for Safe Deep Reinforcement Learning
Venue: arXiv cs.AI 2020
Year: 2020

Deep reinforcement learning has been successfully applied to many control tasks, but the application of such agents in safety-critical scenarios has been limited due to safety concerns. Rigorous testing of these controllers is challenging, particularly when they operate in probabilistic environments due to, for example, hardware faults or noisy sensors. We propose MOSAIC, an algorithm for measuring the safety of deep reinforcement learning agents in stochastic settings. Our approach is based on the iterative construction of a formal abstraction of a controller's execution in an environment, and leverages probabilistic model checking of Markov decision processes to produce probabilistic guarantees on safe behaviour over a finite time horizon. It produces bounds on the probability of safe operation of the controller for different initial configurations and identifies regions where correct behaviour can be guaranteed. We implement and evaluate our approach on agents trained for several benchmark control problems.
A Berkeley View of Teaching CS at Scale
Venue: arXiv cs.CY 2020
Year: 2020

Over the past decade, undergraduate Computer Science (CS) programs across the nation have experienced an explosive growth in enrollment as computational skills have proven increasingly important across many domains and in the workforce at large. Motivated by this unprecedented student demand, the CS program at the University of California, Berkeley has tripled the size of its graduating class in five years. The first two introductory courses for majors, each taught by one faculty instructor and several hundred student teachers, combine to serve nearly 2,900 students per term. This report presents three strategies that have enabled the effective teaching, delivery, and management of large-scale CS courses: (1) the development of autograder infrastructure and online platforms to provide instant feedback with minimal instructor intervention and deliver the course at scale; (2) the expansion of academic and social student support networks resulting from changes in teaching assistant responsibilities and the development of several near-peer mentoring communities; and (3) the expansion of undergraduate teacher preparation programs to meet the increased demand for qualified student teachers. These interventions have helped both introductory and advanced courses address capacity challenges and expand enrollments while receiving among the highest student evaluations of teaching in department history. Implications for inclusivity and diversity are discussed.
Staggered Diagonal Embedding Based Linear Field Size Streaming Codes
Venue: arXiv cs.IT 2020
Year: 2020

An $(a,b,τ)$ streaming code is a packet-level erasure code that can recover under a strict delay constraint of $τ$ time units, from either a burst of $b$ erasures or else of $a$ random erasures, occurring within a sliding window of time duration $w$. While rate-optimal constructions of such streaming codes are available for all parameters $\{a,b,τ,w\}$ in the literature, they require in most instances, a quadratic, $O(τ^2)$ field size. In this work, we make further progress towards field size reduction and present rate-optimal $O(τ)$ field size streaming codes for two regimes: (i) $gcd(b,τ+1-a)\ge a$ (ii) $τ+1 \ge a+b$ and $b \mod \ a \in \{0,a-1\}$.