OpenRaiser/Intern-Atlas huggingface dataset:
A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data
Venue: arXiv cs.LG 2021
Year: 2021

Accurate prognosis for an individual patient is a key component of precision oncology. Recent advances in machine learning have enabled the development of models using a wider range of data, including imaging. Radiomics aims to extract quantitative predictive and prognostic biomarkers from routine medical imaging, but evidence for computed tomography radiomics for prognosis remains inconclusive. We have conducted an institutional machine learning challenge to develop an accurate model for overall survival prediction in head and neck cancer using clinical data etxracted from electronic medical records and pre-treatment radiological images, as well as to evaluate the true added benefit of radiomics for head and neck cancer prognosis. Using a large, retrospective dataset of 2,552 patients and a rigorous evaluation framework, we compared 12 different submissions using imaging and clinical data, separately or in combination. The winning approach used non-linear, multitask learning on clinical data and tumour volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction and outperforming models relying on clinical data only, engineered radiomics and deep learning. Combining all submissions in an ensemble model resulted in improved accuracy, with the highest gain from a image-based deep learning model. Our results show the potential of machine learning and simple, informative prognostic factors in combination with large datasets as a tool to guide personalized cancer care.
Machine learning for cloud resources management -- An overview
Venue: arXiv cs.LG 2021
Year: 2021

Nowadays, an important topic that is considered a lot is how to integrate Machine Learning(ML) to cloud resources management. In this study, our goal is to explore the most important cloud resources management issues that have been combined with ML and which present many promising results. To accomplish this, we used chronological charts based on some keywords that we considered important and tried to answer the question: is ML suitable for resources management problems in the cloud? Furthermore, a short discussion takes place on the data that are available and the open challenges on it. A big collection of researches is used to make sensible comparisons between the ML techniques that are used in the different kinds of cloud resources management fields and we propose the most suitable ML model for each field. 1
A Fully Rigorous Proof of the Derivation of Xavier and He's Initialization for Deep ReLU Networks
Venue: arXiv cs.LG 2021
Year: 2021

A fully rigorous proof of the derivation of Xavier/He's initialization for ReLU nets is given.
Modeling Ground-to-Air Path Loss for Millimeter Wave UAV Networks
Venue: arXiv cs.IT 2021
Year: 2021

Path loss is a significant component of wireless communication channel design and analysis and reflects the reduction in a transmitted signal's power density. Due to the differences in the propagation conditions, wireless aerial channels' features differ from those of terrestrial wireless channels; therefore, unmanned aerial vehicle path loss models are often different from conventional terrestrial wireless channel path loss models. A mathematical propagation model is proposed in this paper to estimate the Ground-to-Air path loss between a wireless device and a low-altitude platform using the frequency bands of the millimeter wave. The suggested model of Ground-to-Air path loss will assist academic researchers in formulating several vital problems.
BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks (DNNs) used for brain-computer-interface (BCI) classification are commonly expected to learn general features when trained across a variety of contexts, such that these features could be fine-tuned to specific contexts. While some success is found in such an approach, we suggest that this interpretation is limited and an alternative would better leverage the newly (publicly) available massive EEG datasets. We consider how to adapt techniques and architectures used for language modelling (LM), that appear capable of ingesting awesome amounts of data, towards the development of encephalography modelling (EM) with DNNs in the same vein. We specifically adapt an approach effectively used for automatic speech recognition, which similarly (to LMs) uses a self-supervised training objective to learn compressed representations of raw data signals. After adaptation to EEG, we find that a single pre-trained model is capable of modelling completely novel raw EEG sequences recorded with differing hardware, and different subjects performing different tasks. Furthermore, both the internal representations of this model and the entire architecture can be fine-tuned to a variety of downstream BCI and EEG classification tasks, outperforming prior work in more task-specific (sleep stage classification) self-supervision.
Wiring diagrams as normal forms for computing in symmetric monoidal categories
Venue: arXiv cs.LO 2021
Year: 2021

Applications of category theory often involve symmetric monoidal categories (SMCs), in which abstract processes or operations can be composed in series and parallel. However, in 2020 there remains a dearth of computational tools for working with SMCs. We present an "unbiased" approach to implementing symmetric monoidal categories, based on an operad of directed, acyclic wiring diagrams. Because the interchange law and other laws of a SMC hold identically in a wiring diagram, no rewrite rules are needed to compare diagrams. We discuss the mathematics of the operad of wiring diagrams, as well as its implementation in the software package Catlab.
Domain Adaptation by Topology Regularization
Venue: arXiv cs.CV 2021
Year: 2021

Deep learning has become the leading approach to assisted target recognition. While these methods typically require large amounts of labeled training data, domain adaptation (DA) or transfer learning (TL) enables these algorithms to transfer knowledge from a labelled (source) data set to an unlabelled but related (target) data set of interest. DA enables networks to overcome the distribution mismatch between the source and target that leads to poor generalization in the target domain. DA techniques align these distributions by minimizing a divergence measurement between source and target, making the transfer of knowledge from source to target possible. While these algorithms have advanced significantly in recent years, most do not explicitly leverage global data manifold structure in aligning the source and target. We propose to leverage global data structure by applying a topological data analysis (TDA) technique called persistent homology to TL. In this paper, we examine the use of persistent homology in a domain adversarial (DAd) convolutional neural network (CNN) architecture. The experiments show that aligning persistence alone is insufficient for transfer, but must be considered along with the lifetimes of the topological singularities. In addition, we found that longer lifetimes indicate robust discriminative features and more favorable structure in data. We found that existing divergence minimization based approaches to DA improve the topological structure, as indicated over a baseline without these regularization techniques. We hope these experiments highlight how topological structure can be leveraged to boost performance in TL tasks.
On Mutual Information Analysis of Infectious Disease Transmission via Particle Propagation
Venue: arXiv cs.IT 2021
Year: 2021

Besides mimicking bio-chemical and multi-scale communication mechanisms, molecular communication forms a theoretical framework for virus infection processes. Towards this goal, aerosol and droplet transmission has recently been modeled as a multiuser scenario. In this letter, the "infection performance" is evaluated by means of a mutual information analysis, and by an even simpler probabilistic performance measure which is closely related to absorbed viruses. The so-called infection rate depends on the distribution of the channel input events as well as on the transition probabilities between channel input and output events. The infection rate is investigated analytically for five basic discrete memoryless channel models. Numerical results for the transition probabilities are obtained by Monte Carlo simulations for pathogen-laden particle transmission in four typical indoor environments: two-person office, corridor, classroom, and bus. Particle transfer contributed significantly to infectious diseases like SARS-CoV-2 and influenza.
tf.data: A Machine Learning Data Processing Framework
Venue: arXiv cs.LG 2021
Year: 2021

Training machine learning models requires feeding input data for models to ingest. Input pipelines for machine learning jobs are often challenging to implement efficiently as they require reading large volumes of data, applying complex transformations, and transferring data to hardware accelerators while overlapping computation and communication to achieve optimal performance. We present tf.data, a framework for building and executing efficient input pipelines for machine learning jobs. The tf.data API provides operators which can be parameterized with user-defined computation, composed, and reused across different machine learning domains. These abstractions allow users to focus on the application logic of data processing, while tf.data's runtime ensures that pipelines run efficiently. We demonstrate that input pipeline performance is critical to the end-to-end training time of state-of-the-art machine learning models. tf.data delivers the high performance required, while avoiding the need for manual tuning of performance knobs. We show that tf.data features, such as parallelism, caching, static optimizations, and non-deterministic execution are essential for high performance. Finally, we characterize machine learning input pipelines for millions of jobs that ran in Google's fleet, showing that input data processing is highly diverse and consumes a significant fraction of job resources. Our analysis motivates future research directions, such as sharing computation across jobs and pushing data projection to the storage layer.
Differential Privacy Meets Federated Learning under Communication Constraints
Venue: arXiv cs.LG 2021
Year: 2021

The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, model compression, partial device participation, and periodic aggregation, at the cost of increased training variance. Different from traditional distributed learning systems, federated learning suffers from data heterogeneity (since the devices sample their data from possibly different distributions), which induces additional variance among devices during training. Various variance-reduced training algorithms have been introduced to combat the effects of data heterogeneity, while they usually cost additional communication resources to deliver necessary control information. Additionally, data privacy remains a critical issue in FL, and thus there have been attempts at bringing Differential Privacy to this framework as a mediator between utility and privacy requirements. This paper investigates the trade-offs between communication costs and training variance under a resource-constrained federated system theoretically and experimentally, and how communication reduction techniques interplay in a differentially private setting. The results provide important insights into designing practical privacy-aware federated learning systems.
Revisiting Non-Specific Syndromic Surveillance
Venue: arXiv cs.LG 2021
Year: 2021

Infectious disease surveillance is of great importance for the prevention of major outbreaks. Syndromic surveillance aims at developing algorithms which can detect outbreaks as early as possible by monitoring data sources which allow to capture the occurrences of a certain disease. Recent research mainly focuses on the surveillance of specific, known diseases, putting the focus on the definition of the disease pattern under surveillance. Until now, only little effort has been devoted to what we call non-specific syndromic surveillance, i.e., the use of all available data for detecting any kind of outbreaks, including infectious diseases which are unknown beforehand. In this work, we revisit published approaches for non-specific syndromic surveillance and present a set of simple statistical modeling techniques which can serve as benchmarks for more elaborate machine learning approaches. Our experimental comparison on established synthetic data and real data in which we injected synthetic outbreaks shows that these benchmarks already achieve very competitive results and often outperform more elaborate algorithms.
An Experimental Analysis of Attack Classification Using Machine Learning in IoT Networks
Venue: arXiv cs.CR 2021
Year: 2021

In recent years, there has been a massive increase in the amount of Internet of Things (IoT) devices as well as the data generated by such devices. The participating devices in IoT networks can be problematic due to their resource-constrained nature, and integrating security on these devices is often overlooked. This has resulted in attackers having an increased incentive to target IoT devices. As the number of attacks possible on a network increases, it becomes more difficult for traditional intrusion detection systems (IDS) to cope with these attacks efficiently. In this paper, we highlight several machine learning (ML) methods such as k-nearest neighbour (KNN), support vector machine (SVM), decision tree (DT), naive Bayes (NB), random forest (RF), artificial neural network (ANN), and logistic regression (LR) that can be used in IDS. In this work, ML algorithms are compared for both binary and multi-class classification on Bot-IoT dataset. Based on several parameters such as accuracy, precision, recall, F1 score, and log loss, we experimentally compared the aforementioned ML algorithms. In the case of HTTP distributed denial-of-service (DDoS) attack, the accuracy of RF is 99%. Furthermore, other simulation results-based precision, recall, F1 score, and log loss metric reveal that RF outperforms on all types of attacks in binary classification. However, in multi-class classification, KNN outperforms other ML algorithms with an accuracy of 99%, which is 4% higher than RF.
Evaluating Complex Queries on Streaming Graphs
Venue: arXiv cs.DB 2021
Year: 2021

We study the problem of evaluating persistent queries over streaming graphs in a principled fashion. These queries need to be evaluated over unbounded and very high speed graph streams. We define a streaming graph data model and query model incorporating navigational queries, subgraph queries and paths as first-class citizens. To support this full-fledged query model we develop a streaming graph algebra that describes the precise semantics of persistent graph queries with their complex constructs. We present transformation rules and describe query formulation and plan generation for persistent graph queries over streaming graphs. Our implementation of a streaming graph query processor shows the feasibility of our approach and allows us to gauge the high performance gains obtained for query processing over streaming graphs.
Personalization and Recommendation Technologies for MaaS
Venue: arXiv cs.IR 2021
Year: 2021

Over the last few years, MaaS has been extensively studied and evolved into offering a multitude of mobility services that continuously increase, from alternative car or bike-sharing modes to autonomous vehicles, that aspire to become a part of this novel ecosystem. MaaS provides end-users with multimodal, integrated, and digital mobility solutions, including a multitude of different choices able to cover users specific needs in a personalized manner. This practically leads to a range of novel MaaS products, that may have complex structures and the challenge of matching them to user preferences and needs so that suitable products can be provided to end-users. Moreover, in the everyday use of MaaS, travelers require support to identify routes to reach their destination that adhere to their personal preferences and are aligned to the MaaS product they have purchased. This paper tackles these two user-centric challenges by exploiting state-of-the-art techniques from the field of Personalization and Recommendation systems and integrating them in MaaS platforms and route planning applications.
Community Detection in the Stochastic Block Model by Mixed Integer Programming
Venue: arXiv cs.SI 2021
Year: 2021

The Degree-Corrected Stochastic Block Model (DCSBM) is a popular model to generate random graphs with community structure given an expected degree sequence. The standard approach of community detection based on the DCSBM is to search for the model parameters that are the most likely to have produced the observed network data through maximum likelihood estimation (MLE). Current techniques for the MLE problem are heuristics, and therefore do not guarantee convergence to the optimum. We present mathematical programming formulations and exact solution methods that can provably find the model parameters and community assignments of maximum likelihood given an observed graph. We compare these exact methods with classical heuristic algorithms based on expectation-maximization (EM). The solutions given by exact methods give us a principled way of measuring the experimental performance of classical heuristics and comparing different variations thereof.
Catching up with Method and Process Practice: An Industry-Informed Baseline for Researchers
Venue: arXiv cs.SE 2021
Year: 2021

Software development methods are usually not applied by the book. Companies are under pressure to continuously deploy software products that meet market needs and stakeholders' requests. To implement efficient and effective development processes, companies utilize multiple frameworks, methods and practices, and combine these into hybrid methods. A common combination contains a rich management framework to organize and steer projects complemented with a number of smaller practices providing the development teams with tools to complete their tasks. In this paper, based on 732 data points collected through an international survey, we study the software development process use in practice. Our results show that 76.8% of the companies implement hybrid methods. Company size as well as the strategy in devising and evolving hybrid methods affect the suitability of the chosen process to reach company or project goals. Our findings show that companies that combine planned improvement programs with process evolution can increase their process' suitability by up to 5%.
The significance of user-defined identifiers in Java source code authorship identification
Venue: arXiv cs.SE 2021
Year: 2021

When writing source code, programmers have varying levels of freedom when it comes to the creation and use of identifiers. Do they habitually use the same identifiers, names that are different to those used by others? Is it then possible to tell who the author of a piece of code is by examining these identifiers? If so, can we use the presence or absence of identifiers to assist in correctly classifying programs to authors? Is it possible to hide the provenance of programs by identifier renaming? In this study, we assess the importance of three types of identifiers in source code author classification for two different Java program data sets. We do this through a sequence of experiments in which we disguise one type of identifier at a time. These experiments are performed using as a tool the Source Code Author Profiles (SCAP) method. The results show that, although identifiers when examined as a whole do not seem to reflect program authorship for these data sets, when examined separately there is evidence that class names do signal the author of the program. In contrast, simple variables and method names used in Java programs do not appear to reflect program authorship. On the contrary, our analysis suggests that such identifiers are so common as to mask authorship. We believe that these results have applicability in relation to the robustness of code plagiarism analysis and that the underlying methods could be valuable in cases of litigation arising from disputes over program authorship.
Multi-Threshold Attention U-Net (MTAU) based Model for Multimodal Brain Tumor Segmentation in MRI scans
Venue: arXiv cs.CV 2021
Year: 2021

Gliomas are one of the most frequent brain tumors and are classified into high grade and low grade gliomas. The segmentation of various regions such as tumor core, enhancing tumor etc. plays an important role in determining severity and prognosis. Here, we have developed a multi-threshold model based on attention U-Net for identification of various regions of the tumor in magnetic resonance imaging (MRI). We propose a multi-path segmentation and built three separate models for the different regions of interest. The proposed model achieved mean Dice Coefficient of 0.59, 0.72, and 0.61 for enhancing tumor, whole tumor and tumor core respectively on the training dataset. The same model gave mean Dice Coefficient of 0.57, 0.73, and 0.61 on the validation dataset and 0.59, 0.72, and 0.57 on the test dataset.
Open problems in cross-chain protocols
Venue: arXiv cs.CR 2021
Year: 2021

Blockchain interoperability is a prominent research field which aims to build bridges between otherwise isolated blockchains. With advances in cryptography, novel protocols are published by academia and applied in different applications and products in the industry. In theory, these innovative protocols provide strong privacy and security guarantees by including formal proofs. However, pure theoretical work often lacks the perspective of real world applications. In this work, we describe a number of hardly researched problems which developers encounter when building cross-chain products.
lrsarith: a small fixed/hybrid arithmetic C library
Venue: arXiv cs.MS 2021
Year: 2021

We describe lrsarith which is a small fixed precision and hybrid arithmetic C library for integers and rationals that we developed for use in the lrslib library for polyhedral computation. Using a generic set of operations, a program can be compiled with either 64-bit or 128-bit (if available) fixed precision, with an extended precision library such as GMP or the built-in MP routines. A simple scheme checks for overflow and either terminates the program or, in hybrid mode, changes to a higher precision arithmetic. Implementing these arithmetics in lrslib resulted in only minimal changes to the original code. We give computational results using lrs and mplrs, vertex/facet enumeration codes in lrslib, using 64 and 128 bit fixed integer arithmetic with and without overflow checking, GMP arithmetic, lrsarith hybrid arithmetic with both GMP and MP, and FLINT hybrid arithmetic. We give a small self-contained example C program using the lrsarith package in both fixed precision and hybrid mode.