OpenRaiser/Intern-Atlas huggingface dataset:
Be Prepared When Network Goes Bad: An Asynchronous View-Change Protocol
Venue: arXiv cs.DC 2021
Year: 2021

The popularity of permissioned blockchain systems demands BFT SMR protocols that are efficient under good network conditions (synchrony) and robust under bad network conditions (asynchrony). The state-of-the-art partially synchronous BFT SMR protocols provide optimal linear communication cost per decision under synchrony and good leaders, but lose liveness under asynchrony. On the other hand, the state-of-the-art asynchronous BFT SMR protocols are live even under asynchrony, but always pay quadratic cost even under synchrony. In this paper, we propose a BFT SMR protocol that achieves the best of both worlds -- optimal linear cost per decision under good networks and leaders, optimal quadratic cost per decision under bad networks, and remains always live.
Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression
Venue: arXiv cs.CV 2021
Year: 2021

Prostate cancer (PCa) is one of the most common and aggressive cancers worldwide. The Gleason score (GS) system is the standard way of classifying prostate cancer and the most reliable method to determine the severity and treatment to follow. The pathologist looks at the arrangement of cancer cells in the prostate and assigns a score on a scale that ranges from 6 to 10. Automatic analysis of prostate whole-slide images (WSIs) is usually addressed as a binary classification problem, which misses the finer distinction between stages given by the GS. This paper presents a probabilistic deep learning ordinal classification method that can estimate the GS from a prostate WSI. Approaching the problem as an ordinal regression task using a differentiable probabilistic model not only improves the interpretability of the results, but also improves the accuracy of the model when compared to conventional deep classification and regression architectures.
GenoML: Automated Machine Learning for Genomics
Venue: arXiv cs.LG 2021
Year: 2021

GenoML is a Python package automating machine learning workflows for genomics (genetics and multi-omics) with an open science philosophy. Genomics data require significant domain expertise to clean, pre-process, harmonize and perform quality control of the data. Furthermore, tuning, validation, and interpretation involve taking into account the biology and possibly the limitations of the underlying data collection, protocols, and technology. GenoML's mission is to bring machine learning for genomics and clinical data to non-experts by developing an easy-to-use tool that automates the full development, evaluation, and deployment process. Emphasis is put on open science to make workflows easily accessible, replicable, and transferable within the scientific community. Source code and documentation is available at https://genoml.com.
Modified Erlang loss system for cognitive wireless networks
Venue: arXiv cs.PF 2021
Year: 2021

This paper considers a modified Erlang loss system for cognitive wireless networks and related applications. A primary user has preemptive priority over secondary users and the primary customer is lost if upon arrival all the channels are used by other primary users. Secondary users cognitively use idle channels and they can wait at an infinite buffer in cases idle channels are not available upon arrival or they are interrupted by primary users. We obtain explicit stability condition for the cases where arrival processes of primary users and secondary users follow Poisson processes and their service times follow two distinct arbitrary distributions. The stability condition is insensitive to the service time distributions and implies the maximal throughout of secondary users. For a special case of exponential service time distributions, we analyze in depth to show the effect of parameters on the delay performance and the mean number of interruptions of secondary users. Our simulations for distributions rather than exponential reveal that the mean number of terminations for secondary users is less sensitive to the service time distribution of primary users.
CLAIMED, a visual and scalable component library for Trusted AI
Venue: arXiv cs.LG 2021
Year: 2021

Deep Learning models are getting more and more popular but constraints on explainability, adversarial robustness and fairness are often major concerns for production deployment. Although the open source ecosystem is abundant on addressing those concerns, fully integrated, end to end systems are lacking in open source. Therefore we provide an entirely open source, reusable component framework, visual editor and execution engine for production grade machine learning on top of Kubernetes, a joint effort between IBM and the University Hospital Basel. It uses Kubeflow Pipelines, the AI Explainability360 toolkit, the AI Fairness360 toolkit and the Adversarial Robustness Toolkit on top of ElyraAI, Kubeflow, Kubernetes and JupyterLab. Using the Elyra pipeline editor, AI pipelines can be developed visually with a set of jupyter notebooks.
PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction
Venue: arXiv cs.CL 2021
Year: 2021

Active research pertaining to the affective phenomenon of empathy and distress is invaluable for improving human-machine interaction. Predicting intensities of such complex emotions from textual data is difficult, as these constructs are deeply rooted in the psychological theory. Consequently, for better prediction, it becomes imperative to take into account ancillary factors such as the psychological test scores, demographic features, underlying latent primitive emotions, along with the text's undertone and its psychological complexity. This paper proffers team PVG's solution to the WASSA 2021 Shared Task on Predicting Empathy and Emotion in Reaction to News Stories. Leveraging the textual data, demographic features, psychological test score, and the intrinsic interdependencies of primitive emotions and empathy, we propose a multi-input, multi-task framework for the task of empathy score prediction. Here, the empathy score prediction is considered the primary task, while emotion and empathy classification are considered secondary auxiliary tasks. For the distress score prediction task, the system is further boosted by the addition of lexical features. Our submission ranked 1$^{st}$ based on the average correlation (0.545) as well as the distress correlation (0.574), and 2$^{nd}$ for the empathy Pearson correlation (0.517).
Conservative Optimistic Policy Optimization via Multiple Importance Sampling
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement Learning (RL) has been able to solve hard problems such as playing Atari games or solving the game of Go, with a unified approach. Yet modern deep RL approaches are still not widely used in real-world applications. One reason could be the lack of guarantees on the performance of the intermediate executed policies, compared to an existing (already working) baseline policy. In this paper, we propose an online model-free algorithm that solves conservative exploration in the policy optimization problem. We show that the regret of the proposed approach is bounded by $\tilde{\mathcal{O}}(\sqrt{T})$ for both discrete and continuous parameter spaces.
Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture
Venue: arXiv cs.LG 2021
Year: 2021

Graph Convolutional Networks (GCNs) are increasingly adopted in large-scale graph-based recommender systems. Training GCN requires the minibatch generator traversing graphs and sampling the sparsely located neighboring nodes to obtain their features. Since real-world graphs often exceed the capacity of GPU memory, current GCN training systems keep the feature table in host memory and rely on the CPU to collect sparse features before sending them to the GPUs. This approach, however, puts tremendous pressure on host memory bandwidth and the CPU. This is because the CPU needs to (1) read sparse features from memory, (2) write features into memory as a dense format, and (3) transfer the features from memory to the GPUs. In this work, we propose a novel GPU-oriented data communication approach for GCN training, where GPU threads directly access sparse features in host memory through zero-copy accesses without much CPU help. By removing the CPU gathering stage, our method significantly reduces the consumption of the host resources and data access latency. We further present two important techniques to achieve high host memory access efficiency by the GPU: (1) automatic data access address alignment to maximize PCIe packet efficiency, and (2) asynchronous zero-copy access and kernel execution to fully overlap data transfer with training. We incorporate our method into PyTorch and evaluate its effectiveness using several graphs with sizes up to 111 million nodes and 1.6 billion edges. In a multi-GPU training setup, our method is 65-92% faster than the conventional data transfer method, and can even match the performance of all-in-GPU-memory training for some graphs that fit in GPU memory.
Maximising the total weight of on-time jobs on parallel machines subject to a conflict graph
Venue: arXiv cs.DS 2021
Year: 2021

The paper considers scheduling on parallel machines under the constraint that some pairs of jobs cannot be processed concurrently. Each job has an associated weight, and all jobs have the same deadline. The objective is to maximise the total weight of on-time jobs. The problem is known to be strongly NP-hard in general. A polynomial-time algorithm for scheduling unit execution time jobs on two machines is proposed. The performance of a broad family of approximation algorithms for scheduling unit execution time jobs on more than two machines is analysed. For the case of arbitrary job processing times, two integer linear programming formulations are proposed and compared with two formulations known from the earlier literature. An iterated variable neighborhood search algorithm is also proposed and evaluated by means of computational experiments.
Cognitive Homeostatic Agents
Venue: arXiv cs.AI 2021
Year: 2021

Human brain has been used as an inspiration for building autonomous agents, but it is not obvious what level of computational description of the brain one should use. This has led to overly opinionated symbolic approaches and overly unstructured connectionist approaches. We propose that using homeostasis as the computational description provides a good compromise. Similar to how physiological homeostasis is the regulation of certain homeostatic variables, cognition can be interpreted as the regulation of certain 'cognitive homeostatic variables'. We present an outline of a Cognitive Homeostatic Agent, built as a hierarchy of physiological and cognitive homeostatic subsystems and describe structures and processes to guide future exploration. We expect this to be a fruitful line of investigation towards building sophisticated artificial agents that can act flexibly in complex environments, and produce behaviors indicating planning, thinking and feelings.
From Quantifying Vagueness To Pan-niftyism
Venue: arXiv cs.AI 2021
Year: 2021

In this short paper, we will introduce a simple model for quantifying philosophical vagueness. There is growing interest in this endeavor to quantify vague concepts of consciousness, agency, etc. We will then discuss some of the implications of this model including the conditions under which the quantification of `nifty' leads to pan-nifty-ism. Understanding this leads to an interesting insight - the reason a framework to quantify consciousness like Integrated Information Theory implies (forms of) panpsychism is because there is favorable structure already implicitly encoded in the construction of the quantification metric.
Onshore to Near-Shore Outsourcing Transitions: Unpacking Tensions
Venue: arXiv cs.SE 2021
Year: 2021

This study is directed towards highlighting tensions of incoming and outgoing vendors during outsourcing in a near-shore context. Incoming-and-outgoing of vendors generate a complex form of relationship in which the participating organizations cooperate and compete simultaneously. It is of great importance to develop knowledge about this kind of relationship typically in the current GSE-related multi-sourcing environment. We carried out a longitudinal case study and utilized data from the 'Novopay' project, which is available in the public domain. This project involved an outgoing New Zealand based vendor and incoming Australian based vendor. The results show that the demand for the same human resources, dependency upon cooperation and collaboration between vendors, reliance on each other system's configurations and utilizing similar strategies by the client, which worked for the previous vendor, generated a set of tensions which needed to be continuously managed throughout the project.
Gaussian processes meet NeuralODEs: A Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents a machine learning framework (GP-NODE) for Bayesian systems identification from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. Moreover, the use of sparsity-promoting priors such as the Finnish Horseshoe for free model parameters enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator-prey systems, systems biology, and a 50-dimensional human motion dynamical system. Taken together, our findings put forth a novel, flexible and robust workflow for data-driven model discovery under uncertainty. All code and data accompanying this manuscript are available online at \url{https://github.com/PredictiveIntelligenceLab/GP-NODEs}.
Progress Report on a Proposed Theory for Software Development
Venue: arXiv cs.SE 2021
Year: 2021

There is growing acknowledgement within the software engineering community that a theory of software development is needed to integrate the myriad methodologies that are currently popular, some of which are based on opposing perspectives. We have been developing such a theory for a number of years. In this position paper, we overview our theory along with progress made thus far. We suggest that, once fully developed, this theory, or one similar to it, may be applied to support situated software development, by providing an overarching model within which software initiatives might be categorised and understood. Such understanding would inevitably lead to greater predictability with respect to outcomes.
Routing algorithms as tools for integrating social distancing with emergency evacuation
Venue: arXiv cs.AI 2021
Year: 2021

One of the lessons from the COVID-19 pandemic is the importance of social distancing, even in challenging circumstances such as pre-hurricane evacuation. To explore the implications of integrating social distancing with evacuation operations, we describe this evacuation process as a Capacitated Vehicle Routing Problem (CVRP) and solve it using a DNN (Deep Neural Network)-based solution (Deep Reinforcement Learning) and a non-DNN solution (Sweep Algorithm). A central question is whether Deep Reinforcement Learning provides sufficient extra routing efficiency to accommodate increased social distancing in a time-constrained evacuation operation. We found that, in comparison to the Sweep Algorithm, Deep Reinforcement Learning can provide decision-makers with more efficient routing. However, the evacuation time saved by Deep Reinforcement Learning does not come close to compensating for the extra time required for social distancing, and its advantage disappears as the emergency vehicle capacity approaches the number of people per household.
Vicinal and categorical domain adaptation
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised domain adaptation aims to learn a task classifier that performs well on the unlabeled target domain, by utilizing the labeled source domain. Inspiring results have been acquired by learning domain-invariant deep features via domain-adversarial training. However, its parallel design of task and domain classifiers limits the ability to achieve a finer category-level domain alignment. To promote categorical domain adaptation (CatDA), based on a joint category-domain classifier, we propose novel losses of adversarial training at both domain and category levels. Since the joint classifier can be regarded as a concatenation of individual task classifiers respectively for the two domains, our design principle is to enforce consistency of category predictions between the two task classifiers. Moreover, we propose a concept of vicinal domains whose instances are produced by a convex combination of pairs of instances respectively from the two domains. Intuitively, alignment of the possibly infinite number of vicinal domains enhances that of original domains. We propose novel adversarial losses for vicinal domain adaptation (VicDA) based on CatDA, leading to Vicinal and Categorical Domain Adaptation (ViCatDA). We also propose Target Discriminative Structure Recovery (TDSR) to recover the intrinsic target discrimination damaged by adversarial feature alignment. We also analyze the principles underlying the ability of our key designs to align the joint distributions. Extensive experiments on several benchmark datasets demonstrate that we achieve the new state of the art.
Compressed Communication Complexity of Hamming Distance
Venue: arXiv cs.DS 2021
Year: 2021

We consider the communication complexity of the Hamming distance of two strings. Bille et al. [SPIRE 2018] considered the communication complexity of the longest common prefix (LCP) problem in the setting where the two parties have their strings in a compressed form, i.e., represented by the Lempel-Ziv 77 factorization (LZ77) with/without self-references. We present a randomized public-coin protocol for a joint computation of the Hamming distance of two strings represented by LZ77 without self-references. While our scheme is heavily based on Bille et al.'s LCP protocol, our complexity analysis is original which uses Crochemore's C-factorization and Rytter's AVL-grammar. As a byproduct, we also show that LZ77 with/without self-references are not monotonic in the sense that their sizes can increase by a factor of 4/3 when a prefix of the string is removed.
A Novel Framework for Threat Analysis of Machine Learning-based Smart Healthcare Systems
Venue: arXiv cs.CR 2021
Year: 2021

Smart healthcare systems (SHSs) are providing fast and efficient disease treatment leveraging wireless body sensor networks (WBSNs) and implantable medical devices (IMDs)-based internet of medical things (IoMT). In addition, IoMT-based SHSs are enabling automated medication, allowing communication among myriad healthcare sensor devices. However, adversaries can launch various attacks on the communication network and the hardware/firmware to introduce false data or cause data unavailability to the automatic medication system endangering the patient's life. In this paper, we propose SHChecker, a novel threat analysis framework that integrates machine learning and formal analysis capabilities to identify potential attacks and corresponding effects on an IoMT-based SHS. Our framework can provide us with all potential attack vectors, each representing a set of sensor measurements to be altered, for an SHS given a specific set of attack attributes, allowing us to realize the system's resiliency, thus the insight to enhance the robustness of the model. We implement SHChecker on a synthetic and a real dataset, which affirms that our framework can reveal potential attack vectors in an IoMT system. This is a novel effort to formally analyze supervised and unsupervised machine learning models for black-box SHS threat analysis.
Moving from Cross-Project Defect Prediction to Heterogeneous Defect Prediction: A Partial Replication Study
Venue: arXiv cs.SE 2021
Year: 2021

Software defect prediction heavily relies on the metrics collected from software projects. Earlier studies often used machine learning techniques to build, validate, and improve bug prediction models using either a set of metrics collected within a project or across different projects. However, techniques applied and conclusions derived by those models are restricted by how identical those metrics are. Knowledge coming from those models will not be extensible to a target project if no sufficient overlapping metrics have been collected in the source projects. To explore the feasibility of transferring knowledge across projects without common labeled metrics, we systematically integrated Heterogeneous Defect Prediction (HDP) by replicating and validating the obtained results. Our main goal is to extend prior research and explore the feasibility of HDP and finally to compare its performance with that of its predecessor, Cross-Project Defect Prediction. We construct an HDP model on different publicly available datasets. Moreover, we propose a new ensemble voting approach in the HDP context to utilize the predictive power of multiple available datasets. The result of our experiment is comparable to that of the original study. However, we also explored the feasibility of HDP in real cases. Our results shed light on the infeasibility of many cases for the HDP algorithm due to its sensitivity to the parameter selection. In general, our analysis gives a deep insight into why and how to perform transfer learning from one domain to another, and in particular, provides a set of guidelines to help researchers and practitioners to disseminate knowledge to the defect prediction domain.
Update the Root of Integrity Tree in Secure Non-Volatile Memory Systems with Low Overhead
Venue: arXiv cs.CR 2021
Year: 2021

Data integrity is important for non-volatile memory (NVM) systems that maintain data even without power. The data integrity in NVM is possibly compromised by integrity attacks, which can be defended against by integrity verification via integrity trees. After NVM system failures and reboots, the integrity tree root is responsible for providing a trusted execution environment. However, the root often becomes a performance bottleneck, since updating the root requires high latency on the write critical path to propagate the modifications from leaf nodes to the root. The root and leaf nodes have to ensure the crash consistency between each other to avoid any update failures that potentially result in misreporting the attacks after system reboots. In this paper, we propose an efficient and low-latency scheme, called SCUE, to directly update the root on the SGX integrity tree (SIT) by overlooking the updates upon the intermediate tree nodes. The idea behind SCUE explores and exploits the observation that only the persistent leaf nodes and root are useful to ensure the integrity after system failures and reboots, due to the loss of the cached intermediate tree nodes. To achieve the crash consistency between root and leaf nodes, we accurately predict the updates upon the root and pre-update the root before the leaf nodes are modified. Moreover, the SIT root is difficult to be reconstructed from the leaf nodes since updating one tree node needs its parent node as input. We use a counter-summing approach to reconstructing the SIT from leaf nodes. Our evaluation results show that compared with the state-of-the-art integrity tree update schemes, our SCUE scheme delivers high performance while ensuring the system integrity.