OpenRaiser/Intern-Atlas huggingface dataset:
A Multifaceted Benchmarking of Synthetic Electronic Health Record Generation Models
Venue: arXiv cs.LG 2022
Year: 2022

Synthetic health data have the potential to mitigate privacy concerns when sharing data to support biomedical research and the development of innovative healthcare applications. Modern approaches for data generation based on machine learning, generative adversarial networks (GAN) methods in particular, continue to evolve and demonstrate remarkable potential. Yet there is a lack of a systematic assessment framework to benchmark methods as they emerge and determine which methods are most appropriate for which use cases. In this work, we introduce a generalizable benchmarking framework to appraise key characteristics of synthetic health data with respect to utility and privacy metrics. We apply the framework to evaluate synthetic data generation methods for electronic health records (EHRs) data from two large academic medical centers with respect to several use cases. The results illustrate that there is a utility-privacy tradeoff for sharing synthetic EHR data. The results further indicate that no method is unequivocally the best on all criteria in each use case, which makes it evident why synthetic data generation methods need to be assessed in context.
Geometric Interaction Augmented Graph Collaborative Filtering
Venue: arXiv cs.IR 2022
Year: 2022

Graph-based collaborative filtering is capable of capturing the essential and abundant collaborative signals from the high-order interactions, and thus received increasingly research interests. Conventionally, the embeddings of users and items are defined in the Euclidean spaces, along with the propagation on the interaction graphs. Meanwhile, recent works point out that the high-order interactions naturally form up the tree-likeness structures, which the hyperbolic models thrive on. However, the interaction graphs inherently exhibit the hybrid and nested geometric characteristics, while the existing single geometry-based models are inadequate to fully capture such sophisticated topological patterns. In this paper, we propose to model the user-item interactions in a hybrid geometric space, in which the merits of Euclidean and hyperbolic spaces are simultaneously enjoyed to learn expressive representations. Experimental results on public datasets validate the effectiveness of our proposal.
Automatic Classification of Bug Reports Based on Multiple Text Information and Reports' Intention
Venue: arXiv cs.SE 2022
Year: 2022

With the rapid growth of software scale and complexity, a large number of bug reports are submitted to the bug tracking system. In order to speed up defect repair, these reports need to be accurately classified so that they can be sent to the appropriate developers. However, the existing classification methods only use the text information of the bug report, which leads to their low performance. To solve the above problems, this paper proposes a new automatic classification method for bug reports. The innovation is that when categorizing bug reports, in addition to using the text information of the report, the intention of the report (i.e. suggestion or explanation) is also considered, thereby improving the performance of the classification. First, we collect bug reports from four ecosystems (Apache, Eclipse, Gentoo, Mozilla) and manually annotate them to construct an experimental data set. Then, we use Natural Language Processing technology to preprocess the data. On this basis, BERT and TF-IDF are used to extract the features of the intention and the multiple text information. Finally, the features are used to train the classifiers. The experimental result on five classifiers (including K-Nearest Neighbor, Naive Bayes, Logistic Regression, Support Vector Machine, and Random Forest) show that our proposed method achieves better performance and its F-Measure achieves from 87.3% to 95.5%.
In-Hand Pose Estimation and Pin Inspection for Insertion of Through-Hole Components
Venue: arXiv cs.CV 2022
Year: 2022

The insertion of through-hole components is a difficult task. As the tolerances of the holes are very small, minor errors in the insertion will result in failures. These failures can damage components and will require manual intervention for recovery. Errors can occur both from imprecise object grasps and bent pins. Therefore, it is important that a system can accurately determine the object's position and reject components with bent pins. By utilizing the constraints inherent in the object grasp a method using template matching is able to obtain very precise pose estimates. Methods for pin-checking are also implemented, compared, and a successful method is shown. The set-up is performed automatically, with two novel contributions. A deep learning segmentation of the pins is performed and the inspection pose is found by simulation. From the inspection pose and the segmented pins, the templates for pose estimation and pin check are then generated. To train the deep learning method a dataset of segmented through-hole components is created. The network shows a 97.3 % accuracy on the test set. The pin-segmentation network is also tested on the insertion CAD models and successfully segment the pins. The complete system is tested on three different objects, and experiments show that the system is able to insert all objects successfully. Both by correcting in-hand grasp errors and rejecting objects with bent pins.
Humble Machines: Attending to the Underappreciated Costs of Misplaced Distrust
Venue: arXiv cs.CY 2022
Year: 2022

It is curious that AI increasingly outperforms human decision makers, yet much of the public distrusts AI to make decisions affecting their lives. In this paper we explore a novel theory that may explain one reason for this. We propose that public distrust of AI is a moral consequence of designing systems that prioritize reduction of costs of false positives over less tangible costs of false negatives. We show that such systems, which we characterize as 'distrustful', are more likely to miscategorize trustworthy individuals, with cascading consequences to both those individuals and the overall human-AI trust relationship. Ultimately, we argue that public distrust of AI stems from well-founded concern about the potential of being miscategorized. We propose that restoring public trust in AI will require that systems are designed to embody a stance of 'humble trust', whereby the moral costs of the misplaced distrust associated with false negatives is weighted appropriately during development and use.
BEIKE NLP at SemEval-2022 Task 4: Prompt-Based Paragraph Classification for Patronizing and Condescending Language Detection
Venue: arXiv cs.CL 2022
Year: 2022

PCL detection task is aimed at identifying and categorizing language that is patronizing or condescending towards vulnerable communities in the general media.Compared to other NLP tasks of paragraph classification, the negative language presented in the PCL detection task is usually more implicit and subtle to be recognized, making the performance of common text-classification approaches disappointed. Targeting the PCL detection problem in SemEval-2022 Task 4, in this paper, we give an introduction to our team's solution, which exploits the power of prompt-based learning on paragraph classification. We reformulate the task as an appropriate cloze prompt and use pre-trained Masked Language Models to fill the cloze slot. For the two subtasks, binary classification and multi-label classification, DeBERTa model is adopted and fine-tuned to predict masked label words of task-specific prompts. On the evaluation dataset, for binary classification, our approach achieves an F1-score of 0.6406; for multi-label classification, our approach achieves an macro-F1-score of 0.4689 and ranks first in the leaderboard.
How UMass-FSD Inadvertently Leverages Temporal Bias
Venue: arXiv cs.IR 2022
Year: 2022

First Story Detection describes the task of identifying new events in a stream of documents. The UMass-FSD system is known for its strong performance in First Story Detection competitions. Recently, it has been frequently used as a high accuracy baseline in research publications. We are the first to discover that UMass-FSD inadvertently leverages temporal bias. Interestingly, the discovered bias contrasts previously known biases and performs significantly better. Our analysis reveals an increased contribution of temporally distant documents, resulting from an unusual way of handling incremental term statistics. We show that this form of temporal bias is also applicable to other well-known First Story Detection systems, where it improves the detection accuracy. To provide a more generalizable conclusion and demonstrate that the observed bias is not only an artefact of a particular implementation, we present a model that intentionally leverages a bias on temporal distance. Our model significantly improves the detection effectiveness of state-of-the-art First Story Detection systems.
Lattice Linear Predicate Algorithms for the Constrained Stable Marriage Problem with Ties
Venue: arXiv cs.DS 2022
Year: 2022

We apply Lattice-Linear Predicate Detection Technique to derive parallel and distributed algorithms for various variants of the stable matching problem. These problems are: (a) the constrained stable marriage problem (b) the super stable marriage problem in presence of ties, and (c) the strongly stable marriage in presence of ties. All these problems are solved using the Lattice-Linear Predicate (LLP) algorithm showing its generality. The constrained stable marriage problem is a version of finding the stable marriage in presence of lattice-linear constraints such as ``Peter's regret is less than that of Paul.'' For the constrained stable marriage problem, we present a distributed algorithm that takes $O(n^2)$ messages each of size $O(\log n)$ where $n$ is the number of men in the problem. Our algorithm is completely asynchronous. Our algorithms for the stable marriage problem with ties are also parallel with no synchronization.
Distributed Computations with Layered Resolution
Venue: arXiv cs.DC 2022
Year: 2022

Modern computationally-heavy applications are often time-sensitive, demanding distributed strategies to accelerate them. On the other hand, distributed computing suffers from the bottleneck of slow workers in practice. Distributed coded computing is an attractive solution that adds redundancy such that a subset of distributed computations suffices to obtain the final result. However, the final result is still either obtained within a desired time or not, and for the latter, the resources that are spent are wasted. In this paper, we introduce the novel concept of layered-resolution distributed coded computations such that lower resolutions of the final result are obtained from collective results of the workers -- at an earlier stage than the final result. This innovation makes it possible to have more effective deadline-based systems, since even if a computational job is terminated because of timing, an approximated version of the final result can be released. Based on our theoretical and empirical results, the average execution delay for the first resolution is notably smaller than the one for the final resolution. Moreover, the probability of meeting a deadline is one for the first resolution in a setting where the final resolution exceeds the deadline almost all the time, reducing the success rate of the systems with no layering.
A Secure Dynamic Edge Resource Federation Architecture for Cross-Domain IoT Systems
Venue: arXiv cs.NI 2022
Year: 2022

The fast integration of 5G communication, Artificial Intelligence (AI), and Internet-of-Things (IoT) technologies is envisioned to enable Next Generation Networks (NGNs) for diverse smart services and user-defined applications for Smart Cities. However, it is still challenging to build a scalable and efficient infrastructure that satisfies the various performance, security, and management demands by heterogeneous IoT applications across multiple administrative domains. This paper presents a dynamic edge resource federation architecture, which integrates the concept of network slicing (NS) and blockchain to improve scalability, dynamicity, and security for multi-domain IoT applications. A NS-enabled dynamic edge resource federation framework adopts intelligent mechanisms to support efficient multi-domain service coordination that satisfies diverse Quality of Service (QoS) and security requirements. We propose a Hierarchical Integrated Federated Ledger (HIFL), which aims to guarantee decentralized security and privacy-preserving properties in multi-domain resource orchestration and service re-adjustment. As a secure-by-design solution, HIFL is promising to support efficient, trust and secured end-to-end IoT services. A preliminary proof-of-concept prototype has been implemented for comparing intra- and inter-domain performance expectations.
Human Aspect of Threat Analysis: A Replication
Venue: arXiv cs.SE 2022
Year: 2022

Background: Organizations are experiencing an increasing demand for security-by-design activities (e.g., STRIDE analyses) which require a high manual effort. This situation is worsened by the current lack of diverse (and sufficient) security workforce and inconclusive results from past studies. To date, the deciding human factors (e.g., diversity dimensions) that play a role in threat analysis have not been sufficiently explored. Objective: To address this issue, we plan to conduct a series of exploratory controlled experiments. The main objective is to empirically measure the human-aspects that play a role in threat analysis alongside the more well-known measures of analysis performance. Method: We design the experiments as a differentiated replication of past experiments with STRIDE. The replication design is aimed at capturing some similar measures (e.g., of outcome quality) and additional measures (e.g., diversity dimensions). We plan to conduct the experiments in an academic setting. Limitations: Obtaining a balanced population (e.g., wrt gender) in advanced computer science courses is not realistic. The experiments we plan to conduct with MSc level students will certainly suffer this limitation.
MBSE analysis for energy sustainability improvement in manufacturing industry
Venue: arXiv cs.CL 2022
Year: 2022

With the ever increasing complexity of Industry 4.0 systems, plant energy management systems developed to improve energy sustainability become equally complex. Based on a Model-Based Systems Engineering analysis, this paper aims to provide a general approach to perform holistic development of an autonomous energy management system for manufacturing industries. This Energy Management System (EMS) will be capable of continuously improving its ability to assess, predict, and act, in order to improve by monitoring and controlling the energy sustainability of manufacturing systems. The approach was implemented with the System Modeling Language (SysML).
A replication of a controlled experiment with two STRIDE variants
Venue: arXiv cs.CR 2022
Year: 2022

To avoid costly security patching after software deployment, security-by-design techniques (e.g., STRIDE threat analysis) are adopted in organizations to root out security issues before the system is ever implemented. Despite the global gap in cybersecurity workforce and the high manual effort required for performing threat analysis, organizations are ramping up threat analysis activities. However, past experimental results were inconclusive regarding some performance indicators of threat analysis techniques thus practitioners have little evidence for choosing the technique to adopt. To address this issue, we replicated a controlled experiment with STRIDE. Our study was aimed at measuring and comparing the performance indicators (productivity and precision) of two STRIDE variants (element and interaction). We conclude the paper by comparing our results to the original study.
CIPCaD-Bench: Continuous Industrial Process datasets for benchmarking Causal Discovery methods
Venue: arXiv cs.LG 2022
Year: 2022

Causal relationships are commonly examined in manufacturing processes to support faults investigations, perform interventions, and make strategic decisions. Industry 4.0 has made available an increasing amount of data that enable data-driven Causal Discovery (CD). Considering the growing number of recently proposed CD methods, it is necessary to introduce strict benchmarking procedures on publicly available datasets since they represent the foundation for a fair comparison and validation of different methods. This work introduces two novel public datasets for CD in continuous manufacturing processes. The first dataset employs the well-known Tennessee Eastman simulator for fault detection and process control. The second dataset is extracted from an ultra-processed food manufacturing plant, and it includes a description of the plant, as well as multiple ground truths. These datasets are used to propose a benchmarking procedure based on different metrics and evaluated on a wide selection of CD algorithms. This work allows testing CD methods in realistic conditions enabling the selection of the most suitable method for specific target applications. The datasets are available at the following link: https://github.com/giovanniMen
The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalence
Venue: arXiv cs.LG 2022
Year: 2022

Recently, it has been observed that a transfer learning solution might be all we need to solve many few-shot learning benchmarks -- thus raising important questions about when and how meta-learning algorithms should be deployed. In this paper, we seek to clarify these questions by 1. proposing a novel metric -- the diversity coefficient -- to measure the diversity of tasks in a few-shot learning benchmark and 2. by comparing Model-Agnostic Meta-Learning (MAML) and transfer learning under fair conditions (same architecture, same optimizer, and all models trained to convergence). Using the diversity coefficient, we show that the popular MiniImageNet and CIFAR-FS few-shot learning benchmarks have low diversity. This novel insight contextualizes claims that transfer learning solutions are better than meta-learned solutions in the regime of low diversity under a fair comparison. Specifically, we empirically find that a low diversity coefficient correlates with a high similarity between transfer learning and MAML learned solutions in terms of accuracy at meta-test time and classification layer similarity (using feature based distance metrics like SVCCA, PWCCA, CKA, and OPD). To further support our claim, we find this meta-test accuracy holds even as the model size changes. Therefore, we conclude that in the low diversity regime, MAML and transfer learning have equivalent meta-test performance when both are compared fairly. We also hope our work inspires more thoughtful constructions and quantitative evaluations of meta-learning benchmarks in the future.
Learning Skill-based Industrial Robot Tasks with User Priors
Venue: arXiv cs.RO 2022
Year: 2022

Robot skills systems are meant to reduce robot setup time for new manufacturing tasks. Yet, for dexterous, contact-rich tasks, it is often difficult to find the right skill parameters. One strategy is to learn these parameters by allowing the robot system to learn directly on the task. For a learning problem, a robot operator can typically specify the type and range of values of the parameters. Nevertheless, given their prior experience, robot operators should be able to help the learning process further by providing educated guesses about where in the parameter space potential optimal solutions could be found. Interestingly, such prior knowledge is not exploited in current robot learning frameworks. We introduce an approach that combines user priors and Bayesian optimization to allow fast optimization of robot industrial tasks at robot deployment time. We evaluate our method on three tasks that are learned in simulation as well as on two tasks that are learned directly on a real robot system. Additionally, we transfer knowledge from the corresponding simulation tasks by automatically constructing priors from well-performing configurations for learning on the real system. To handle potentially contradicting task objectives, the tasks are modeled as multi-objective problems. Our results show that operator priors, both user-specified and transferred, vastly accelerate the discovery of rich Pareto fronts, and typically produce final performance far superior to proposed baselines.
Enabling scalable clinical interpretation of ML-based phenotypes using real world data
Venue: arXiv cs.LG 2022
Year: 2022

The availability of large and deep electronic healthcare records (EHR) datasets has the potential to enable a better understanding of real-world patient journeys, and to identify novel subgroups of patients. ML-based aggregation of EHR data is mostly tool-driven, i.e., building on available or newly developed methods. However, these methods, their input requirements, and, importantly, resulting output are frequently difficult to interpret, especially without in-depth data science or statistical training. This endangers the final step of analysis where an actionable and clinically meaningful interpretation is needed.This study investigates approaches to perform patient stratification analysis at scale using large EHR datasets and multiple clustering methods for clinical research. We have developed several tools to facilitate the clinical evaluation and interpretation of unsupervised patient stratification results, namely pattern screening, meta clustering, surrogate modeling, and curation. These tools can be used at different stages within the analysis. As compared to a standard analysis approach, we demonstrate the ability to condense results and optimize analysis time. In the case of meta clustering, we demonstrate that the number of patient clusters can be reduced from 72 to 3 in one example. In another stratification result, by using surrogate models, we could quickly identify that heart failure patients were stratified if blood sodium measurements were available. As this is a routine measurement performed for all patients with heart failure, this indicated a data bias. By using further cohort and feature curation, these patients and other irrelevant features could be removed to increase the clinical meaningfulness. These examples show the effectiveness of the proposed methods and we hope to encourage further research in this field.
On the Age of Status Updates in Unreliable Multi-Source M/G/1 Queueing Systems
Venue: arXiv cs.NI 2022
Year: 2022

The timeliness of status message delivery in communications networks is subjective to time-varying wireless channel transmissions. In this paper, we investigate the age of information (AoI) of each source in a multi-source M/G/1 queueing update system with active server failures. In particular, we adopt the method of supplementary variables to derive a closed-form expression for the average AoI in terms of system parameters, where the server repair time follows a general distribution and the service time of packets generated by independent sources is a general random variable. Numerical results are provided to validate the effectiveness of the proposed packet serving policy under different parametric settings.