OpenRaiser/Intern-Atlas huggingface dataset:
Link Partitioning on Simplicial Complexes Using Higher-Order Laplacians
Venue: arXiv cs.SI 2022
Year: 2022

Link partitioning is a popular approach in network science used for discovering overlapping communities by identifying clusters of strongly connected links. Current link partitioning methods are specifically designed for networks modelled by graphs representing pairwise relationships. Therefore, these methods are incapable of utilizing higher-order information about group interactions in network data which is increasingly available. Simplicial complexes extend the dyadic model of graphs and can model polyadic relationships which are ubiquitous and crucial in many complex social and technological systems. In this paper, we introduce a link partitioning method that leverages higher-order (i.e. triadic and higher) information in simplicial complexes for better community detection. Our method utilizes a novel random walk on links of simplicial complexes defined by the higher-order Laplacian--a generalization of the graph Laplacian that incorporates polyadic relationships of the network. We transform this random walk into a graph-based random walk on a lifted line graph--a dual graph in which links are nodes while nodes and higher-order connections are links--and optimize for the standard notion of modularity. We show that our method is guaranteed to provide interpretable link partitioning results under mild assumptions. We also offer new theoretical results on the spectral properties of simplicial complexes by studying the spectrum of the link random walk. Experiment results on real-world community detection tasks show that our higher-order approach significantly outperforms existing graph-based link partitioning methods.
Privacy-Patterns for IoT Application Developers
Venue: arXiv cs.CR 2022
Year: 2022

Designing Internet of things (IoT) applications (apps) is challenging due to the heterogeneous nature of the systems on which these apps are deployed. Personal data, often classified as sensitive, may be collected and analysed by IoT apps, where data privacy laws are expected to protect such information. Various approaches already exist to support privacy-by-design (PbD) schemes, enabling developers to take data privacy into account at the design phase of application development. However, developers are not widely adopting these approaches because of understandability and interpretation challenges. A limited number of tools currently exist to assist developers in this context -- leading to our proposal for "PARROT" (PrivAcy by design tool foR inteRnet Of Things). PARROT supports a number of techniques to enable PbD techniques to be more widely used. We present the findings of a controlled study and discuss how this privacy-preserving tool increases the ability of IoT developers to apply privacy laws (such as GDPR) and privacy patterns. Our students demonstrate that the PARROT prototype tool increases the awareness of privacy requirements in design and increases the likelihood of the subsequent design to be more cognisant of data privacy requirements.
Benchmarking Learnt Radio Localisation under Distribution Shift
Venue: arXiv cs.LG 2022
Year: 2022

Deploying radio frequency (RF) localisation systems invariably entails non-trivial effort, particularly for the latest learning-based breeds. There has been little prior work on characterising and comparing how learnt localiser networks can be deployed in the field under real-world RF distribution shifts. In this paper, we present RadioBench: a suite of 8 learnt localiser nets from the state-of-the-art to study and benchmark their real-world deployability, utilising five novel industry-grade datasets. We train 10k models to analyse the inner workings of these learnt localiser nets and uncover their differing behaviours across three performance axes: (i) learning, (ii) proneness to distribution shift, and (iii) localisation. We use insights gained from this analysis to recommend best practices for the deployability of learning-based RF localisation under practical constraints.
PreprintMatch: a tool for preprint publication detection applied to analyze global inequities in scientific publishing
Venue: arXiv cs.DL 2022
Year: 2022

Preprints, versions of scientific manuscripts that precede peer review, are growing in popularity. They offer an opportunity to democratize and accelerate research, as they have no publication costs or a lengthy peer review process. Preprints are often later published in peer-reviewed venues, but these publications and the original preprints are frequently not linked in any way. To this end, we developed a tool, PreprintMatch, to find matches between preprints and their corresponding published papers, if they exist. This tool outperforms existing techniques to match preprints and papers, both on matching performance and speed. PreprintMatch was applied to search for matches between preprints (from bioRxiv and medRxiv), and PubMed. The preliminary nature of preprints offers a unique perspective into scientific projects at a relatively early stage, and with better matching between preprint and paper, we explored questions related to research inequity. We found that preprints from low income countries are published as peer-reviewed papers at a lower rate than high income countries (39.6\% and 61.1\%, respectively), and our data is consistent with previous work that cite a lack of resources, lack of stability, and policy choices to explain this discrepancy. Preprints from low income countries were also found to be published quicker (178 vs 203 days) and with less title, abstract, and author similarity to the published version compared to high income countries. Low income countries add more authors from the preprint to the published version than high income countries (0.42 authors vs 0.32, respectively), a practice that is significantly more frequent in China compared to similar countries. Finally, we find that some publishers publish work with authors from lower income countries more frequently than others. PreprintMatch is available at \url{https://github.com/PeterEckmann1/preprint-match}.
Split Federated Learning on Micro-controllers: A Keyword Spotting Showcase
Venue: arXiv cs.LG 2022
Year: 2022

Nowadays, AI companies improve service quality by aggressively collecting users' data generated by edge devices, which jeopardizes data privacy. To prevent this, Federated Learning is proposed as a private learning scheme, using which users can locally train the model without collecting users' raw data to servers. However, for machine-learning applications on edge devices that have hard memory constraints, implementing a large model using FL is infeasible. To meet the memory requirement, a recent collaborative learning scheme named split federal learning is a potential solution since it keeps a small model on the device and keeps the rest of the model on the server. In this work, we implement a simply SFL framework on the Arduino board and verify its correctness on the Chinese digits audio dataset for keyword spotting application with over 90% accuracy. Furthermore, on the English digits audio dataset, our SFL implementation achieves 13.89% higher accuracy compared to a state-of-the-art FL implementation.
Infectious Probability Analysis on COVID-19 Spreading with Wireless Edge Networks
Venue: arXiv cs.SI 2022
Year: 2022

The emergence of infectious disease COVID-19 has challenged and changed the world in an unprecedented manner. The integration of wireless networks with edge computing (namely wireless edge networks) brings opportunities to address this crisis. In this paper, we aim to investigate the prediction of the infectious probability and propose precautionary measures against COVID-19 with the assistance of wireless edge networks. Due to the availability of the recorded detention time and the density of individuals within a wireless edge network, we propose a stochastic geometry-based method to analyze the infectious probability of individuals. The proposed method can well keep the privacy of individuals in the system since it does not require to know the location or trajectory of each individual. Moreover, we also consider three types of mobility models and the static model of individuals. Numerical results show that analytical results well match with simulation results, thereby validating the accuracy of the proposed model. Moreover, numerical results also offer many insightful implications. Thereafter, we also offer a number of countermeasures against the spread of COVID-19 based on wireless edge networks. This study lays the foundation toward predicting the infectious risk in realistic environment and points out directions in mitigating the spread of infectious diseases with the aid of wireless edge networks.
Point Cloud Recognition with Position-to-Structure Attention Transformers
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we present Position-to-Structure Attention Transformers (PS-Former), a Transformer-based algorithm for 3D point cloud recognition. PS-Former deals with the challenge in 3D point cloud representation where points are not positioned in a fixed grid structure and have limited feature description (only 3D coordinates ($x, y, z$) for scattered points). Existing Transformer-based architectures in this domain often require a pre-specified feature engineering step to extract point features. Here, we introduce two new aspects in PS-Former: 1) a learnable condensation layer that performs point downsampling and feature extraction; and 2) a Position-to-Structure Attention mechanism that recursively enriches the structural information with the position attention branch. Compared with the competing methods, while being generic with less heuristics feature designs, PS-Former demonstrates competitive experimental results on three 3D point cloud tasks including classification, part segmentation, and scene segmentation.
Learning to Act: Novel Integration of Algorithms and Models for Epidemic Preparedness
Venue: arXiv cs.LG 2022
Year: 2022

In this work we present a framework which may transform research and praxis in epidemic planning. Introduced in the context of the ongoing COVID-19 pandemic, we provide a concrete demonstration of the way algorithms may learn from epidemiological models to scale their value for epidemic preparedness. Our contributions in this work are two fold: 1) a novel platform which makes it easy for decision making stakeholders to interact with epidemiological models and algorithms developed within the Machine learning community, and 2) the release of this work under the Apache-2.0 License. The objective of this paper is not to look closely at any particular models or algorithms, but instead to highlight how they can be coupled and shared to empower evidence-based decision making.
Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planning
Venue: arXiv cs.LG 2022
Year: 2022

We study the problem of episodic reinforcement learning in continuous state-action spaces with unknown rewards and transitions. Specifically, we consider the setting where the rewards and transitions are modeled using parametric bilinear exponential families. We propose an algorithm, BEF-RLSVI, that a) uses penalized maximum likelihood estimators to learn the unknown parameters, b) injects a calibrated Gaussian noise in the parameter of rewards to ensure exploration, and c) leverages linearity of the exponential family with respect to an underlying RKHS to perform tractable planning. We further provide a frequentist regret analysis of BEF-RLSVI that yields an upper bound of $\tilde{\mathcal{O}}(\sqrt{d^3H^3K})$, where $d$ is the dimension of the parameters, $H$ is the episode length, and $K$ is the number of episodes. Our analysis improves the existing bounds for the bilinear exponential family of MDPs by $\sqrt{H}$ and removes the handcrafted clipping deployed in existing \RLSVI-type algorithms. Our regret bound is order-optimal with respect to $H$ and $K$.
WUDA: Unsupervised Domain Adaptation Based on Weak Source Domain Labels
Venue: arXiv cs.CV 2022
Year: 2022

Unsupervised domain adaptation (UDA) for semantic segmentation addresses the cross-domain problem with fine source domain labels. However, the acquisition of semantic labels has always been a difficult step, many scenarios only have weak labels (e.g. bounding boxes). For scenarios where weak supervision and cross-domain problems coexist, this paper defines a new task: unsupervised domain adaptation based on weak source domain labels (WUDA). To explore solutions for this task, this paper proposes two intuitive frameworks: 1) Perform weakly supervised semantic segmentation in the source domain, and then implement unsupervised domain adaptation; 2) Train an object detection model using source domain data, then detect objects in the target domain and implement weakly supervised semantic segmentation. We observe that the two frameworks behave differently when the datasets change. Therefore, we construct dataset pairs with a wide range of domain shifts and conduct extended experiments to analyze the impact of different domain shifts on the two frameworks. In addition, to measure domain shift, we apply the metric representation shift to urban landscape image segmentation for the first time. The source code and constructed datasets are available at \url{https://github.com/bupt-ai-cz/WUDA}.
Common Vulnerability Scoring System Prediction based on Open Source Intelligence Information Sources
Venue: arXiv cs.CR 2022
Year: 2022

The number of newly published vulnerabilities is constantly increasing. Until now, the information available when a new vulnerability is published is manually assessed by experts using a Common Vulnerability Scoring System (CVSS) vector and score. This assessment is time consuming and requires expertise. Various works already try to predict CVSS vectors or scores using machine learning based on the textual descriptions of the vulnerability to enable faster assessment. However, for this purpose, previous works only use the texts available in databases such as National Vulnerability Database. With this work, the publicly available web pages referenced in the National Vulnerability Database are analyzed and made available as sources of texts through web scraping. A Deep Learning based method for predicting the CVSS vector is implemented and evaluated. The present work provides a classification of the National Vulnerability Database's reference texts based on the suitability and crawlability of their texts. While we identified the overall influence of the additional texts is negligible, we outperformed the state-of-the-art with our Deep Learning prediction models.
Security and Privacy Concerns in Cloud-based Scientific and Business Workflows: A Systematic Review
Venue: arXiv cs.CR 2022
Year: 2022

Today, the number of data-intensive and compute-intensive applications like business and scientific workflows has dramatically increased, which made cloud computing more popular in the matter of delivering a large amount of computing resources on demand. On the other hand, security is a critical issue affecting the wide adoption of cloud technologies, especially for workflows that are mostly dealing with sensitive data and tasks. In this paper, we carry out a review of the state-of-the-art on how security and privacy concerns in scientific and business workflows in cloud environments are being addressed and identify the limitations and gaps in the current body of knowledge in this area. In this extensive literature review, we first present a classification of the state-of-the-art security solutions organized according to the phases of the workflow life cycle they target. Based on our findings, we provide a detailed review and classification of the most relevant available literature focusing on the execution, monitoring, and adaptation phases of workflows. Finally, we present a list of open research issues related to the security of cloud-based workflows and discuss them.
Feature Importance for Time Series Data: Improving KernelSHAP
Venue: arXiv cs.LG 2022
Year: 2022

Feature importance techniques have enjoyed widespread attention in the explainable AI literature as a means of determining how trained machine learning models make their predictions. We consider Shapley value based approaches to feature importance, applied in the context of time series data. We present closed form solutions for the SHAP values of a number of time series models, including VARMAX. We also show how KernelSHAP can be applied to time series tasks, and how the feature importances that come from this technique can be combined to perform "event detection". Finally, we explore the use of Time Consistent Shapley values for feature importance.
Game Theoretic Rating in N-player general-sum games with Equilibria
Venue: arXiv cs.GT 2022
Year: 2022

Rating strategies in a game is an important area of research in game theory and artificial intelligence, and can be applied to any real-world competitive or cooperative setting. Traditionally, only transitive dependencies between strategies have been used to rate strategies (e.g. Elo), however recent work has expanded ratings to utilize game theoretic solutions to better rate strategies in non-transitive games. This work generalizes these ideas and proposes novel algorithms suitable for N-player, general-sum rating of strategies in normal-form games according to the payoff rating system. This enables well-established solution concepts, such as equilibria, to be leveraged to efficiently rate strategies in games with complex strategic interactions, which arise in multiagent training and real-world interactions between many agents. We empirically validate our methods on real world normal-form data (Premier League) and multiagent reinforcement learning agent evaluation.
Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we consider privacy aspects of wireless federated learning (FL) with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. By exploiting the waveform superposition property of multiple access channels, OtA FL enables the users to transmit their updates simultaneously with linear processing techniques, which improves resource efficiency. However, this setting is vulnerable to privacy leakage since an adversary node can hear directly the uncoded message. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy due to the reduced signal-to-noise ratio. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server at the same time. More explicitly, spatially correlated perturbations are added to the gradient vectors at the users before transmission. Using the zero-sum property of the correlated perturbations, the side effect of the added perturbation on the aggregated gradients at the edge server can be minimized. In the meanwhile, the added perturbation will not be canceled out at the adversary, which prevents privacy leakage. Theoretical analysis of the perturbation covariance matrix, differential privacy, and model convergence is provided, based on which an optimization problem is formulated to jointly design the covariance matrix and the power scaling factor to balance between privacy protection and convergence performance. Simulation results validate the correlated perturbation approach can provide strong defense ability while guaranteeing high learning accuracy.
On the Use of Deep Learning in Software Defect Prediction
Venue: arXiv cs.SE 2022
Year: 2022

Context: Automated software defect prediction (SDP) methods are increasingly applied, often with the use of machine learning (ML) techniques. Yet, the existing ML-based approaches require manually extracted features, which are cumbersome, time consuming and hardly capture the semantic information reported in bug reporting tools. Deep learning (DL) techniques provide practitioners with the opportunities to automatically extract and learn from more complex and high-dimensional data. Objective: The purpose of this study is to systematically identify, analyze, summarize, and synthesize the current state of the utilization of DL algorithms for SDP in the literature. Method: We systematically selected a pool of 102 peer-reviewed studies and then conducted a quantitative and qualitative analysis using the data extracted from these studies. Results: Main highlights include: (1) most studies applied supervised DL; (2) two third of the studies used metrics as an input to DL algorithms; (3) Convolutional Neural Network is the most frequently used DL algorithm. Conclusion: Based on our findings, we propose to (1) develop more comprehensive DL approaches that automatically capture the needed features; (2) use diverse software artifacts other than source code; (3) adopt data augmentation techniques to tackle the class imbalance problem; (4) publish replication packages.
Imagen Video: High Definition Video Generation with Diffusion Models
Venue: arXiv cs.CV 2022
Year: 2022

We present Imagen Video, a text-conditional video generation system based on a cascade of video diffusion models. Given a text prompt, Imagen Video generates high definition videos using a base video generation model and a sequence of interleaved spatial and temporal video super-resolution models. We describe how we scale up the system as a high definition text-to-video model including design decisions such as the choice of fully-convolutional temporal and spatial super-resolution models at certain resolutions, and the choice of the v-parameterization of diffusion models. In addition, we confirm and transfer findings from previous work on diffusion-based image generation to the video generation setting. Finally, we apply progressive distillation to our video models with classifier-free guidance for fast, high quality sampling. We find Imagen Video not only capable of generating videos of high fidelity, but also having a high degree of controllability and world knowledge, including the ability to generate diverse videos and text animations in various artistic styles and with 3D object understanding. See https://imagen.research.google/video/ for samples.
A Reputation System for Market Security and Equity
Venue: arXiv cs.MA 2022
Year: 2022

We simulate a reputation system in a market to optimise the balance between market security and market equity. We introduce a method of using a reputation system that will stabilise the distribution of wealth in a market in a fair manner. We also introduce metrics of a modified Gini that takes production quality into account, a way to use a weighted Pearson as a tool to optimise balance.