OpenRaiser/Intern-Atlas huggingface dataset:
Reconstruction of Sets of Strings from Prefix/Suffix Compositions
Venue: arXiv cs.IT 2021
Year: 2021

The problem of reconstructing strings from substring information has found many applications due to its importance in genomic data sequencing and DNA- and polymer-based data storage. One practically important and challenging paradigm requires reconstructing mixtures of strings based on the union of compositions of their prefixes and suffixes, generated by mass spectrometry devices. We describe new coding methods that allow for unique joint reconstruction of subsets of strings selected from a code and provide upper and lower bounds on the asymptotic rate of the underlying codebooks. Our code constructions combine properties of binary Bh and Dyck strings and that can be extended to accommodate missing substrings in the pool. As auxiliary results, we obtain the first known bounds on binary Bh sequences for arbitrary even parameters h, and also describe various error models inherent to mass spectrometry analysis. This paper contains a correction of the prior work by the authors, published in [24]. In particular, the bounds on the prefix codes are now corrected.
Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance
Venue: arXiv cs.CL 2021
Year: 2021

Multilingual language models achieve impressive zero-shot accuracies in many languages in complex tasks such as Natural Language Inference (NLI). Examples in NLI (and equivalent complex tasks) often pertain to various types of sub-tasks, requiring different kinds of reasoning. Certain types of reasoning have proven to be more difficult to learn in a monolingual context, and in the crosslingual context, similar observations may shed light on zero-shot transfer efficiency and few-shot sample selection. Hence, to investigate the effects of types of reasoning on transfer performance, we propose a category-annotated multilingual NLI dataset and discuss the challenges to scale monolingual annotations to multiple languages. We statistically observe interesting effects that the confluence of reasoning types and language similarities have on transfer performance.
Sum-rate Maximization in Uplink CRAN with a Massive MIMO Fronthaul
Venue: arXiv cs.IT 2021
Year: 2021

The limited fronthaul capacity is known to be one of the main problems in cloud radio access networks (CRANs), especially in the wireless fronthaul links. In this paper, we consider the uplink of a CRAN system, where massive multiple-input multiple-output (MIMO) is utilized in the fronthaul link. Considering multi-antenna user equipment (UEs) and multi-antenna remote radio heads (RRHs), we maximize the system sum-rate by jointly optimizing the precoders at the UEs and the quantization noise covariance matrices and transmit powers at the RRHs. To solve the resulting nonconvex problem, an iterative algorithm based on the majorization-minimization (MM) method is proposed. Two schemes at the central unit are considered, namely maximum ratio (MR) and zero-forcing (ZF) combining. Numerical results show that the sum-rate has an asymptotic behaviour with respect to the maximum available power at RRHs and that the MR scheme goes to its asymptote faster than the ZF scheme.
Post-hoc Models for Performance Estimation of Machine Learning Inference
Venue: arXiv cs.CV 2021
Year: 2021

Estimating how well a machine learning model performs during inference is critical in a variety of scenarios (for example, to quantify uncertainty, or to choose from a library of available models). However, the standard accuracy estimate of softmax confidence is not versatile and cannot reliably predict different performance metrics (e.g., F1-score, recall) or the performance in different application scenarios or input domains. In this work, we systematically generalize performance estimation to a diverse set of metrics and scenarios and discuss generalized notions of uncertainty calibration. We propose the use of post-hoc models to accomplish this goal and investigate design parameters, including the model type, feature engineering, and performance metric, to achieve the best estimation quality. Emphasis is given to object detection problems and, unlike prior work, our approach enables the estimation of per-image metrics such as recall and F1-score. Through extensive experiments with computer vision models and datasets in three use cases -- mobile edge offloading, model selection, and dataset shift -- we find that proposed post-hoc models consistently outperform the standard calibrated confidence baselines. To the best of our knowledge, this is the first work to develop a unified framework to address different performance estimation problems for machine learning inference.
Shapley variable importance clouds for interpretable machine learning
Venue: arXiv cs.LG 2021
Year: 2021

Interpretable machine learning has been focusing on explaining final models that optimize performance. The current state-of-the-art is the Shapley additive explanations (SHAP) that locally explains variable impact on individual predictions, and it is recently extended for a global assessment across the dataset. Recently, Dong and Rudin proposed to extend the investigation to models from the same class as the final model that are "good enough", and identified a previous overclaim of variable importance based on a single model. However, this method does not directly integrate with existing Shapley-based interpretations. We close this gap by proposing a Shapley variable importance cloud that pools information across good models to avoid biased assessments in SHAP analyses of final models, and communicate the findings via novel visualizations. We demonstrate the additional insights gain compared to conventional explanations and Dong and Rudin's method using criminal justice and electronic medical records data.
UGV-assisted Wireless Powered Backscatter Communications for Large-Scale IoT Networks
Venue: arXiv cs.IT 2021
Year: 2021

Wireless powered backscatter communications (WPBC) is capable of implementing ultra-low-power communication, thus promising in the Internet of Things (IoT) networks. In practice, however, it is challenging to apply WPBC in large-scale IoT networks because of its short communication range. To address this challenge, this paper exploits an unmanned ground vehicle (UGV) to assist WPBC in large-scale IoT networks. In particular, we investigate the joint design of network planning and dynamic resource allocation of the access point (AP), tag reader, and UGV to minimize the total energy consumption. Also, the AP can operate in either half-duplex (HD) or full-duplex (FD) multiplexing mode. Under HD mode, the optimal cell radius is derived and the optimal power allocation and transmit/receive beamforming are obtained in closed form. Under FD mode, the optimal resource allocation, as well as two suboptimal ones with low computational complexity, is developed. Simulation results disclose that dynamic power allocation at the tag reader rather than at the AP dominates the network energy efficiency while the AP operating in FD mode outperforms that in HD mode concerning energy efficienc
A Local Updating Algorithm for Personalized PageRank via Chebyshev Polynomials
Venue: arXiv cs.SI 2021
Year: 2021

The personalized PageRank algorithm is one of the most versatile tools for the analysis of networks. In spite of its ubiquity, maintaining personalized PageRank vectors when the underlying network constantly evolves is still a challenging task. To address this limitation, this work proposes a novel distributed algorithm to locally update personalized PageRank vectors when the graph topology changes. The proposed algorithm is based on the use of Chebyshev polynomials and a novel update equation that encompasses a large family of PageRank-based methods. In particular, the algorithm has the following advantages: (i) it has faster convergence speed than state-of-the-art alternatives for local PageRank updating; and (ii) it can update the solution of recent generalizations of PageRank for which no updating algorithms have been developed. Experiments in a real-world temporal network of an autonomous system validate the effectiveness of the proposed algorithm.
A Novel Approach for Attack Tree to Attack Graph Transformation: Extended Version
Venue: arXiv cs.CR 2021
Year: 2021

Attack trees and attack graphs are both common graphical threat models used by organizations to better understand possible cybersecurity threats. These models have been primarily seen as separate entities, to be used and researched in entirely different contexts, but recently there has emerged a new interest in combining the strengths of these models and in transforming models from one notation into the other. The existing works in this area focus on transforming attack graphs into attack trees. In this paper, we propose an approach to transform attack trees into attack graphs based on the fundamental understanding of how actions are represented in both structures. From this, we hope to enable more versatility in both structures.
Empowering Citizens by a Blockchain-Based Robinson List
Venue: arXiv cs.CR 2021
Year: 2021

A Robinson list protects phone subscribers against commercial spam calls. Its least basic functionality is to collect the denial of the subscribers to be contacted by market operators. Nowadays, Robinson lists run as centralised services, which implies that citizens should trust third parties for the management of their choices. In this paper, we show a design that allows us to realise a Robinson list as a decentralised service. Our work leverages the experience developed by Fondazione Ugo Bordoni as the manager of the Italian Robinson list. We present a general solution and a proof-of-concept (PoC) adopting the Algorand technology. We evaluate the performances of our PoC in terms of its scalability and of the latency perceived by the involved actors. We also discuss aspects related to identity management and privacy.
Cookie Banners, What's the Purpose? Analyzing Cookie Banner Text Through a Legal Lens
Venue: arXiv cs.HC 2021
Year: 2021

A cookie banner pops up when a user visits a website for the first time, requesting consent to the use of cookies and other trackers for a variety of purposes. Unlike prior work that has focused on evaluating the user interface (UI) design of cookie banners, this paper presents an in-depth analysis of what cookie banners say to users to get their consent. We took an interdisciplinary approach to determining what cookie banners should say. Following the legal requirements of the ePrivacy Directive (ePD) and the General Data Protection Regulation (GDPR), we manually annotated around 400 cookie banners presented on the most popular English-speaking websites visited by users residing in the EU. We focused on analyzing the purposes of cookie banners and how these purposes were expressed (e.g., any misleading or vague language, any use of jargon). We found that 89% of cookie banners violated applicable laws. In particular, 61% of banners violated the purpose specificity requirement by mentioning vague purposes, including "user experience enhancement". Further, 30% of banners used positive framing, breaching the freely given and informed consent requirements. Based on these findings, we provide recommendations that regulators can find useful. We also describe future research directions.
Inference Attacks Against Graph Neural Networks
Venue: arXiv cs.CR 2021
Year: 2021

Graph is an important data representation ubiquitously existing in the real world. However, analyzing the graph data is computationally difficult due to its non-Euclidean nature. Graph embedding is a powerful tool to solve the graph analytics problem by transforming the graph data into low-dimensional vectors. These vectors could also be shared with third parties to gain additional insights of what is behind the data. While sharing graph embedding is intriguing, the associated privacy risks are unexplored. In this paper, we systematically investigate the information leakage of the graph embedding by mounting three inference attacks. First, we can successfully infer basic graph properties, such as the number of nodes, the number of edges, and graph density, of the target graph with up to 0.89 accuracy. Second, given a subgraph of interest and the graph embedding, we can determine with high confidence that whether the subgraph is contained in the target graph. For instance, we achieve 0.98 attack AUC on the DD dataset. Third, we propose a novel graph reconstruction attack that can reconstruct a graph that has similar graph structural statistics to the target graph. We further propose an effective defense mechanism based on graph embedding perturbation to mitigate the inference attacks without noticeable performance degradation for graph classification tasks. Our code is available at https://github.com/Zhangzhk0819/GNN-Embedding-Leaks.
2nd Place Solution to Google Landmark Recognition Competition 2021
Venue: arXiv cs.CV 2021
Year: 2021

As Transformer-based architectures have recently shown encouraging progresses in computer vision. In this work, we present the solution to the Google Landmark Recognition 2021 Challenge held on Kaggle, which is an improvement on our last year's solution by changing three designs, including (1) Using Swin and CSWin as backbone for feature extraction, (2) Train on full GLDv2, and (3) Using full GLDv2 images as index image set for kNN search. With these modifications, our solution significantly improves last year solution on this year competition. Our full pipeline, after ensembling Swin, CSWin, EfficientNet B7 models, scores 0.4907 on the private leaderboard which help us to get the 2nd place in the competition.
Proactive Scheduling and Caching for Wireless VR Viewport Streaming
Venue: arXiv cs.NI 2021
Year: 2021

Virtual Reality (VR) applications require high data rate for a high-quality immersive experience, in addition to low latency to avoid dizziness and motion sickness. One of the key wireless VR challenges is providing seamless connectivity and meeting the stringent latency and bandwidth requirements. This work proposes a proactive wireless VR system that utilizes information about the user's future orientation for proactive scheduling and caching. This is achieved by leveraging deep neural networks to predict users' orientation trained on a real dataset. The 360° scene is then partitioned using an overlapping viewports scheme so that only portions of the scene covered by the users' perceptive field-of-view are streamed. Furthermore, to minimize the backhaul latency, popular viewports are cached at the edge cloud based on spatial popularity profiles. Through extensive simulations, we show that the proposed system provides significant latency and throughput performance improvement, especially in fluctuating channels and heavy load conditions. The proactive scheduling enabled by the combination of machine learning prediction and the proposed viewport scheme reduces the mean latency by more than 80% while achieving successful delivery rate close to 100%.
PlumeCityNet: Multi-Resolution Air Quality Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents an engine able to forecast jointly the concentrations of the main pollutants harming people's health: nitrogen dioxide (NO2), ozone (O3) and particulate matter (PM2.5 and PM10, which are respectively the particles whose diameters are below 2.5um and 10um respectively). The engine is fed with air quality monitoring stations' measurements, weather forecasts, physical models' outputs and traffic estimates to produce forecasts up to 24 hours. The forecasts are produced with several spatial resolutions, from a few dozens of meters to dozens of kilometers, fitting several use-cases needing air quality data. We introduce the Scale-Unit block, which enables to integrate seamlessly all available inputs at a given resolution to return forecasts at the same resolution. Then, the engine is based on a U-Net architecture built with several of those blocks, giving it the ability to process inputs and to output predictions at different resolutions. We have implemented and evaluated the engine on the largest cities in Europe and the United States, and it clearly outperforms other prediction methods. In particular, the out-of-sample accuracy remains high, meaning that the engine can be used in cities which are not included in the training dataset. A valuable advantage of the engine is that it does not need much computing power: the forecasts can be built in a few minutes on a standard CPU. Thus, they can be updated very frequently, as soon as new air quality monitoring stations' measurements are available (generally every hour), which is not the case of physical models traditionally used for air quality forecasting.
Trustworthy Artificial Intelligence and Process Mining: Challenges and Opportunities
Venue: arXiv cs.SE 2021
Year: 2021

The premise of this paper is that compliance with Trustworthy AI governance best practices and regulatory frameworks is an inherently fragmented process spanning across diverse organizational units, external stakeholders, and systems of record, resulting in process uncertainties and in compliance gaps that may expose organizations to reputational and regulatory risks. Moreover, there are complexities associated with meeting the specific dimensions of Trustworthy AI best practices such as data governance, conformance testing, quality assurance of AI model behaviors, transparency, accountability, and confidentiality requirements. These processes involve multiple steps, hand-offs, re-works, and human-in-the-loop oversight. In this paper, we demonstrate that process mining can provide a useful framework for gaining fact-based visibility to AI compliance process execution, surfacing compliance bottlenecks, and providing for an automated approach to analyze, remediate and monitor uncertainty in AI regulatory compliance processes.
Graphon based Clustering and Testing of Networks: Algorithms and Theory
Venue: arXiv cs.LG 2021
Year: 2021

Network-valued data are encountered in a wide range of applications and pose challenges in learning due to their complex structure and absence of vertex correspondence. Typical examples of such problems include classification or grouping of protein structures and social networks. Various methods, ranging from graph kernels to graph neural networks, have been proposed that achieve some success in graph classification problems. However, most methods have limited theoretical justification, and their applicability beyond classification remains unexplored. In this work, we propose methods for clustering multiple graphs, without vertex correspondence, that are inspired by the recent literature on estimating graphons -- symmetric functions corresponding to infinite vertex limit of graphs. We propose a novel graph distance based on sorting-and-smoothing graphon estimators. Using the proposed graph distance, we present two clustering algorithms and show that they achieve state-of-the-art results. We prove the statistical consistency of both algorithms under Lipschitz assumptions on the graph degrees. We further study the applicability of the proposed distance for graph two-sample testing problems.
Spell my name: keyword boosted speech recognition
Venue: arXiv cs.SD 2021
Year: 2021

Recognition of uncommon words such as names and technical terminology is important to understanding conversations in context. However, the ability to recognise such words remains a challenge in modern automatic speech recognition (ASR) systems. In this paper, we propose a simple but powerful ASR decoding method that can better recognise these uncommon keywords, which in turn enables better readability of the results. The method boosts the probabilities of given keywords in a beam search based on acoustic model predictions. The method does not require any training in advance. We demonstrate the effectiveness of our method on the LibriSpeeech test sets and also internal data of real-world conversations. Our method significantly boosts keyword accuracy on the test sets, while maintaining the accuracy of the other words, and as well as providing significant qualitative improvements. This method is applicable to other tasks such as machine translation, or wherever unseen and difficult keywords need to be recognised in beam search.
Accelerated First Order Methods for Variational Imaging
Venue: arXiv cs.CV 2021
Year: 2021

In this thesis, we offer a thorough investigation of different regularisation terms used in variational imaging problems, together with detailed optimisation processes of these problems. We begin by studying smooth problems and partially non-smooth problems in the form of Tikhonov denoising and Total Variation (TV) denoising, respectively. For Tikhonov denoising, we study an accelerated gradient method with adaptive restart, which shows a very rapid convergence rate. However, it is not straightforward to apply this fast algorithm to TV denoising, due to the non-smoothness of its built-in regularisation. To tackle this issue, we propose to utilise duality to convert such a non-smooth problem into a smooth one so that the accelerated gradient method with restart applies naturally. However, we notice that both Tikhonov and TV regularisations have drawbacks, in the form of blurred image edges and staircase artefacts, respectively. To overcome these drawbacks, we propose a novel adaption to Total Generalised Variation (TGV) regularisation called Total Smooth Variation (TSV), which retains edges and meanwhile does not produce results which contain staircase artefacts. To optimise TSV effectively, we then propose the Accelerated Proximal Gradient Algorithm (APGA) which also utilises adaptive restart techniques. Compared to existing state-of-the-art regularisations (e.g. TV), TSV is shown to obtain more effective results on denoising problems as well as advanced imaging applications such as magnetic resonance imaging (MRI) reconstruction and optical flow. TSV removes the staircase artefacts observed when using TV regularisation, but has the added advantage over TGV that it can be efficiently optimised using gradient based methods with Nesterov acceleration and adaptive restart. Code is available at https://github.com/Jbartlett6/Accelerated-First-Order-Method-for-Variational-Imaging.
Parallel Composition of Weighted Finite-State Transducers
Venue: arXiv cs.CL 2021
Year: 2021

Finite-state transducers (FSTs) are frequently used in speech recognition. Transducer composition is an essential operation for combining different sources of information at different granularities. However, composition is also one of the more computationally expensive operations. Due to the heterogeneous structure of FSTs, parallel algorithms for composition are suboptimal in efficiency, generality, or both. We propose an algorithm for parallel composition and implement it on graphics processing units. We benchmark our parallel algorithm on the composition of random graphs and the composition of graphs commonly used in speech recognition. The parallel composition scales better with the size of the input graphs and for large graphs can be as much as 10 to 30 times faster than a sequential CPU algorithm.