OpenRaiser/Intern-Atlas huggingface dataset:
On Differentially Private Federated Linear Contextual Bandits
Venue: arXiv cs.LG 2023
Year: 2023

We consider cross-silo federated linear contextual bandit (LCB) problem under differential privacy, where multiple silos (agents) interact with the local users and communicate via a central server to realize collaboration while without sacrificing each user's privacy. We identify three issues in the state-of-the-art: (i) failure of claimed privacy protection and (ii) incorrect regret bound due to noise miscalculation and (iii) ungrounded communication cost. To resolve these issues, we take a two-step principled approach. First, we design an algorithmic framework consisting of a generic federated LCB algorithm and flexible privacy protocols. Then, leveraging the proposed framework, we study federated LCBs under two different privacy constraints. We first establish privacy and regret guarantees under silo-level local differential privacy, which fix the issues present in state-of-the-art algorithm. To further improve the regret performance, we next consider shuffle model of differential privacy, under which we show that our algorithm can achieve nearly ``optimal'' regret without a trusted server. We accomplish this via two different schemes -- one relies on a new result on privacy amplification via shuffling for DP mechanisms and another one leverages the integration of a shuffle protocol for vector sum into the tree-based mechanism, both of which might be of independent interest. Finally, we support our theoretical results with numerical evaluations over contextual bandit instances generated from both synthetic and real-life data.
Priorities for more effective tech regulation
Venue: arXiv cs.CY 2023
Year: 2023

Ample research has demonstrated that compliance with data protection principles remains limited on the web and mobile. For example, almost none of the apps on the Google Play Store fulfil the minimum requirements regarding consent under EU and UK law, while most of them share tracking data with companies like Google/Alphabet and Facebook/Meta and would likely need to seek consent from their users. Indeed, recent privacy efforts and enforcement by Apple have had - in some regards - a more pronounced effect on apps' data practices than the EU's ambitious General Data Protection Regulation (GDPR). Given the current mismatch between the law on the books and data practices in reality, iterative changes to current legal practice will not be enough to meaningfully tame egregious data practices. Hence, this technical report proposes a range of priorities for academia, regulators and the interested public in order to move beyond the status quo.
Bridging the Bubbles: Connecting Academia and Industry in Cybersecurity Research
Venue: arXiv cs.CR 2023
Year: 2023

There is a perceived disconnect between how ad hoc industry solutions and academic research solutions in cyber security are developed and applied. Is there a difference in philosophy in how solutions to cyber security problems are developed by industry and by academia. What could academia and industry do to bridge this gap and speed up the development and use of effective cybersecurity solutions? This paper provides an overview of the most critical gaps and solutions identified by an interdisciplinary expert exchange on the topic. The discussion was held in the form of the webinar "Bridging the Bubbles: Connecting Academia and Industry in Cybersecurity Research" in November 2022 as part of the Rogers Cybersecure Catalyst webinar series. Panelists included researchers from academia and industry as well as experts from industry and business development. The key findings and recommendations of this exchange are supported by the relevant scientific literature on the topic within this paper. Different approaches and time frames in development and lifecycle management, challenges in knowledge transfer and communication as well as heterogeneous metrics for success in projects are examples of the evaluated subject areas.
Quantifying Valence and Arousal in Text with Multilingual Pre-trained Transformers
Venue: arXiv cs.CL 2023
Year: 2023

The analysis of emotions expressed in text has numerous applications. In contrast to categorical analysis, focused on classifying emotions according to a pre-defined set of common classes, dimensional approaches can offer a more nuanced way to distinguish between different emotions. Still, dimensional methods have been less studied in the literature. Considering a valence-arousal dimensional space, this work assesses the use of pre-trained Transformers to predict these two dimensions on a continuous scale, with input texts from multiple languages and domains. We specifically combined multiple annotated datasets from previous studies, corresponding to either emotional lexica or short text documents, and evaluated models of multiple sizes and trained under different settings. Our results show that model size can have a significant impact on the quality of predictions, and that by fine-tuning a large model we can confidently predict valence and arousal in multiple languages. We make available the code, models, and supporting data.
Online Sparse Streaming Feature Selection Using Adapted Classification
Venue: arXiv cs.LG 2023
Year: 2023

Traditional feature selections need to know the feature space before learning, and online streaming feature selection (OSFS) is proposed to process streaming features on the fly. Existing methods divide features into relevance or irrelevance without missing data, and deleting irrelevant features may lead to in-formation loss. Motivated by this, we focus on completing the streaming feature matrix and division of feature correlation and propose online sparse streaming feature selection based on adapted classification (OS2FS-AC). This study uses Latent Factor Analysis (LFA) to pre-estimate missed data. Besides, we use the adaptive method to obtain the threshold, divide the features into strongly relevant, weakly relevant, and irrelevant features, and then divide weak relevance with more information. Experimental results on ten real-world data sets demonstrate that OS2FS-AC performs better than state-of-the-art algo-rithms.
Analyzing Populations of Neural Networks via Dynamical Model Embedding
Venue: arXiv cs.LG 2023
Year: 2023

A core challenge in the interpretation of deep neural networks is identifying commonalities between the underlying algorithms implemented by distinct networks trained for the same task. Motivated by this problem, we introduce DYNAMO, an algorithm that constructs low-dimensional manifolds where each point corresponds to a neural network model, and two points are nearby if the corresponding neural networks enact similar high-level computational processes. DYNAMO takes as input a collection of pre-trained neural networks and outputs a meta-model that emulates the dynamics of the hidden states as well as the outputs of any model in the collection. The specific model to be emulated is determined by a model embedding vector that the meta-model takes as input; these model embedding vectors constitute a manifold corresponding to the given population of models. We apply DYNAMO to both RNNs and CNNs, and find that the resulting model embedding spaces enable novel applications: clustering of neural networks on the basis of their high-level computational processes in a manner that is less sensitive to reparameterization; model averaging of several neural networks trained on the same task to arrive at a new, operable neural network with similar task performance; and semi-supervised learning via optimization on the model embedding space. Using a fixed-point analysis of meta-models trained on populations of RNNs, we gain new insights into how similarities of the topology of RNN dynamics correspond to similarities of their high-level computational processes.
An Optimal Joint Antenna and User Selection Algorithm for QoS-based CR-NOMA
Venue: arXiv cs.IT 2023
Year: 2023

Both non-orthogonal multiple access (NOMA), which can serve multiple users simultaneously and on the same frequency, and cognitive radio (CR) can contribute to eliminating the spectrum scarcity problem. In this work, an uplink CR-based NOMA (CR-NOMA) system, which is equipped with multiple users and a base station with a multi-antenna, is proposed to improve spectral efficiency. By considering the users' quality of service (QoS), the system performance of successive interference cancellation (SIC) is investigated in this system. Two different antenna and secondary user selection algorithms are proposed to improve the outage performance and retard the effect of the error floor. The multi-antenna CR-NOMA with QoS-based SIC system outperforms conventional channel state information (CSI)-based SIC. In addition, it is shown that the outage performance of this system on both proposed algorithms is better than in the case of not using the algorithm. The closed-form outage probability expression of this system for the suboptimal joint antenna and user selection algorithm is derived. Furthermore, when the proposed algorithms are not used, the closed-form expression of the outage probability for this system with a single-antenna base station is derived. Extensive simulation results verify the accuracy of theoretical analyses.
Scalable End-to-End ML Platforms: from AutoML to Self-serve
Venue: arXiv cs.LG 2023
Year: 2023

ML platforms help enable intelligent data-driven applications and maintain them with limited engineering effort. Upon sufficiently broad adoption, such platforms reach economies of scale that bring greater component reuse while improving efficiency of system development and maintenance. For an end-to-end ML platform with broad adoption, scaling relies on pervasive ML automation and system integration to reach the quality we term self-serve that we define with ten requirements and six optional capabilities. With this in mind, we identify long-term goals for platform development, discuss related tradeoffs and future work. Our reasoning is illustrated on two commercially-deployed end-to-end ML platforms that host hundreds of real-time use cases -- one general-purpose and one specialized.
TabGenie: A Toolkit for Table-to-Text Generation
Venue: arXiv cs.CL 2023
Year: 2023

Heterogenity of data-to-text generation datasets limits the research on data-to-text generation systems. We present TabGenie - a toolkit which enables researchers to explore, preprocess, and analyze a variety of data-to-text generation datasets through the unified framework of table-to-text generation. In TabGenie, all the inputs are represented as tables with associated metadata. The tables can be explored through the web interface, which also provides an interactive mode for debugging table-to-text generation, facilitates side-by-side comparison of generated system outputs, and allows easy exports for manual analysis. Furthermore, TabGenie is equipped with command line processing tools and Python bindings for unified dataset loading and processing. We release TabGenie as a PyPI package and provide its open-source code and a live demo at https://github.com/kasnerz/tabgenie.
Nautilus: A Framework for Cross-Layer Cartography of Submarine Cables and IP Links
Venue: arXiv cs.NI 2023
Year: 2023

Submarine cables constitute the backbone of the Internet. However, these critical infrastructure components are vulnerable to several natural and man-made threats, and during failures, are difficult to repair in their remote oceanic environments. In spite of their crucial role, we have a limited understanding of the impact of submarine cable failures on global connectivity, particularly on the higher layers of the Internet. In this paper, we present Nautilus, a framework for cross-layer cartography of submarine cables and IP links. Using a corpus of public datasets and Internet cartographic techniques, Nautilus identifies IP links that are likely traversing submarine cables and maps them to one or more potential cables. Nautilus also gives each IP to cable assignment a prediction score that reflects the confidence in the mapping. Nautilus generates a mapping for 3.05 million and 1.43 million IPv4 and IPv6 links respectively, covering 91% of all active cables. In the absence of ground truth data, we validate Nautilus mapping using three techniques: analyzing past cable failures, using targeted traceroute measurements, and comparing with public network maps of two operators.
Near-ideal predictors and causal filters for discrete time signals
Venue: arXiv cs.IT 2023
Year: 2023

The paper presents linear predictors and causal filters for discrete time signals featuring some different kinds of spectrum degeneracy. These predictors and filters are based on approximation of ideal non-causal transfer functions by causal transfer functions represented by polynomials of Z-transform of the unit step signal.
WISK: A Workload-aware Learned Index for Spatial Keyword Queries
Venue: arXiv cs.DB 2023
Year: 2023

Spatial objects often come with textual information, such as Points of Interest (POIs) with their descriptions, which are referred to as geo-textual data. To retrieve such data, spatial keyword queries that take into account both spatial proximity and textual relevance have been extensively studied. Existing indexes designed for spatial keyword queries are mostly built based on the geo-textual data without considering the distribution of queries already received. However, previous studies have shown that utilizing the known query distribution can improve the index structure for future query processing. In this paper, we propose WISK, a learned index for spatial keyword queries, which self-adapts for optimizing querying costs given a query workload. One key challenge is how to utilize both structured spatial attributes and unstructured textual information during learning the index. We first divide the data objects into partitions, aiming to minimize the processing costs of the given query workload. We prove the NP-hardness of the partitioning problem and propose a machine learning model to find the optimal partitions. Then, to achieve more pruning power, we build a hierarchical structure based on the generated partitions in a bottom-up manner with a reinforcement learning-based approach. We conduct extensive experiments on real-world datasets and query workloads with various distributions, and the results show that WISK outperforms all competitors, achieving up to 8x speedup in querying time with comparable storage overhead.
A Comprehensive Study on Robustness of Image Classification Models: Benchmarking and Rethinking
Venue: arXiv cs.CV 2023
Year: 2023

The robustness of deep neural networks is usually lacking under adversarial examples, common corruptions, and distribution shifts, which becomes an important research problem in the development of deep learning. Although new deep learning methods and robustness improvement techniques have been constantly proposed, the robustness evaluations of existing methods are often inadequate due to their rapid development, diverse noise patterns, and simple evaluation metrics. Without thorough robustness evaluations, it is hard to understand the advances in the field and identify the effective methods. In this paper, we establish a comprehensive robustness benchmark called \textbf{ARES-Bench} on the image classification task. In our benchmark, we evaluate the robustness of 55 typical deep learning models on ImageNet with diverse architectures (e.g., CNNs, Transformers) and learning algorithms (e.g., normal supervised training, pre-training, adversarial training) under numerous adversarial attacks and out-of-distribution (OOD) datasets. Using robustness curves as the major evaluation criteria, we conduct large-scale experiments and draw several important findings, including: 1) there is an inherent trade-off between adversarial and natural robustness for the same model architecture; 2) adversarial training effectively improves adversarial robustness, especially when performed on Transformer architectures; 3) pre-training significantly improves natural robustness based on more training data or self-supervised learning. Based on ARES-Bench, we further analyze the training tricks in large-scale adversarial training on ImageNet. By designing the training settings accordingly, we achieve the new state-of-the-art adversarial robustness. We have made the benchmarking results and code platform publicly available.
A Token-Wise Beam Search Algorithm for RNN-T
Venue: arXiv cs.LG 2023
Year: 2023

Standard Recurrent Neural Network Transducers (RNN-T) decoding algorithms for speech recognition are iterating over the time axis, such that one time step is decoded before moving on to the next time step. Those algorithms result in a large number of calls to the joint network, which were shown in previous work to be an important factor that reduces decoding speed. We present a decoding beam search algorithm that batches the joint network calls across a segment of time steps, which results in 20%-96% decoding speedups consistently across all models and settings experimented with. In addition, aggregating emission probabilities over a segment may be seen as a better approximation to finding the most likely model output, causing our algorithm to improve oracle word error rate by up to 11% relative as the segment size increases, and to slightly improve general word error rate.
A Study of Comfortability between Interactive AI and Human
Venue: arXiv cs.HC 2023
Year: 2023

As the use of interactive AI systems becomes increasingly prevalent in our daily lives, it is crucial to understand how individuals feel when interacting with such systems. In this work, we investigate the comfort level of individuals when interacting with intent-predicting AI systems and identify the factors of influence. We introduce a study protocol to analyze human comfortability when interacting with intent-predicting AI systems and execute the study with over a dozen participants. The study findings suggest that users are comfortable with AI systems if they have control and their privacy is not affected. Additionally, the study found that users could differentiate between AI and human responses, but this did not significantly affect their comfort levels. This research paper's significance lies in its contribution to the growing body of literature on interactive AI systems, and it emphasizes the need to consider user perceptions in the development and deployment.
RemoteTouch: Enhancing Immersive 3D Video Communication with Hand Touch
Venue: arXiv cs.CV 2023
Year: 2023

Recent research advance has significantly improved the visual realism of immersive 3D video communication. In this work we present a method to further enhance this immersive experience by adding the hand touch capability ("remote hand clapping"). In our system, each meeting participant sits in front of a large screen with haptic feedback. The local participant can reach his hand out to the screen and perform hand clapping with the remote participant as if the two participants were only separated by a virtual glass. A key challenge in emulating the remote hand touch is the realistic rendering of the participant's hand and arm as the hand touches the screen. When the hand is very close to the screen, the RGBD data required for realistic rendering is no longer available. To tackle this challenge, we present a dual representation of the user's hand. Our dual representation not only preserves the high-quality rendering usually found in recent image-based rendering systems but also allows the hand to reach the screen. This is possible because the dual representation includes both an image-based model and a 3D geometry-based model, with the latter driven by a hand skeleton tracked by a side view camera. In addition, the dual representation provides a distance-based fusion of the image-based and 3D geometry-based models as the hand moves closer to the screen. The result is that the image-based and 3D geometry-based models mutually enhance each other, leading to realistic and seamless rendering. Our experiments demonstrate that our method provides consistent hand contact experience between remote users and improves the immersive experience of 3D video communication.
Practical Algorithms for Orientations of Partially Directed Graphical Models
Venue: arXiv cs.AI 2023
Year: 2023

In observational studies, the true causal model is typically unknown and needs to be estimated from available observational and limited experimental data. In such cases, the learned causal model is commonly represented as a partially directed acyclic graph (PDAG), which contains both directed and undirected edges indicating uncertainty of causal relations between random variables. The main focus of this paper is on the maximal orientation task, which, for a given PDAG, aims to orient the undirected edges maximally such that the resulting graph represents the same Markov equivalent DAGs as the input PDAG. This task is a subroutine used frequently in causal discovery, e. g., as the final step of the celebrated PC algorithm. Utilizing connections to the problem of finding a consistent DAG extension of a PDAG, we derive faster algorithms for computing the maximal orientation by proposing two novel approaches for extending PDAGs, both constructed with an emphasis on simplicity and practical effectiveness.
Item Cold Start Recommendation via Adversarial Variational Auto-encoder Warm-up
Venue: arXiv cs.IR 2023
Year: 2023

The gap between the randomly initialized item ID embedding and the well-trained warm item ID embedding makes the cold items hard to suit the recommendation system, which is trained on the data of historical warm items. To alleviate the performance decline of new items recommendation, the distribution of the new item ID embedding should be close to that of the historical warm items. To achieve this goal, we propose an Adversarial Variational Auto-encoder Warm-up model (AVAEW) to generate warm-up item ID embedding for cold items. Specifically, we develop a conditional variational auto-encoder model to leverage the side information of items for generating the warm-up item ID embedding. Particularly, we introduce an adversarial module to enforce the alignment between warm-up item ID embedding distribution and historical item ID embedding distribution. We demonstrate the effectiveness and compatibility of the proposed method by extensive offline experiments on public datasets and online A/B tests on a real-world large-scale news recommendation platform.
The Ethics of Social Media Analytics in Migration Studies
Venue: arXiv cs.CY 2023
Year: 2023

The prevalence of social media platforms and their use across the globe makes them attractive options for studying large groups of people, particularly when some of these platforms provide access to large amounts of structured data. However, with the collection, storage, and use of this data comes ethical and legal responsibilities, which are particularly important when looking at social groups such as migrants, who are often stigmatised and criminalised. Various guidelines, frameworks and laws have been developed to ensure social media data is used in the most ethical way. However, they have quickly evolved within the past few years and are scattered across various fields and domains. To help researchers navigate these issues, this chapter provides an overview of the ethical considerations of studying migration via social media platforms. Building on relevant academic literature, as well as national and supranational frameworks and legislations, we review how the main ethical issues related to social media research have been discussed in the past twenty years and outline good practice examples to mitigate them. This overview is designed to provide researchers with theoretical and practical tools to consider and mitigate the ethical challenges related to social media research in migration-related contexts.