OpenRaiser/Intern-Atlas huggingface dataset:
ADePT: Auto-encoder based Differentially Private Text Transformation
Venue: arXiv cs.CR 2021
Year: 2021

Privacy is an important concern when building statistical models on data containing personal information. Differential privacy offers a strong definition of privacy and can be used to solve several privacy concerns (Dwork et al., 2014). Multiple solutions have been proposed for the differentially-private transformation of datasets containing sensitive information. However, such transformation algorithms offer poor utility in Natural Language Processing (NLP) tasks due to noise added in the process. In this paper, we address this issue by providing a utility-preserving differentially private text transformation algorithm using auto-encoders. Our algorithm transforms text to offer robustness against attacks and produces transformations with high semantic quality that perform well on downstream NLP tasks. We prove the theoretical privacy guarantee of our algorithm and assess its privacy leakage under Membership Inference Attacks(MIA) (Shokri et al., 2017) on models trained with transformed data. Our results show that the proposed model performs better against MIA attacks while offering lower to no degradation in the utility of the underlying transformation process compared to existing baselines.
Guidance on the Assurance of Machine Learning in Autonomous Systems (AMLAS)
Venue: arXiv cs.LG 2021
Year: 2021

Machine Learning (ML) is now used in a range of systems with results that are reported to exceed, under certain conditions, human performance. Many of these systems, in domains such as healthcare , automotive and manufacturing, exhibit high degrees of autonomy and are safety critical. Establishing justified confidence in ML forms a core part of the safety case for these systems. In this document we introduce a methodology for the Assurance of Machine Learning for use in Autonomous Systems (AMLAS). AMLAS comprises a set of safety case patterns and a process for (1) systematically integrating safety assurance into the development of ML components and (2) for generating the evidence base for explicitly justifying the acceptable safety of these components when integrated into autonomous system applications.
The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication
Venue: arXiv cs.LG 2021
Year: 2021

We resolve the min-max complexity of distributed stochastic convex optimization (up to a log factor) in the intermittent communication setting, where $M$ machines work in parallel over the course of $R$ rounds of communication to optimize the objective, and during each round of communication, each machine may sequentially compute $K$ stochastic gradient estimates. We present a novel lower bound with a matching upper bound that establishes an optimal algorithm.
Reed-Muller Subcodes: Machine Learning-Aided Design of Efficient Soft Recursive Decoding
Venue: arXiv cs.IT 2021
Year: 2021

Reed-Muller (RM) codes are conjectured to achieve the capacity of any binary-input memoryless symmetric (BMS) channel, and are observed to have a comparable performance to that of random codes in terms of scaling laws. On the negative side, RM codes lack efficient decoders with performance close to that of a maximum likelihood decoder for general parameters. Also, they only admit certain discrete sets of rates. In this paper, we focus on subcodes of RM codes with flexible rates that can take any code dimension from 1 to n, where n is the blocklength. We first extend the recursive projection-aggregation (RPA) algorithm proposed recently by Ye and Abbe for decoding RM codes. To lower the complexity of our decoding algorithm, referred to as subRPA in this paper, we investigate different ways for pruning the projections. We then derive the soft-decision based version of our algorithm, called soft-subRPA, that is shown to improve upon the performance of subRPA. Furthermore, it enables training a machine learning (ML) model to search for \textit{good} sets of projections in the sense of minimizing the decoding error rate. Training our ML model enables achieving very close to the performance of full-projection decoding with a significantly reduced number of projections. For instance, our simulation results on a (64,14) RM subcode show almost identical performance for full-projection decoding and pruned-projection decoding with 15 projections picked via training our ML model. This is equivalent to lowering the complexity by a factor of more than 4 without sacrificing the decoding performance.
A Historical and Statistical Studyof the Software Vulnerability Landscape
Venue: arXiv cs.CR 2021
Year: 2021

Understanding the landscape of software vulnerabilities is key for developing effective security solutions. Fortunately, the evaluation of vulnerability databases that use a framework for communicating vulnerability attributes and their severity scores, such as the Common Vulnerability Scoring System (CVSS), can help shed light on the nature of publicly published vulnerabilities. In this paper, we characterize the software vulnerability landscape by performing a historical and statistical analysis of CVSS vulnerability metrics over the period of 2005 to 2019 through using data from the National Vulnerability Database. We conduct three studies analyzing the following: the distribution of CVSS scores (both empirical and theoretical), the distribution of CVSS metric values and how vulnerability characteristics change over time, and the relative rankings of the most frequent metric value over time. Our resulting analysis shows that the vulnerability threat landscape has been dominated by only a few vulnerability types and has changed little during the time period of the study. The overwhelming majority of vulnerabilities are exploitable over the network. The complexity to successfully exploit these vulnerabilities is dominantly low; very little authentication to the target victim is necessary for a successful attack. And most of the flaws require very limited interaction with users. However on the positive side, the damage of these vulnerabilities is mostly confined within the security scope of the impacted components. A discussion of lessons that could be learned from this analysis is presented.
On Greedy Approaches to Hierarchical Aggregation
Venue: arXiv cs.DS 2021
Year: 2021

We analyze greedy algorithms for the Hierarchical Aggregation (HAG) problem, a strategy introduced in [Jia et al., KDD 2020] for speeding up learning on Graph Neural Networks (GNNs). The idea of HAG is to identify and remove redundancies in computations performed when training GNNs. The associated optimization problem is to identify and remove the most redundancies. Previous work introduced a greedy approach for the HAG problem and claimed a 1-1/e approximation factor. We show by example that this is not correct, and one cannot hope for better than a 1/2 approximation factor. We prove that this greedy algorithm does satisfy some (weaker) approximation guarantee, by showing a new connection between the HAG problem and maximum matching problems in hypergraphs. We also introduce a second greedy algorithm which can out-perform the first one, and we show how to implement it efficiently in some parameter regimes. Finally, we introduce some greedy heuristics that are much faster than the above greedy algorithms, and we demonstrate that they perform well on real-world graphs.
Leave No User Behind: Towards Improving the Utility of Recommender Systems for Non-mainstream Users
Venue: arXiv cs.IR 2021
Year: 2021

In a collaborative-filtering recommendation scenario, biases in the data will likely propagate in the learned recommendations. In this paper we focus on the so-called mainstream bias: the tendency of a recommender system to provide better recommendations to users who have a mainstream taste, as opposed to non-mainstream users. We propose NAECF, a conceptually simple but effective idea to address this bias. The idea consists of adding an autoencoder (AE) layer when learning user and item representations with text-based Convolutional Neural Networks. The AEs, one for the users and one for the items, serve as adversaries to the process of minimizing the rating prediction error when learning how to recommend. They enforce that the specific unique properties of all users and items are sufficiently well incorporated and preserved in the learned representations. These representations, extracted as the bottlenecks of the corresponding AEs, are expected to be less biased towards mainstream users, and to provide more balanced recommendation utility across all users. Our experimental results confirm these expectations, significantly improving the recommendations for non-mainstream users while maintaining the recommendation quality for mainstream users. Our results emphasize the importance of deploying extensive content-based features, such as online reviews, in order to better represent users and items to maximize the de-biasing effect.
Distributed Conditional Generative Adversarial Networks (GANs) for Data-Driven Millimeter Wave Communications in UAV Networks
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, a novel framework is proposed to perform data-driven air-to-ground (A2G) channel estimation for millimeter wave (mmWave) communications in an unmanned aerial vehicle (UAV) wireless network. First, an effective channel estimation approach is developed to collect mmWave channel information, allowing each UAV to train a stand-alone channel model via a conditional generative adversarial network (CGAN) along each beamforming direction. Next, in order to expand the application scenarios of the trained channel model into a broader spatial-temporal domain, a cooperative framework, based on a distributed CGAN architecture, is developed, allowing each UAV to collaboratively learn the mmWave channel distribution in a fully-distributed manner. To guarantee an efficient learning process, necessary and sufficient conditions for the optimal UAV network topology that maximizes the learning rate for cooperative channel modeling are derived, and the optimal CGAN learning solution per UAV is subsequently characterized, based on the distributed network structure. Simulation results show that the proposed distributed CGAN approach is robust to the local training error at each UAV. Meanwhile, a larger airborne network size requires more communication resources per UAV to guarantee an efficient learning rate. The results also show that, compared with a stand-alone CGAN without information sharing and two other distributed schemes, namely: A multi-discriminator CGAN and a federated CGAN method, the proposed distributed CGAN approach yields a higher modeling accuracy while learning the environment, and it achieves a larger average data rate in the online performance of UAV downlink mmWave communications.
MANA: Microarchitecting an Instruction Prefetcher
Venue: arXiv cs.AR 2021
Year: 2021

L1 instruction (L1-I) cache misses are a source of performance bottleneck. Sequential prefetchers are simple solutions to mitigate this problem; however, prior work has shown that these prefetchers leave considerable potentials uncovered. This observation has motivated many researchers to come up with more advanced instruction prefetchers. In 2011, Proactive Instruction Fetch (PIF) showed that a hardware prefetcher could effectively eliminate all of the instruction-cache misses. However, its enormous storage cost makes it an impractical solution. Consequently, reducing the storage cost was the main research focus in the instruction prefetching in the past decade. Several instruction prefetchers, including RDIP and Shotgun, were proposed to offer PIF-level performance with significantly lower storage overhead. However, our findings show that there is a considerable performance gap between these proposals and PIF. While these proposals use different mechanisms for instruction prefetching, the performance gap is largely not because of the mechanism, and instead, is due to not having sufficient storage. Prior proposals suffer from one or both of the following shortcomings: (1) a large number of metadata records to cover the potential, and (2) a high storage cost of each record. The first problem causes metadata miss, and the second problem prohibits the prefetcher from storing enough records within reasonably-sized storage.
A privacy-preserving approach to streaming eye-tracking data
Venue: arXiv cs.HC 2021
Year: 2021

Eye-tracking technology is being increasingly integrated into mixed reality devices. Although critical applications are being enabled, there are significant possibilities for violating user privacy expectations. We show that there is an appreciable risk of unique user identification even under natural viewing conditions in virtual reality. This identification would allow an app to connect a user's personal ID with their work ID without needing their consent, for example. To mitigate such risks we propose a framework that incorporates gatekeeping via the design of the application programming interface and via software-implemented privacy mechanisms. Our results indicate that these mechanisms can reduce the rate of identification from as much as 85% to as low as 30%. The impact of introducing these mechanisms is less than 1.5$^\circ$ error in gaze position for gaze prediction. Gaze data streams can thus be made private while still allowing for gaze prediction, for example, during foveated rendering. Our approach is the first to support privacy-by-design in the flow of eye-tracking data within mixed reality use cases.
Analyzing the Impact of Molecular Re-Radiation on the MIMO Capacity in High-Frequency Bands
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we show how the absorption and re-radiation energy from molecules in the air can influence the Multiple Input Multiple Output (MIMO) performance in high-frequency bands, e.g., millimeter wave (mmWave) and terahertz. In more detail, some common atmosphere molecules, such as oxygen and water, can absorb and re-radiate energy in their natural resonance frequencies, such as 60 GHz, 180 GHz and 320 GHz. Hence, when hit by electromagnetic waves, molecules will get excited and absorb energy, which leads to an extra path loss and is known as molecular attenuation. Meanwhile, the absorbed energy will be re-radiated towards a random direction with a random phase. These re-radiated waves also interfere with the signal transmission. Although, the molecular re-radiation was mostly considered as noise in literature, recent works show that it is correlated to the main signal and can be viewed as a composition of multiple delayed or scattered signals. Such a phenomenon can provide non-line-of-sight (NLoS) paths in an environment that lacks scatterers, which increases spatial multiplexing and thus greatly enhances the performance of MIMO systems. Therefore in this paper, we explore the scattering model and noise models of molecular re-radiation to characterize the channel transfer function of the NLoS channels created by atmosphere molecules. Our simulation results show that the re-radiation can increase MIMO capacity up to 3 folds in mmWave and 6 folds in terahertz for a set of realistic transmit power, distance, and antenna numbers. We also show that in the high SNR, the re-radiation makes the open-loop precoding viable, which is an alternative to beamforming to avoid beam alignment sensitivity in high mobility applications.
The Ethical Implications of Shared Medical Decision Making without Providing Adequate Computational Support to the Care Provider and to the Patient
Venue: arXiv cs.AI 2021
Year: 2021

There is a clear need to involve patients in medical decisions. However, cognitive psychological research has highlighted the cognitive limitations of humans with respect to 1. Probabilistic assessment of the patient state and of potential outcomes of various decisions, 2. Elicitation of the patient utility function, and 3. Integration of the probabilistic knowledge and of patient preferences to determine the optimal strategy. Therefore, without adequate computational support, current shared decision models have severe ethical deficiencies. An informed consent model unfairly transfers the responsibility to a patient who does not have the necessary knowledge, nor the integration capability. A paternalistic model endows with exaggerated power a physician who might not be aware of the patient preferences, is prone to multiple cognitive biases, and whose computational integration capability is bounded. Recent progress in Artificial Intelligence suggests adding a third agent: a computer, in all deliberative medical decisions: Non emergency medical decisions in which more than one alternative exists, the patient preferences can be elicited, the therapeutic alternatives might be influenced by these preferences, medical knowledge exists regarding the likelihood of the decision outcomes, and there is sufficient decision time. Ethical physicians should exploit computational decision support technologies, neither making the decisions solely on their own, nor shirking their duty and shifting the responsibility to patients in the name of informed consent. The resulting three way (patient, care provider, computer) human machine model that we suggest emphasizes the patient preferences, the physician knowledge, and the computational integration of both aspects, does not diminish the physician role, but rather brings out the best in human and machine.
Task Planning on Stochastic Aisle Graphs for Precision Agriculture
Venue: arXiv cs.RO 2021
Year: 2021

This work addresses task planning under uncertainty for precision agriculture applications whereby task costs are uncertain and the gain of completing a task is proportional to resource consumption (such as water consumption in precision irrigation). The goal is to complete all tasks while prioritizing those that are more urgent, and subject to diverse budget thresholds and stochastic costs for tasks. To describe agriculture-related environments that incorporate stochastic costs to complete tasks, a new Stochastic-Vertex-Cost Aisle Graph (SAG) is introduced. Then, a task allocation algorithm, termed Next-Best-Action Planning (NBA-P), is proposed. NBA-P utilizes the underlying structure enabled by SAG, and tackles the task planning problem by simultaneously determining the optimal tasks to perform and an optimal time to exit (i.e. return to a base station), at run-time. The proposed approach is tested with both simulated data and real-world experimental datasets collected in a commercial vineyard, in both single- and multi-robot scenarios. In all cases, NBA-P outperforms other evaluated methods in terms of return per visited vertex, wasted resources resulting from aborted tasks (i.e. when a budget threshold is exceeded), and total visited vertices.
A Computational Framework for Slang Generation
Venue: arXiv cs.CL 2021
Year: 2021

Slang is a common type of informal language, but its flexible nature and paucity of data resources present challenges for existing natural language systems. We take an initial step toward machine generation of slang by developing a framework that models the speaker's word choice in slang context. Our framework encodes novel slang meaning by relating the conventional and slang senses of a word while incorporating syntactic and contextual knowledge in slang usage. We construct the framework using a combination of probabilistic inference and neural contrastive learning. We perform rigorous evaluations on three slang dictionaries and show that our approach not only outperforms state-of-the-art language models, but also better predicts the historical emergence of slang word usages from 1960s to 2000s. We interpret the proposed models and find that the contrastively learned semantic space is sensitive to the similarities between slang and conventional senses of words. Our work creates opportunities for the automated generation and interpretation of informal language.
QuizCram: A Quiz-Driven Lecture Viewing Interface
Venue: arXiv cs.HC 2021
Year: 2021

QuizCram is an interface for navigating lecture videos that uses quizzes to help users determine what they should view. We developed it in response to observing peaks in video seeking behaviors centered around Coursera's in-video quizzes. QuizCram shows users a question to answer, with an associated video segment. Users can use these questions to navigate through video segments, and find video segments they need to review. We also allow users to review using a timeline of previously answered questions and videos. To encourage users to review the material, QuizCram keeps track of their question-answering and video-watching history and schedules sections they likely have not mastered for review. QuizCram-format materials can be generated from existing lectures with in-video quizzes. Our user study comparing QuizCram to in-video quizzes found that users practice answering and reviewing questions more when using QuizCram, and are better able to remember answers to questions they encountered.
DQN-Based Multi-User Power Allocation for Hybrid RF/VLC Networks
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, a Deep Q-Network (DQN) based multi-agent multi-user power allocation algorithm is proposed for hybrid networks composed of radio frequency (RF) and visible light communication (VLC) access points (APs). The users are capable of multihoming, which can bridge RF and VLC links for accommodating their bandwidth requirements. By leveraging a non-cooperative multi-agent DQN algorithm, where each AP is an agent, an online power allocation strategy is developed to optimize the transmit power for providing users' required data rate. Our simulation results demonstrate that DQN's median convergence time training is 90% shorter than the Q-Learning (QL) based algorithm. The DQN-based algorithm converges to the desired user rate in half duration on average while converging with the rate of 96.1% compared to the QL-based algorithm's convergence rate of 72.3% Additionally, thanks to its continuous state-space definition, the DQN-based power allocation algorithm provides average user data rates closer to the target rates than the QL-based algorithm when it converges.
Temporal Motifs in Smart Grid
Venue: arXiv cs.SI 2021
Year: 2021

A complex network can be characterized by patterns. Such frequently occurring significant patterns are called motifs and in a time dependent network, they are called temporal motifs. One of the temporal networks where temporal motifs are observed and play a major role; is the Smart Grid. The energy consumption pattern across the appliances, houses, communities and entire cities help energy utility companies and consumers plan their electricity generation and consumption. The temporal motifs for the smart grid constitutes of the consumers and producers and the edge or connection represents energy flow between two participants of the network, these connections last till the power is being consumed/generated. This paper formally defines the temporal motifs for smart grid network and proposes a way to create such temporal motifs in the network. We also discuss how the temporal motifs fit into the hierarchical structure of power distribution system of Smart Grid.