OpenRaiser/Intern-Atlas huggingface dataset:
Undersensitivity in Neural Reading Comprehension
Venue: arXiv cs.CL 2020
Year: 2020

Current reading comprehension models generalise well to in-distribution test sets, yet perform poorly on adversarially selected inputs. Most prior work on adversarial inputs studies oversensitivity: semantically invariant text perturbations that cause a model's prediction to change when it should not. In this work we focus on the complementary problem: excessive prediction undersensitivity, where input text is meaningfully changed but the model's prediction does not, even though it should. We formulate a noisy adversarial attack which searches among semantic variations of the question for which a model erroneously predicts the same answer, and with even higher probability. Despite comprising unanswerable questions, both SQuAD2.0 and NewsQA models are vulnerable to this attack. This indicates that although accurate, models tend to rely on spurious patterns and do not fully consider the information specified in a question. We experiment with data augmentation and adversarial training as defences, and find that both substantially decrease vulnerability to attacks on held out data, as well as held out attack spaces. Addressing undersensitivity also improves results on AddSent and AddOneSent, and models furthermore generalise better when facing train/evaluation distribution mismatch: they are less prone to overly rely on predictive cues present only in the training set, and outperform a conventional model by as much as 10.9% F1.
SAD: Saliency-based Defenses Against Adversarial Examples
Venue: arXiv cs.CV 2020
Year: 2020

With the rise in popularity of machine and deep learning models, there is an increased focus on their vulnerability to malicious inputs. These adversarial examples drift model predictions away from the original intent of the network and are a growing concern in practical security. In order to combat these attacks, neural networks can leverage traditional image processing approaches or state-of-the-art defensive models to reduce perturbations in the data. Defensive approaches that take a global approach to noise reduction are effective against adversarial attacks, however their lossy approach often distorts important data within the image. In this work, we propose a visual saliency based approach to cleaning data affected by an adversarial attack. Our model leverages the salient regions of an adversarial image in order to provide a targeted countermeasure while comparatively reducing loss within the cleaned images. We measure the accuracy of our model by evaluating the effectiveness of state-of-the-art saliency methods prior to attack, under attack, and after application of cleaning methods. We demonstrate the effectiveness of our proposed approach in comparison with related defenses and against established adversarial attack methods, across two saliency datasets. Our targeted approach shows significant improvements in a range of standard statistical and distance saliency metrics, in comparison with both traditional and state-of-the-art approaches.
RealityCheck: Bringing Modularity, Hierarchy, and Abstraction to Automated Microarchitectural Memory Consistency Verification
Venue: arXiv cs.DC 2020
Year: 2020

Modern SoCs are heterogeneous parallel systems comprised of components developed by distinct teams and possibly even different vendors. The memory consistency model (MCM) of processors in such SoCs specifies the ordering rules which constrain the values that can be read by load instructions in parallel programs running on such systems. The implementation of required MCM orderings can span components which may be designed and implemented by many different teams. Ideally, each team would be able to specify the orderings enforced by their components independently and then connect them together when conducting MCM verification. However, no prior automated approach for formal hardware MCM verification provided this. To bring automated hardware MCM verification in line with the realities of the design process, we present RealityCheck, a methodology and tool for automated formal MCM verification of modular microarchitectural ordering specifications. RealityCheck allows users to specify their designs as a hierarchy of distinct modules connected to each other rather than a single flat specification. It can then automatically verify litmus test programs against these modular specifications. RealityCheck also provides support for abstraction, which enables scalable verification by breaking up the verification of the entire design into smaller verification problems. We present results for verifying litmus tests on 7 different designs using RealityCheck. These include in-order and out-of-order pipelines, a non-blocking cache, and a heterogeneous processor. Our case studies cover the TSO and RISC-V (RVWMO) weak memory models. RealityCheck is capable of verifying 98 RVWMO litmus tests in under 4 minutes each, and its capability for abstraction enables up to a 32.1% reduction in litmus test verification time for RVWMO.
A Practical Approach to Navigating the Tradeoff Between Privacy and Precise Utility
Venue: arXiv cs.IT 2020
Year: 2020

Due to the recent popularity of online social networks, coupled with people's propensity to disclose personal information in an effort to achieve certain gratifications, the problem of navigating the tradeoff between privacy and utility attracted a lot of recent interest and generated a rich body of research. A critical prerequisite to solving the problem is to appropriately capture the privacy and the utility aspects in the problem formulation. Most of the existing works' focus is on the notion of privacy, while utility loss is often treated as the undesirable but necessary distortion of the true data, introduced by the privacy mechanism. By contrast, we are interested in modelling utility differently, by associating it with specific attributes of a user, just like privacy is associated with specific private attributes in the literature. Our model of utility facilitates a better and more precise privacy mechanism, and achieves better privacy-utility tradeoffs. We further incorporate into our problem formulation a practical constraint on acceptable loss in utility per unit gain in privacy, which allows users to customize the privacy mechanisms in order to account for the relative values that each user associates with their own privacy and utility. This paper discusses the intricacies of our utility model and the corresponding privacy-utility tradeoff, and introduces a heuristic greedy algorithm to solve the problem.
A Fractional-Order Normalized Bouc-Wen Model for Piezoelectric Hysteresis Nonlinearity
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a new fractional-order normalized Bouc-Wen (BW) (FONBW) model to describe the asymmetric and rate-dependent hysteresis nonlinearity of piezoelectric actuators (PEAs). In view of the fact that the classical BW (CBW) model is only efficient for the symmetric and rate-independent hysteresis description, the FONBW model is devoted to characterizing the asymmetric and rate-dependent behaviors of the hysteresis in PEAs by adopting an Nth-order polynomial input function and two fractional operators, respectively. Different from the traditional modified BW models, the proposed FONBW model also eliminates the redundancy of parameters in the CBW model via the normalization processing. By this way, the developed FONBW model has a relatively simple mathematic expression with fewer parameters to simultaneously characterize the asymmetric and rate-dependent hysteresis behaviors of PEAs. Model parameters are identified by the self-adaptive differential evolution algorithm. To validate the effectiveness of the proposed model, a series of model verification and inverse-multiplicative-structure-based feedforward control experiments are carried out on a PEA system. Results show that the proposed model is superior to the CBW model and traditional modified BW model in modeling accuracy and hysteresis compensation.
IoT Expunge: Implementing Verifiable Retention of IoT Data
Venue: arXiv cs.CR 2020
Year: 2020

The growing deployment of Internet of Things (IoT) systems aims to ease the daily life of end-users by providing several value-added services. However, IoT systems may capture and store sensitive, personal data about individuals in the cloud, thereby jeopardizing user-privacy. Emerging legislation, such as California's CalOPPA and GDPR in Europe, support strong privacy laws to protect an individual's data in the cloud. One such law relates to strict enforcement of data retention policies. This paper proposes a framework, entitled IoT Expunge that allows sensor data providers to store the data in cloud platforms that will ensure enforcement of retention policies. Additionally, the cloud provider produces verifiable proofs of its adherence to the retention policies. Experimental results on a real-world smart building testbed show that IoT Expunge imposes minimal overheads to the user to verify the data against data retention policies.
Fake News Detection with Different Models
Venue: arXiv cs.CL 2020
Year: 2020

This is a paper for exploring various different models aiming at developing fake news detection models and we had used certain machine learning algorithms and we had used pretrained algorithms such as TFIDF and CV and W2V as features for processing textual data.
Improving Reliability of Latent Dirichlet Allocation by Assessing Its Stability Using Clustering Techniques on Replicated Runs
Venue: arXiv cs.CL 2020
Year: 2020

For organizing large text corpora topic modeling provides useful tools. A widely used method is Latent Dirichlet Allocation (LDA), a generative probabilistic model which models single texts in a collection of texts as mixtures of latent topics. The assignments of words to topics rely on initial values such that generally the outcome of LDA is not fully reproducible. In addition, the reassignment via Gibbs Sampling is based on conditional distributions, leading to different results in replicated runs on the same text data. This fact is often neglected in everyday practice. We aim to improve the reliability of LDA results. Therefore, we study the stability of LDA by comparing assignments from replicated runs. We propose to quantify the similarity of two generated topics by a modified Jaccard coefficient. Using such similarities, topics can be clustered. A new pruning algorithm for hierarchical clustering results based on the idea that two LDA runs create pairs of similar topics is proposed. This approach leads to the new measure S-CLOP ({\bf S}imilarity of multiple sets by {\bf C}lustering with {\bf LO}cal {\bf P}runing) for quantifying the stability of LDA models. We discuss some characteristics of this measure and illustrate it with an application to real data consisting of newspaper articles from \textit{USA Today}. Our results show that the measure S-CLOP is useful for assessing the stability of LDA models or any other topic modeling procedure that characterize its topics by word distributions. Based on the newly proposed measure for LDA stability, we propose a method to increase the reliability and hence to improve the reproducibility of empirical findings based on topic modeling. This increase in reliability is obtained by running the LDA several times and taking as prototype the most representative run, that is the LDA run with highest average similarity to all other runs.
Understanding the Downstream Instability of Word Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

Many industrial machine learning (ML) systems require frequent retraining to keep up-to-date with constantly changing data. This retraining exacerbates a large challenge facing ML systems today: model training is unstable, i.e., small changes in training data can cause significant changes in the model's predictions. In this paper, we work on developing a deeper understanding of this instability, with a focus on how a core building block of modern natural language processing (NLP) pipelines---pre-trained word embeddings---affects the instability of downstream NLP models. We first empirically reveal a tradeoff between stability and memory: increasing the embedding memory 2x can reduce the disagreement in predictions due to small changes in training data by 5% to 37% (relative). To theoretically explain this tradeoff, we introduce a new measure of embedding instability---the eigenspace instability measure---which we prove bounds the disagreement in downstream predictions introduced by the change in word embeddings. Practically, we show that the eigenspace instability measure can be a cost-effective way to choose embedding parameters to minimize instability without training downstream models, outperforming other embedding distance measures and performing competitively with a nearest neighbor-based measure. Finally, we demonstrate that the observed stability-memory tradeoffs extend to other types of embeddings as well, including knowledge graph and contextual word embeddings.
ScopeIt: Scoping Task Relevant Sentences in Documents
Venue: arXiv cs.CL 2020
Year: 2020

Intelligent assistants like Cortana, Siri, Alexa, and Google Assistant are trained to parse information when the conversation is synchronous and short; however, for email-based conversational agents, the communication is asynchronous, and often contains information irrelevant to the assistant. This makes it harder for the system to accurately detect intents, extract entities relevant to those intents and thereby perform the desired action. We present a neural model for scoping relevant information for the agent from a large query. We show that when used as a preprocessing step, the model improves performance of both intent detection and entity extraction tasks. We demonstrate the model's impact on Scheduler (Cortana is the persona of the agent, while Scheduler is the name of the service. We use them interchangeably in the context of this paper.) - a virtual conversational meeting scheduling assistant that interacts asynchronously with users through email. The model helps the entity extraction and intent detection tasks requisite by Scheduler achieve an average gain of 35% in precision without any drop in recall. Additionally, we demonstrate that the same approach can be used for component level analysis in large documents, such as signature block identification.
Using an ensemble color space model to tackle adversarial examples
Venue: arXiv cs.CV 2020
Year: 2020

Minute pixel changes in an image drastically change the prediction that the deep learning model makes. One of the most significant problems that could arise due to this, for instance, is autonomous driving. Many methods have been proposed to combat this with varying amounts of success. We propose a 3 step method for defending such attacks. First, we denoise the image using statistical methods. Second, we show that adopting multiple color spaces in the same model can help us to fight these adversarial attacks further as each color space detects certain features explicit to itself. Finally, the feature maps generated are enlarged and sent back as an input to obtain even smaller features. We show that the proposed model does not need to be trained to defend an particular type of attack and is inherently more robust to black-box, white-box, and grey-box adversarial attack techniques. In particular, the model is 56.12 percent more robust than compared models in case of white box attacks when the models are not subject to adversarial example training.
Coded Caching in Multi-server System with Random Topology
Venue: arXiv cs.IT 2020
Year: 2020

Cache-aided content delivery is studied in a multi-server system with $P$ servers and $K$ users, each equipped with a local cache memory. In the delivery phase, each user connects randomly to any $ρ$ out of $P$ servers. Thanks to the availability of multiple servers, which model small-cell base stations (SBSs), demands can be satisfied with reduced storage capacity at each server and reduced delivery rate per server; however, this also leads to reduced multicasting opportunities compared to the single-server scenario. A joint storage and proactive caching scheme is proposed, which exploits coded storage across the servers, uncoded cache placement at the users, and coded delivery. The delivery \textit{latency} is studied for both \textit{successive} and \textit{parallel} transmissions from the servers. It is shown that, with successive transmissions the achievable average delivery latency is comparable to the one achieved in the single-server scenario, while the gap between the two depends on $ρ$, the available redundancy across the servers, and can be reduced by increasing the storage capacity at the SBSs. The optimality of the proposed scheme with uncoded cache placement and MDS-coded server storage is also proved for successive transmissions.
Designing False Data Injection attacks penetrating AC-based Bad Data Detection System and FDI Dataset generation
Venue: arXiv cs.CR 2020
Year: 2020

The evolution of the traditional power system towards the modern smart grid has posed many new cybersecurity challenges to this critical infrastructure. One of the most dangerous cybersecurity threats is the False Data Injection (FDI) attack, especially when it is capable of completely bypassing the widely deployed Bad Data Detector of State Estimation and interrupting the normal operation of the power system. Most of the simulated FDI attacks are designed using simplified linearized DC model while most industry standard State Estimation systems are based on the nonlinear AC model. In this paper, a comprehensive FDI attack scheme is presented based on the nonlinear AC model. A case study of the nine-bus Western System Coordinated Council (WSCC)'s power system is provided, using an industry standard package to assess the outcomes of the proposed design scheme. A public FDI dataset is generated as a test set for the community to develop and evaluate new detection algorithms, which are lacking in the field. The FDI's stealthy quality of the dataset is assessed and proven through a preliminary analysis based on both physical power law and statistical analysis.
A Simulation Study of Bandit Algorithms to Address External Validity of Software Fault Prediction
Venue: arXiv cs.SE 2020
Year: 2020

Various software fault prediction models and techniques for building algorithms have been proposed. Many studies have compared and evaluated them to identify the most effective ones. However, in most cases, such models and techniques do not have the best performance on every dataset. This is because there is diversity of software development datasets, and therefore, there is a risk that the selected model or technique shows bad performance on a certain dataset. To avoid selecting a low accuracy model, we apply bandit algorithms to predict faults. Consider a case where player has 100 coins to bet on several slot machines. Ordinary usage of software fault prediction is analogous to the player betting all 100 coins in one slot machine. In contrast, bandit algorithms bet one coin on each machine (i.e., use prediction models) step-by-step to seek the best machine. In the experiment, we developed an artificial dataset that includes 100 modules, 15 of which include faults. Then, we developed various artificial fault prediction models and selected them dynamically using bandit algorithms. The Thomson sampling algorithm showed the best or second-best prediction performance compared with using only one prediction model.
Hundred Drones Land in a Minute
Venue: arXiv cs.RO 2020
Year: 2020

Currently, drone research and development has received significant attention worldwide. Particularly, delivery services employ drones as it is a viable method to improve delivery efficiency by using a several unmanned drones. Research has been conducted to realize complete automation of drone control for such services. However, regarding the takeoff and landing port of the drones, conventional methods have focused on the landing operation of a single drone, and the continuous landing of multiple drones has not been realized. To address this issue, we propose a completely novel port system, "EAGLES Port," that allows several drones to continuously land and takeoff in a short time. Experiments verified that the landing time efficiency of the proposed port is ideally 7.5 times higher than that of conventional vertical landing systems. Moreover, the system can tolerate 270 mm of horizontal positional error, +-30 deg of angular error in the drone's approach (+-40 deg with the proposed gate mechanism), and up to 1.9 m/s of drone's approach speed. This technology significantly contributes to the scalability of drone usage. Therefore, it is critical for the development of a future drone port for the landing of automated drone swarms.
Delay-Adaptive Learning in Generalized Linear Contextual Bandits
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we consider online learning in generalized linear contextual bandits where rewards are not immediately observed. Instead, rewards are available to the decision-maker only after some delay, which is unknown and stochastic. We study the performance of two well-known algorithms adapted to this delayed setting: one based on upper confidence bounds, and the other based on Thompson sampling. We describe modifications on how these two algorithms should be adapted to handle delays and give regret characterizations for both algorithms. Our results contribute to the broad landscape of contextual bandits literature by establishing that both algorithms can be made to be robust to delays, thereby helping clarify and reaffirm the empirical success of these two algorithms, which are widely deployed in modern recommendation engines.
Ethical Guidelines for the Construction of Digital Nudges
Venue: arXiv cs.HC 2020
Year: 2020

Under certain circumstances, humans tend to behave in irrational ways, leading to situations in which they make undesirable choices. The concept of digital nudging addresses these limitations of bounded rationality by establishing a libertarian paternalist alternative to nudge users in virtual environments towards their own preferential choices. Thereby, choice architectures are designed to address biases and heuristics involved in cognitive thinking. As research on digital nudging has become increasingly popular in the Information Systems community, an increasing necessity for ethical guidelines has emerged around this concept to safeguard its legitimization in distinction to e.g. persuasion or manipulation. However, reflecting on ethical debates regarding digital nudging in academia, we find that current conceptualizations are scare. This is where on the basis of existing literature, we provide a conceptualization of ethical guidelines for the design of digital nudges, and thereby aim to ensure the applicability of nudging mechanisms in virtual environments.