OpenRaiser/Intern-Atlas huggingface dataset:
Charting Coordination Needs in Large-Scale Agile Organisationswith Boundary Objects and Methodological Islands
Venue: arXiv cs.SE 2020
Year: 2020

Large-scale system development companies are increasingly adopting agile methods. While this adoption may improve lead-times, such companies need to balance two trade-offs: (i) the need to have a uniform, consistent development method on system level with the need for specialised methods for teams in different disciplines(e.g., hardware, software, mechanics, sales, support); (ii) the need for comprehensive documentation on system level with the need to have lightweight documentation enabling iterative and agile work. With specialised methods for teams, isolated teams work within larger ecosystems of plan-driven culture, i.e., teams become agile "islands". At the boundaries, these teams share knowledge which needs to be managed well for a correct system to be developed. While it is useful to support diverse and specialised methods, it is important to understand which islands are repeatedly encountered, the reasons or factors triggering their existence, and how best to handle coordination between them. Based on a multiple case study, this work presents a catalogue of islands and the boundary objects between them. We believe this work will be beneficial to practitioners aiming to understand their ecosystems and researchers addressing communication and coordination challenges in large-scale development.
A Secure Federated Learning Framework for 5G Networks
Venue: arXiv cs.CR 2020
Year: 2020

Federated Learning (FL) has been recently proposed as an emerging paradigm to build machine learning models using distributed training datasets that are locally stored and maintained on different devices in 5G networks while providing privacy preservation for participants. In FL, the central aggregator accumulates local updates uploaded by participants to update a global model. However, there are two critical security threats: poisoning and membership inference attacks. These attacks may be carried out by malicious or unreliable participants, resulting in the construction failure of global models or privacy leakage of FL models. Therefore, it is crucial for FL to develop security means of defense. In this article, we propose a blockchain-based secure FL framework to create smart contracts and prevent malicious or unreliable participants from involving in FL. In doing so, the central aggregator recognizes malicious and unreliable participants by automatically executing smart contracts to defend against poisoning attacks. Further, we use local differential privacy techniques to prevent membership inference attacks. Numerical results suggest that the proposed framework can effectively deter poisoning and membership inference attacks, thereby improving the security of FL in 5G networks.
Some Useful Integral Representations for Information-Theoretic Analyses
Venue: arXiv cs.IT 2020
Year: 2020

This work is an extension of our earlier article, where a well-known integral representation of the logarithmic function was explored, and was accompanied with demonstrations of its usefulness in obtaining compact, easily-calculable, exact formulas for quantities that involve expectations of the logarithm of a positive random variable. Here, in the same spirit, we derive an exact integral representation (in one or two dimensions) of the moment of a nonnegative random variable, or the sum of such independent random variables, where the moment order is a general positive noninteger real (also known as fractional moments). The proposed formula is applied to a variety of examples with an information-theoretic motivation, and it is shown how it facilitates their numerical evaluations. In particular, when applied to the calculation of a moment of the sum of a large number, $n$, of nonnegative random variables, it is clear that integration over one or two dimensions, as suggested by our proposed integral representation, is significantly easier than the alternative of integrating over $n$ dimensions, as needed in the direct calculation of the desired moment.
A Distributed Approximate Nearest Neighbor Method for Real-Time Face Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Nowadays, face recognition and more generally image recognition have many applications in the modern world and are widely used in our daily tasks. This paper aims to propose a distributed approximate nearest neighbor (ANN) method for real-time face recognition using a big dataset that involves a lot of classes. The proposed approach is based on using a clustering method to separate the dataset into different clusters and on specifying the importance of each cluster by defining cluster weights. To this end, reference instances are selected from each cluster based on the cluster weights using a maximum likelihood approach. This process leads to a more informed selection of instances, so it enhances the performance of the algorithm. Experimental results confirm the efficiency of the proposed method and its out-performance in terms of accuracy and the processing time.
Unsupervised Anomaly Detection via Deep Metric Learning with End-to-End Optimization
Venue: arXiv cs.LG 2020
Year: 2020

We investigate unsupervised anomaly detection for high-dimensional data and introduce a deep metric learning (DML) based framework. In particular, we learn a distance metric through a deep neural network. Through this metric, we project the data into the metric space that better separates the anomalies from the normal data and reduces the effect of the curse of dimensionality for high-dimensional data. We present a novel data distillation method through self-supervision to remedy the conventional practice of assuming all data as normal. We also employ the hard mining technique from the DML literature. We show these components improve the performance of our model and significantly reduce the running time. Through an extensive set of experiments on the 14 real-world datasets, our method demonstrates significant performance gains compared to the state-of-the-art unsupervised anomaly detection methods, e.g., an absolute improvement between 4.44% and 11.74% on the average over the 14 datasets. Furthermore, we share the source code of our method on Github to facilitate further research.
Recurrent and Spiking Modeling of Sparse Surgical Kinematics
Venue: arXiv cs.CV 2020
Year: 2020

Robot-assisted minimally invasive surgery is improving surgeon performance and patient outcomes. This innovation is also turning what has been a subjective practice into motion sequences that can be precisely measured. A growing number of studies have used machine learning to analyze video and kinematic data captured from surgical robots. In these studies, models are typically trained on benchmark datasets for representative surgical tasks to assess surgeon skill levels. While they have shown that novices and experts can be accurately classified, it is not clear whether machine learning can separate highly proficient surgeons from one another, especially without video data. In this study, we explore the possibility of using only kinematic data to predict surgeons of similar skill levels. We focus on a new dataset created from surgical exercises on a simulation device for skill training. A simple, efficient encoding scheme was devised to encode kinematic sequences so that they were amenable to edge learning. We report that it is possible to identify surgical fellows receiving near perfect scores in the simulation exercises based on their motion characteristics alone. Further, our model could be converted to a spiking neural network to train and infer on the Nengo simulation framework with no loss in accuracy. Overall, this study suggests that building neuromorphic models from sparse motion features may be a potentially useful strategy for identifying surgeons and gestures with chips deployed on robotic systems to offer adaptive assistance during surgery and training with additional latency and privacy benefits.
Heterogeneous CPU/GPU co-execution of CFD simulations on the POWER9 architecture: Application to airplane aerodynamics
Venue: arXiv cs.DC 2020
Year: 2020

High fidelity Computational Fluid Dynamics simulations are generally associated with large computing requirements, which are progressively acute with each new generation of supercomputers. However, significant research efforts are required to unlock the computing power of leading-edge systems, currently referred to as pre-Exascale systems, based on increasingly complex architectures. In this paper, we present the approach implemented in the computational mechanics code Alya. We describe in detail the parallelization strategy implemented to fully exploit the different levels of parallelism, together with a novel co-execution method for the efficient utilization of heterogeneous CPU/GPU architectures. The latter is based on a multi-code co-execution approach with a dynamic load balancing mechanism. The assessment of the performance of all the proposed strategies has been carried out for airplane simulations on the POWER9 architecture accelerated with NVIDIA Volta V100 GPUs.
Initial Conflicts for Transformation Rules with Nested Application Conditions
Venue: arXiv cs.LO 2020
Year: 2020

We extend the theory of initial conflicts in the framework of M-adhesive categories to transformation rules with ACs. We first show that for rules with ACs, conflicts are in general neither inherited from a bigger context any more, nor is it possible to find a finite and complete subset of finite conflicts as illustrated for the category of graphs. We define initial conflicts to be special so-called symbolic transformation pairs, and show that they are minimally complete (and in the case of graphs also finite) in this symbolic way. We show that initial conflicts represent a proper subset of critical pairs again. We moreover demonstrate that (analogous to the case of rules without ACs) for each conflict a unique initial conflict exists representing it. We conclude with presenting a sufficient condition illustrating important special cases for rules with ACs, where we do not only have initial conflicts being complete in a symbolic way, but also find complete (and in the case of graphs also finite) subsets of conflicts in the classical sense.
TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
Venue: arXiv cs.CL 2020
Year: 2020

While there has been substantial research using adversarial attacks to analyze NLP models, each attack is implemented in its own code repository. It remains challenging to develop NLP attacks and utilize them to improve model performance. This paper introduces TextAttack, a Python framework for adversarial attacks, data augmentation, and adversarial training in NLP. TextAttack builds attacks from four components: a goal function, a set of constraints, a transformation, and a search method. TextAttack's modular design enables researchers to easily construct attacks from combinations of novel and existing components. TextAttack provides implementations of 16 adversarial attacks from the literature and supports a variety of models and datasets, including BERT and other transformers, and all GLUE tasks. TextAttack also includes data augmentation and adversarial training modules for using components of adversarial attacks to improve model accuracy and robustness. TextAttack is democratizing NLP: anyone can try data augmentation and adversarial training on any model or dataset, with just a few lines of code. Code and tutorials are available at https://github.com/QData/TextAttack.
An Economic Model for Quantum Key-Recovery Attacks against Ideal Ciphers
Venue: arXiv cs.CR 2020
Year: 2020

It has been established that quantum algorithms can solve several key cryptographic problems more efficiently than classical computers. As progress continues in the field of quantum computing it is important to understand the risks they pose to deployed cryptographic systems. Here we focus on one of these risks - quantum key-recovery attacks against ideal ciphers. Specifically, we seek to model the risk posed by an economically motivated quantum attacker who will choose to run a quantum key-recovery attack against an ideal cipher if the cost to recover the secret key is less than the value of the information at the time when the key-recovery attack is complete. In our analysis we introduce the concept of a quantum cipher circuit year to measure the cost of a quantum attack. This concept can be used to model the inherent tradeoff between the total time to run a quantum key recovery attack and the total work required to run said attack. Our model incorporates the time value of the encrypted information to predict whether any time/work tradeoff results in a key-recovery attack with positive utility for the attacker. We make these predictions under various projections of advances in quantum computing. We use these predictions to make recommendations for the future use and deployment of symmetric key ciphers to secure information against these quantum key-recovery attacks. We argue that, even with optimistic predictions for advances in quantum computing, 128 bit keys (as used in common cipher implementations like AES-128) provide adequate security against quantum attacks in almost all use cases.
Blockchain Superoptimizer
Venue: arXiv cs.LO 2020
Year: 2020

In the blockchain-based, distributed computing platform Ethereum, programs called smart contracts are compiled to bytecode and executed on the Ethereum Virtual Machine (EVM). Executing EVM bytecode is subject to monetary fees---a clear optimization target. Our aim is to superoptimize EVM bytecode by encoding the operational semantics of EVM instructions as SMT formulas and leveraging a constraint solver to automatically find cheaper bytecode. We implement this approach in our EVM Bytecode SuperOptimizer ebso and perform two large scale evaluations on real-world data sets.
Intersectional Bias in Hate Speech and Abusive Language Datasets
Venue: arXiv cs.CL 2020
Year: 2020

Algorithms are widely applied to detect hate speech and abusive language in social media. We investigated whether the human-annotated data used to train these algorithms are biased. We utilized a publicly available annotated Twitter dataset (Founta et al. 2018) and classified the racial, gender, and party identification dimensions of 99,996 tweets. The results showed that African American tweets were up to 3.7 times more likely to be labeled as abusive, and African American male tweets were up to 77% more likely to be labeled as hateful compared to the others. These patterns were statistically significant and robust even when party identification was added as a control variable. This study provides the first systematic evidence on intersectional bias in datasets of hate speech and abusive language.
Localized convolutional neural networks for geospatial wind forecasting
Venue: arXiv cs.LG 2020
Year: 2020

Convolutional Neural Networks (CNN) possess many positive qualities when it comes to spatial raster data. Translation invariance enables CNNs to detect features regardless of their position in the scene. However, in some domains, like geospatial, not all locations are exactly equal. In this work, we propose localized convolutional neural networks that enable convolutional architectures to learn local features in addition to the global ones. We investigate their instantiations in the form of learnable inputs, local weights, and a more general form. They can be added to any convolutional layers, easily end-to-end trained, introduce minimal additional complexity, and let CNNs retain most of their benefits to the extent that they are needed. In this work we address spatio-temporal prediction: test the effectiveness of our methods on a synthetic benchmark dataset and tackle three real-world wind prediction datasets. For one of them, we propose a method to spatially order the unordered data. We compare the recent state-of-the-art spatio-temporal prediction models on the same data. Models that use convolutional layers can be and are extended with our localizations. In all these cases our extensions improve the results, and thus often the state-of-the-art. We share all the code at a public repository.
RSO: A Gradient Free Sampling Based Approach For Training Deep Neural Networks
Venue: arXiv cs.LG 2020
Year: 2020

We propose RSO (random search optimization), a gradient free Markov Chain Monte Carlo search based approach for training deep neural networks. To this end, RSO adds a perturbation to a weight in a deep neural network and tests if it reduces the loss on a mini-batch. If this reduces the loss, the weight is updated, otherwise the existing weight is retained. Surprisingly, we find that repeating this process a few times for each weight is sufficient to train a deep neural network. The number of weight updates for RSO is an order of magnitude lesser when compared to backpropagation with SGD. RSO can make aggressive weight updates in each step as there is no concept of learning rate. The weight update step for individual layers is also not coupled with the magnitude of the loss. RSO is evaluated on classification tasks on MNIST and CIFAR-10 datasets with deep neural networks of 6 to 10 layers where it achieves an accuracy of 99.1% and 81.8% respectively. We also find that after updating the weights just 5 times, the algorithm obtains a classification accuracy of 98% on MNIST.
Towards Memory Safe Python Enclave for Security Sensitive Computation
Venue: arXiv cs.CR 2020
Year: 2020

Intel SGX Guard eXtensions (SGX), a hardware-supported trusted execution environment (TEE), is designed to protect security-sensitive applications. However, since enclave applications are developed with memory unsafe languages such as C/C++, traditional memory corruption is not eliminated in SGX. Rust-SGX is the first toolkit providing enclave developers with a memory-language. However, Rust is considered a Systems language and has become the right choice for concurrent applications and web browsers. Many application domains such as Big Data, Machine Learning, Robotics, Computer Vision are more commonly developed in the python programming language. Therefore, Python application developers cannot benefit from secure enclaves like Intel SGX and rust-SGX. To fill this gap, we propose Python-SGX, which is a memory-safe SGX SDK providing enclave developers a memory-safe Python development environment. The key idea is to enable memory-safe Python language in SGX by solving the following key challenges: (1) defining a memory-safe Python interpreter (2)replacing unsafe elements of Python interpreter with safe ones,(3) achieving comparable performance to non-enclave Python applications, and (4) not introducing any unsafe new code or libraries into SGX. We propose to build Python-SGX with PyPy, a Python interpreter written by RPython, which is a subset of Python, and tame unsafe parts in PyPy by formal verification, security hardening, and memory safe language. We have implemented python-SGX and tested it with a series of benchmarks programs. Our evaluation results show that Python-SGX does not cause significant overhead.
Enabling Deletion in Append-Only Blockchains (Short Summary / Work in Progress)
Venue: arXiv cs.CR 2020
Year: 2020

Conventional blockchain implementations with append-only semantics do not support deleting or overwriting data in confirmed blocks. However, many industry-relevant use cases require the ability to delete data, especially when personally identifiable information is stored or when data growth has to be constrained. Existing attempts to reconcile these contradictions compromise on core qualities of the blockchain paradigm, as they include backdoor-like approaches such as central authorities with elevated rights or usage of specialized chameleon hash algorithms in chaining of the blocks. In this technical report, we outline a novel architecture for the blockchain ledger and consensus, which uses a tree of context chains with simultaneous validity. A context chain captures the transactions of a closed group of entities and persons, thus structuring blocks in a precisely defined way. The resulting context isolation enables consensus-steered deletion of an entire context without side effects to other contextes. This architecture opens the possibility of truncation, data rollover and separation of concerns, and can help to fulfill the GDPR regulations.
Generalized Multi-view Shared Subspace Learning using View Bootstrapping
Venue: arXiv cs.LG 2020
Year: 2020

A key objective in multi-view learning is to model the information common to multiple parallel views of a class of objects/events to improve downstream learning tasks. In this context, two open research questions remain: How can we model hundreds of views per event? Can we learn robust multi-view embeddings without any knowledge of how these views are acquired? We present a neural method based on multi-view correlation to capture the information shared across a large number of views by subsampling them in a view-agnostic manner during training. To provide an upper bound on the number of views to subsample for a given embedding dimension, we analyze the error of the bootstrapped multi-view correlation objective using matrix concentration theory. Our experiments on spoken word recognition, 3D object classification and pose-invariant face recognition demonstrate the robustness of view bootstrapping to model a large number of views. Results underscore the applicability of our method for a view-agnostic learning setting.
Guaranteeing Reproducibility in Deep Learning Competitions
Venue: arXiv cs.LG 2020
Year: 2020

To encourage the development of methods with reproducible and robust training behavior, we propose a challenge paradigm where competitors are evaluated directly on the performance of their learning procedures rather than pre-trained agents. Since competition organizers re-train proposed methods in a controlled setting they can guarantee reproducibility, and -- by retraining submissions using a held-out test set -- help ensure generalization past the environments on which they were trained.
Visual Analytics and Human Involvement in Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

The rapidly developing AI systems and applications still require human involvement in practically all parts of the analytics process. Human decisions are largely based on visualizations, providing data scientists details of data properties and the results of analytical procedures. Different visualizations are used in the different steps of the Machine Learning (ML) process. The decision which visualization to use depends on factors, such as the data domain, the data model and the step in the ML process. In this chapter, we describe the seven steps in the ML process and review different visualization techniques that are relevant for the different steps for different types of data, models and purposes.