OpenRaiser/Intern-Atlas huggingface dataset:
DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips
Venue: arXiv cs.CR 2020
Year: 2020

Security of machine learning is increasingly becoming a major concern due to the ubiquitous deployment of deep learning in many security-sensitive domains. Many prior studies have shown external attacks such as adversarial examples that tamper with the integrity of DNNs using maliciously crafted inputs. However, the security implication of internal threats (i.e., hardware vulnerability) to DNN models has not yet been well understood. In this paper, we demonstrate the first hardware-based attack on quantized deep neural networks-DeepHammer-that deterministically induces bit flips in model weights to compromise DNN inference by exploiting the rowhammer vulnerability. DeepHammer performs aggressive bit search in the DNN model to identify the most vulnerable weight bits that are flippable under system constraints. To trigger deterministic bit flips across multiple pages within reasonable amount of time, we develop novel system-level techniques that enable fast deployment of victim pages, memory-efficient rowhammering and precise flipping of targeted bits. DeepHammer can deliberately degrade the inference accuracy of the victim DNN system to a level that is only as good as random guess, thus completely depleting the intelligence of targeted DNN systems. We systematically demonstrate our attacks on real systems against 12 DNN architectures with 4 different datasets and different application domains. Our evaluation shows that DeepHammer is able to successfully tamper DNN inference behavior at run-time within a few minutes. We further discuss several mitigation techniques from both algorithm and system levels to protect DNNs against such attacks. Our work highlights the need to incorporate security mechanisms in future deep learning system to enhance the robustness of DNN against hardware-based deterministic fault injections.
Concentrated Differentially Private and Utility Preserving Federated Learning
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning is a machine learning setting where a set of edge devices collaboratively train a model under the orchestration of a central server without sharing their local data. At each communication round of federated learning, edge devices perform multiple steps of stochastic gradient descent with their local data and then upload the computation results to the server for model update. During this process, the challenge of privacy leakage arises due to the information exchange between edge devices and the server when the server is not fully trusted. While some previous privacy-preserving mechanisms could readily be used for federated learning, they usually come at a high cost on convergence of the algorithm and utility of the learned model. In this paper, we develop a federated learning approach that addresses the privacy challenge without much degradation on model utility through a combination of local gradient perturbation, secure aggregation, and zero-concentrated differential privacy (zCDP). We provide a tight end-to-end privacy guarantee of our approach and analyze its theoretical convergence rates. Through extensive numerical experiments on real-world datasets, we demonstrate the effectiveness of our proposed method and show its superior trade-off between privacy and model utility.
Optimal Number of Cluster Heads in Wireless Sensors Networks Based on LEACH
Venue: arXiv cs.NI 2020
Year: 2020

The Wireless Sensor Network (WSN) has been one of the leading research fields of wireless networks, particularly in recent year. Sensors are randomly positioned in the region, every node senses the surroundings and sends the data collected to the cluster head (CH), which aggregates and transmits obtained information to the Base Station (BS). A non-rechargeable battery is included on each WSN node. The sensor energy and network life extension of WSN are the most important considerations in the academia and industry. And thus, many routing protocols have been proposed to solve this issue, one of these is LEACH, the early protocol that introduced the clustering idea to extend the life of the WSN. LEACH is affected by the number of heads of the Clusters as it is randomly selected, and this has an impact on network lifetime, furthermore, the nodes are randomly joined in each cluster, this means that some of the cluster heads work more than others with fewer cluster nodes. In this paper, it is proposed an algorithm Maximum Optimal Number of Cluster Heads (MONCH) to identify the optimum cluster heads in WSN and to find which is the nearest one to BS and helps to integrate nodes with the most appropriate cluster. The results show improved energy consumption performance for the LEACH algorithm.
NukeBERT: A Pre-trained language model for Low Resource Nuclear Domain
Venue: arXiv cs.LG 2020
Year: 2020

Significant advances have been made in recent years on Natural Language Processing with machines surpassing human performance in many tasks, including but not limited to Question Answering. The majority of deep learning methods for Question Answering targets domains with large datasets and highly matured literature. The area of Nuclear and Atomic energy has largely remained unexplored in exploiting non-annotated data for driving industry viable applications. Due to lack of dataset, a new dataset was created from the 7000 research papers on nuclear domain. This paper contributes to research in understanding nuclear domain knowledge which is then evaluated on Nuclear Question Answering Dataset (NQuAD) created by nuclear domain experts as part of this research. NQuAD contains 612 questions developed on 181 paragraphs randomly selected from the IGCAR research paper corpus. In this paper, the Nuclear Bidirectional Encoder Representational Transformers (NukeBERT) is proposed, which incorporates a novel technique for building BERT vocabulary to make it suitable for tasks with less training data. The experiments evaluated on NQuAD revealed that NukeBERT was able to outperform BERT significantly, thus validating the adopted methodology. Training NukeBERT is computationally expensive and hence we will be open-sourcing the NukeBERT pretrained weights and NQuAD for fostering further research work in the nuclear domain.
A Pebble in the AI Race
Venue: arXiv cs.AI 2020
Year: 2020

Bhutan is sometimes described as \a pebble between two boulders", a small country caught between the two most populous nations on earth: India and China. This pebble is, however, about to be caught up in a vortex: the transformation of our economic, political and social orders by new technologies like Artificial Intelligence. What can a small nation like Bhutan hope to do in the face of such change? What should the nation do, not just to weather this storm, but to become a better place in which to live?
Mining International Political Norms from the GDELT Database
Venue: arXiv cs.AI 2020
Year: 2020

Researchers have long been interested in the role that norms can play in governing agent actions in multi-agent systems. Much work has been done on formalising normative concepts from human society and adapting them for the government of open software systems, and on the simulation of normative processes in human and artificial societies. However, there has been comparatively little work on applying normative MAS mechanisms to understanding the norms in human society. This work investigates this issue in the context of international politics. Using the GDELT dataset, containing machine-encoded records of international events extracted from news reports, we extracted bilateral sequences of inter-country events and applied a Bayesian norm mining mechanism to identify norms that best explained the observed behaviour. A statistical evaluation showed that the normative model fitted the data significantly better than a probabilistic discrete event model.
Prediction Confidence from Neighbors
Venue: arXiv cs.CV 2020
Year: 2020

The inability of Machine Learning (ML) models to successfully extrapolate correct predictions from out-of-distribution (OoD) samples is a major hindrance to the application of ML in critical applications. Until the generalization ability of ML methods is improved it is necessary to keep humans in the loop. The need for human supervision can only be reduced if it is possible to determining a level of confidence in predictions, which can be used to either ask for human assistance or to abstain from making predictions. We show that feature space distance is a meaningful measure that can provide confidence in predictions. The distance between unseen samples and nearby training samples proves to be correlated to the prediction error of unseen samples. Depending on the acceptable degree of error, predictions can either be trusted or rejected based on the distance to training samples. %Additionally, a novelty threshold can be used to decide whether a sample is worth adding to the training set. This enables earlier and safer deployment of models in critical applications and is vital for deploying models under ever-changing conditions.
ChangeBeadsThreader: An Interactive Environment for Tailoring Automatically Untangled Changes
Venue: arXiv cs.SE 2020
Year: 2020

To improve the usability of a revision history, change untangling, which reconstructs the history to ensure that changes in each commit belong to one intentional task, is important. Although there are several untangling approaches based on the clustering of fine-grained editing operations of source code, they often produce unsuitable result for a developer, and manual tailoring of the result is necessary. In this paper, we propose ChangeBeadsThreader (CBT), an interactive environment for splitting and merging change clusters to support the manual tailoring of untangled changes. CBT provides two features: 1) a two-dimensional space where fine-grained change history is visualized to help users find the clusters to be merged and 2) an augmented diff view that enables users to confirm the consistency of the changes in a specific cluster for finding those to be split. These features allow users to easily tailor automatically untangled changes.
Static vs accumulating priorities in healthcare queues under heavy loads
Venue: arXiv cs.PF 2020
Year: 2020

Amid unprecedented times caused by COVID-19, healthcare systems all over the world are strained to the limits of, or even beyond, capacity. A similar event is experienced by some healthcare systems regularly, due to for instance seasonal spikes in the number of patients. We model this as a queueing system in heavy traffic (where the arrival rate is approaching the service rate from below) or in overload (where the arrival rate exceeds the service rate). In both cases we assume that customers (patients) may have different priorities and we consider two popular service disciplines: static priorities and accumulating priorities. It has been shown that the latter allows for patients of all classes to be seen in a timely manner as long as the system is stable. We demonstrate however that if accumulating priorities are used in the heavy traffic or overload regime, then all patients, including those with the highest priority, will experience very long waiting times. If on the other hand static priorities are applied, then one can ensure that the highest-priority patients will be seen in a timely manner even in overloaded systems.
Security Assurance Cases for Road Vehicles: an Industry Perspective
Venue: arXiv cs.SE 2020
Year: 2020

Assurance cases are structured arguments that are commonly used to reason about the safety of a product or service. Currently, there is an ongoing push towards using assurance cases for also cybersecurity, especially in safety-critical domains, like automotive. While the industry is faced with the challenge of defining a sound methodology to build security assurance cases, the state of the art is rather immature. Therefore, we have conducted a thorough investigation of the (external) constraints and (internal) needs that security assurance cases have to satisfy in the context of the automotive industry. This has been done in the context of two large automotive companies in Sweden. The end result is a set of recommendations that automotive companies can apply in order to define security assurance cases that are (i) aligned with the constraints imposed by the existing and upcoming standards and regulations and (ii)harmonized with the internal product development processes and organizational practices. We expect the results to be also of interest for product companies in other safety-critical domains, like healthcare, transportation, and so on
HATSUKI : An anime character like robot figure platform with anime-style expressions and imitation learning based action generation
Venue: arXiv cs.RO 2020
Year: 2020

Japanese character figurines are popular and have pivot position in Otaku culture. Although numerous robots have been developed, less have focused on otaku-culture or on embodying the anime character figurine. Therefore, we take the first steps to bridge this gap by developing Hatsuki, which is a humanoid robot platform with anime based design. Hatsuki's novelty lies in aesthetic design, 2D facial expressions, and anime-style behaviors that allows it to deliver rich interaction experiences resembling anime-characters. We explain our design implementation process of Hatsuki, followed by our evaluations. In order to explore user impressions and opinions towards Hatsuki, we conducted a questionnaire in the world's largest anime-figurine event. The results indicate that participants were generally very satisfied with Hatsuki's design, and proposed various use case scenarios and deployment contexts for Hatsuki. The second evaluation focused on imitation learning, as such method can provide better interaction ability in the real world and generate rich, context-adaptive behavior in different situations. We made Hatsuki learn 11 actions, combining voice, facial expressions and motions, through neuron network based policy model with our proposed interface. Results show our approach was successfully able to generate the actions through self-organized contexts, which shows the potential for generalizing our approach in further actions under different contexts. Lastly, we present our future research direction for Hatsuki, and provide our conclusion.
Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representation
Venue: arXiv cs.CV 2020
Year: 2020

We infer and generate three-dimensional (3D) scene information from a single input image and without supervision. This problem is under-explored, with most prior work relying on supervision from, e.g., 3D ground-truth, multiple images of a scene, image silhouettes or key-points. We propose Pix2Shape, an approach to solve this problem with four components: (i) an encoder that infers the latent 3D representation from an image, (ii) a decoder that generates an explicit 2.5D surfel-based reconstruction of a scene from the latent code (iii) a differentiable renderer that synthesizes a 2D image from the surfel representation, and (iv) a critic network trained to discriminate between images generated by the decoder-renderer and those from a training distribution. Pix2Shape can generate complex 3D scenes that scale with the view-dependent on-screen resolution, unlike representations that capture world-space resolution, i.e., voxels or meshes. We show that Pix2Shape learns a consistent scene representation in its encoded latent space and that the decoder can then be applied to this latent representation in order to synthesize the scene from a novel viewpoint. We evaluate Pix2Shape with experiments on the ShapeNet dataset as well as on a novel benchmark we developed, called 3D-IQTT, to evaluate models based on their ability to enable 3d spatial reasoning. Qualitative and quantitative evaluation demonstrate Pix2Shape's ability to solve scene reconstruction, generation, and understanding tasks.
A Graph Attention Spatio-temporal Convolutional Network for 3D Human Pose Estimation in Video
Venue: arXiv cs.CV 2020
Year: 2020

Spatio-temporal information is key to resolve occlusion and depth ambiguity in 3D pose estimation. Previous methods have focused on either temporal contexts or local-to-global architectures that embed fixed-length spatio-temporal information. To date, there have not been effective proposals to simultaneously and flexibly capture varying spatio-temporal sequences and effectively achieves real-time 3D pose estimation. In this work, we improve the learning of kinematic constraints in the human skeleton: posture, local kinematic connections, and symmetry by modeling local and global spatial information via attention mechanisms. To adapt to single- and multi-frame estimation, the dilated temporal model is employed to process varying skeleton sequences. Also, importantly, we carefully design the interleaving of spatial semantics with temporal dependencies to achieve a synergistic effect. To this end, we propose a simple yet effective graph attention spatio-temporal convolutional network (GAST-Net) that comprises of interleaved temporal convolutional and graph attention blocks. Experiments on two challenging benchmark datasets (Human3.6M and HumanEva-I) and YouTube videos demonstrate that our approach effectively mitigates depth ambiguity and self-occlusion, generalizes to half upper body estimation, and achieves competitive performance on 2D-to-3D video pose estimation. Code, video, and supplementary information is available at: \href{http://www.juanrojas.net/gast/}{http://www.juanrojas.net/gast/}
UTxO- vs account-based smart contract blockchain programming paradigms
Venue: arXiv cs.LO 2020
Year: 2020

We implement two versions of a simple but illustrative smart contract: one in Solidity on the Ethereum blockchain platform, and one in Plutus on the Cardano platform, with annotated code excerpts and with source code attached. We get a clearer view of the Cardano programming model in particular by introducing a novel mathematical abstraction which we call Idealised EUTxO. For each version of the contract, we trace how the architectures of the underlying platforms and their mathematics affects the natural programming styles and natural classes of errors. We prove some simple but novel results about alpha-conversion and observational equivalence for Cardano, and explain why Ethereum does not have them. We conclude with a wide-ranging and detailed discussion in the light of the examples, mathematical model, and mathematical results so far.
Tactile Presentation of Network Data: Text, Matrix or Diagram?
Venue: arXiv cs.HC 2020
Year: 2020

Visualisations are commonly used to understand social, biological and other kinds of networks. Currently, we do not know how to effectively present network data to people who are blind or have low-vision (BLV). We ran a controlled study with 8 BLV participants comparing four tactile representations: organic node-link diagram, grid node-link diagram, adjacency matrix and braille list. We found that the node-link representations were preferred and more effective for path following and cluster identification while the matrix and list were better for adjacency tasks. This is broadly in line with findings for the corresponding visual representations.
On the Integration of LinguisticFeatures into Statistical and Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

New machine translations (MT) technologies are emerging rapidly and with them, bold claims of achieving human parity such as: (i) the results produced approach "accuracy achieved by average bilingual human translators" (Wu et al., 2017b) or (ii) the "translation quality is at human parity when compared to professional human translators" (Hassan et al., 2018) have seen the light of day (Laubli et al., 2018). Aside from the fact that many of these papers craft their own definition of human parity, these sensational claims are often not supported by a complete analysis of all aspects involved in translation. Establishing the discrepancies between the strengths of statistical approaches to MT and the way humans translate has been the starting point of our research. By looking at MT output and linguistic theory, we were able to identify some remaining issues. The problems range from simple number and gender agreement errors to more complex phenomena such as the correct translation of aspectual values and tenses. Our experiments confirm, along with other studies (Bentivogli et al., 2016), that neural MT has surpassed statistical MT in many aspects. However, some problems remain and others have emerged. We cover a series of problems related to the integration of specific linguistic features into statistical and neural MT, aiming to analyse and provide a solution to some of them. Our work focuses on addressing three main research questions that revolve around the complex relationship between linguistics and MT in general. We identify linguistic information that is lacking in order for automatic translation systems to produce more accurate translations and integrate additional features into the existing pipelines. We identify overgeneralization or 'algorithmic bias' as a potential drawback of neural MT and link it to many of the remaining linguistic issues.
Software-Defined Radio Implementation of Age-of-Information-Oriented Random Access
Venue: arXiv cs.NI 2020
Year: 2020

More and more emerging Internet of Things (IoT) applications involve status updates, where various IoT devices monitor certain physical processes and report their latest statuses to the relevant information fusion nodes. A new performance measure, termed the age of information (AoI), has recently been proposed to quantify the information freshness in time-critical IoT applications. Due to a large number of devices in future IoT networks, the decentralized channel access protocols (e.g. random access) are preferable thanks to their low network overhead. Built on the AoI concept, some recent efforts have developed several AoI-oriented ALOHA-like random access protocols for boosting the network-wide information freshness. However, all relevant works focused on theoretical designs and analysis. The development and implementation of a working prototype to evaluate and further improve these random access protocols in practice have been largely overlooked. Motivated as such, we build a software-defined radio (SDR) prototype for testing and comparing the performance of recently proposed AoI-oriented random access protocols. To this end, we implement a time-slotted wireless system by devising a simple yet effective over-the-air time synchronization scheme, in which beacons that serve as reference timing packets are broadcast by an access point from time to time. For a complete working prototype, we also design the frame structures of various packets exchanged within the system. Finally, we design a set of experiments, implement them on our prototype and test the considered algorithms in an office environment.
Artificial chemistry experiments with chemlambda, lambda calculus, interaction combinators
Venue: arXiv cs.AI 2020
Year: 2020

Given a graph rewrite system, a graph G is a quine graph if it has a non-void maximal collection of non-conflicting matches of left patterns of graphs rewrites, such that after the parallel application of the rewrites we obtain a graph isomorphic with G. Such graphs exhibit a metabolism, they can multiply or they can die, when reduced by a random rewriting algorithm. These are introductory notes to the pages of artificial chemistry experiments with chemlambda, lambda calculus or interaction combinators, available from the entry page https://chemlambda.github.io/index.html . The experiments are bundled into pages, all of them based on a library of programs, on a database which contains hundreds of graphs and on a database of about 150 pages of text comments and a collection of more than 200 animations, most of them which can be re-done live, via the programs. There are links to public repositories of other contributors to these experiments, with versions of these programs in python, haskell, awk or javascript.