OpenRaiser/Intern-Atlas huggingface dataset:
Mid-Air Haptic Bio-Holograms in Mixed Reality
Venue: arXiv cs.HC 2020
Year: 2020

We present a prototype demonstrator that integrates three technologies, mixed reality head-mounted displays, wearable bio-sensors, and mid-air haptic projectors to deliver an interactive tactile experience with a bio-hologram. Users of this prototype are able to see, touch and feel a hologram of a heart that is beating at the same rhythm as their own. The demo uses an Ultrahaptics device, a Magic Leap One Mixed Reality headset, and an Apple Watch that measures the wearer's heart rate, all synchronized and networked together such that updates from the wristband dynamically change the haptic feedback and the animation speed of the beating heart thus creating a more personalised experience.
Issues with rounding in the GCC implementation of the ISO 18037:2008 standard fixed-point arithmetic
Venue: arXiv cs.MS 2020
Year: 2020

We describe various issues caused by the lack of round-to-nearest mode in the \textit{gcc} compiler implementation of the fixed-point arithmetic data types and operations. We demonstrate that round-to-nearest is not performed in the conversion of constants, conversion from one numerical type to a less precise type and results of multiplications. Furthermore, we show that mixed-precision operations in fixed-point arithmetic lose precision on arguments, even before carrying out arithmetic operations. The ISO 18037:2008 standard was created to standardize C language extensions, including fixed-point arithmetic, for embedded systems. Embedded systems are usually based on ARM processors, of which approximately 100 billion have been manufactured by now. Therefore, the observations about numerical issues that we discuss in this paper can be rather dangerous and are important to address, given the wide ranging type of applications that these embedded systems are running.
Think Locally, Act Globally: Federated Learning with Local and Global Representations
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning is a method of training models on private data distributed over multiple devices. To keep device data private, the global model is trained by only communicating parameters and updates which poses scalability challenges for large models. To this end, we propose a new federated learning algorithm that jointly learns compact local representations on each device and a global model across all devices. As a result, the global model can be smaller since it only operates on local representations, reducing the number of communicated parameters. Theoretically, we provide a generalization analysis which shows that a combination of local and global models reduces both variance in the data as well as variance across device distributions. Empirically, we demonstrate that local models enable communication-efficient training while retaining performance. We also evaluate on the task of personalized mood prediction from real-world mobile data where privacy is key. Finally, local models handle heterogeneous data from new devices, and learn fair representations that obfuscate protected attributes such as race, age, and gender.
Speaker-aware speech-transformer
Venue: arXiv cs.CL 2020
Year: 2020

Recently, end-to-end (E2E) models become a competitive alternative to the conventional hybrid automatic speech recognition (ASR) systems. However, they still suffer from speaker mismatch in training and testing condition. In this paper, we use Speech-Transformer (ST) as the study platform to investigate speaker aware training of E2E models. We propose a model called Speaker-Aware Speech-Transformer (SAST), which is a standard ST equipped with a speaker attention module (SAM). The SAM has a static speaker knowledge block (SKB) that is made of i-vectors. At each time step, the encoder output attends to the i-vectors in the block, and generates a weighted combined speaker embedding vector, which helps the model to normalize the speaker variations. The SAST model trained in this way becomes independent of specific training speakers and thus generalizes better to unseen testing speakers. We investigate different factors of SAM. Experimental results on the AISHELL-1 task show that SAST achieves a relative 6.5% CER reduction (CERR) over the speaker-independent (SI) baseline. Moreover, we demonstrate that SAST still works quite well even if the i-vectors in SKB all come from a different data source other than the acoustic training set.
Learning Reusable Options for Multi-Task Reinforcement Learning
Venue: arXiv cs.AI 2020
Year: 2020

Reinforcement learning (RL) has become an increasingly active area of research in recent years. Although there are many algorithms that allow an agent to solve tasks efficiently, they often ignore the possibility that prior experience related to the task at hand might be available. For many practical applications, it might be unfeasible for an agent to learn how to solve a task from scratch, given that it is generally a computationally expensive process; however, prior experience could be leveraged to make these problems tractable in practice. In this paper, we propose a framework for exploiting existing experience by learning reusable options. We show that after an agent learns policies for solving a small number of problems, we are able to use the trajectories generated from those policies to learn reusable options that allow an agent to quickly learn how to solve novel and related problems.
Exploring Adversarial Attack in Spiking Neural Networks with Spike-Compatible Gradient
Venue: arXiv cs.NE 2020
Year: 2020

Recently, backpropagation through time inspired learning algorithms are widely introduced into SNNs to improve the performance, which brings the possibility to attack the models accurately given Spatio-temporal gradient maps. We propose two approaches to address the challenges of gradient input incompatibility and gradient vanishing. Specifically, we design a gradient to spike converter to convert continuous gradients to ternary ones compatible with spike inputs. Then, we design a gradient trigger to construct ternary gradients that can randomly flip the spike inputs with a controllable turnover rate, when meeting all zero gradients. Putting these methods together, we build an adversarial attack methodology for SNNs trained by supervised algorithms. Moreover, we analyze the influence of the training loss function and the firing threshold of the penultimate layer, which indicates a "trap" region under the cross-entropy loss that can be escaped by threshold tuning. Extensive experiments are conducted to validate the effectiveness of our solution. Besides the quantitative analysis of the influence factors, we evidence that SNNs are more robust against adversarial attack than ANNs. This work can help reveal what happens in SNN attack and might stimulate more research on the security of SNN models and neuromorphic devices.
Why and How to Balance Alignment and Diversity of Requirements Engineering Practices in Automotive
Venue: arXiv cs.SE 2020
Year: 2020

In large-scale automotive companies, various requirements engineering (RE) practices are used across teams. RE practices manifest in Requirements Information Models (RIM) that define what concepts and information should be captured for requirements. Collaboration of practitioners from different parts of an organization is required to define a suitable RIM that balances support for diverse practices in individual teams with the alignment needed for a shared view and team support on system level. There exists no guidance for this challenging task. This paper presents a mixed methods study to examine the role of RIMs in balancing alignment and diversity of RE practices in four automotive companies. Our analysis is based on data from systems engineering tools, 11 semi-structured interviews, and a survey to validate findings and suggestions. We found that balancing alignment and diversity of RE practices is important to consider when defining RIMs. We further investigated enablers for this balance and actions that practitioners take to achieve it. From these factors, we derived and evaluated recommendations for managing RIMs in practice that take into account the lifecycle of requirements and allow for diverse practices across sub-disciplines in early development, while enforcing alignment of requirements that are close to release.
KYChain: User-Controlled KYC Data Sharing and Certification
Venue: arXiv cs.CR 2020
Year: 2020

Under Know Your Customer (KYC) regulations, financial institutions are required to verify the identity and assess the trustworthiness of any new client during on-boarding, and maintain up-to-date records for risk management. These processes are time consuming, expensive, typically have sub-par record-keeping steps, and disadvantage clients with nomad lifestyle. In this paper, we introduce KYChain as a privacy-preserving certification mechanism that allows users to share (certified) up-to-date KYC data across multiple financial institutions. We base KYChain on immutable ledgers and show that it offers confidentiality and certification compliance of KYC data.
Secret key authentication capacity region, Part I: average authentication rate
Venue: arXiv cs.IT 2020
Year: 2020

This paper investigates the secret key authentication capacity region. Specifically, the focus is on a model where a source must transmit information over an adversary controlled channel where the adversary, prior to the source's transmission, decides whether or not to replace the destination's observation with an arbitrary one of their choosing (done in hopes of having the destination accept a false message). To combat the adversary, the source and destination share a secret key which they may use to guarantee authenticated communications. The secret key authentication capacity region here is then defined as the region of jointly achievable message rate, authentication rate, and key consumption rate (i.e., how many bits of secret key are needed). This is the first of a two part study, with the parts differing in how the authentication rate is measured. In this first study the authenticated rate is measured by the traditional metric of the maximum expected probability of false authentication. For this metric, we provide an inner bound which improves on those existing in the literature. This is achieved by adopting and merging different classical techniques in novel ways. Within these classical techniques, one technique derives authentication capability directly from the noisy communications channel, and the other technique derives its' authentication capability directly from obscuring the source.
Investigation and Analysis of Hyper and Hypo neuron pruning to selectively update neurons during Unsupervised Adaptation
Venue: arXiv cs.LG 2020
Year: 2020

Unseen or out-of-domain data can seriously degrade the performance of a neural network model, indicating the model's failure to generalize to unseen data. Neural net pruning can not only help to reduce a model's size but can improve the model's generalization capacity as well. Pruning approaches look for low-salient neurons that are less contributive to a model's decision and hence can be removed from the model. This work investigates if pruning approaches are successful in detecting neurons that are either high-salient (mostly active or hyper) or low-salient (barely active or hypo), and whether removal of such neurons can help to improve the model's generalization capacity. Traditional blind adaptation techniques update either the whole or a subset of layers, but have never explored selectively updating individual neurons across one or more layers. Focusing on the fully connected layers of a convolutional neural network (CNN), this work shows that it may be possible to selectively adapt certain neurons (consisting of the hyper and the hypo neurons) first, followed by a full-network fine tuning. Using the task of automatic speech recognition, this work demonstrates how the removal of hyper and hypo neurons from a model can improve the model's performance on out-of-domain speech data and how selective neuron adaptation can ensure improved performance when compared to traditional blind model adaptation.
Modeling Uncertainty and Imprecision in Nonmonotonic Reasoning using Fuzzy Numbers
Venue: arXiv cs.AI 2020
Year: 2020

To deal with uncertainty in reasoning, interval-valued logic has been developed. But uniform intervals cannot capture the difference in degrees of belief for different values in the interval. To salvage the problem triangular and trapezoidal fuzzy numbers are used as the set of truth values along with traditional intervals. Preorder-based truth and knowledge ordering are defined over the set of fuzzy numbers defined over $[0,1]$. Based on this enhanced set of epistemic states, an answer set framework is developed, with properly defined logical connectives. This type of framework is efficient in knowledge representation and reasoning with vague and uncertain information under nonmonotonic environment where rules may posses exceptions.
The Pedestrian Patterns Dataset
Venue: arXiv cs.CV 2020
Year: 2020

We present the pedestrian patterns dataset for autonomous driving. The dataset was collected by repeatedly traversing the same three routes for one week starting at different specific timeslots. The purpose of the dataset is to capture the patterns of social and pedestrian behavior along the traversed routes at different times and to eventually use this information to make predictions about the risk associated with autonomously traveling along different routes. This dataset contains the Full HD videos and GPS data for each traversal. Fast R-CNN pedestrian detection method is applied to the captured videos to count the number of pedestrians at each video frame in order to assess the density of pedestrians along a route. By providing this large-scale dataset to researchers, we hope to accelerate autonomous driving research not only to estimate the risk, both to the public and to the autonomous vehicle but also accelerate research on long-term vision-based localization of mobile robots and autonomous vehicles of the future.
Protect Edge Privacy in Path Publishing with Differential Privacy
Venue: arXiv cs.CR 2020
Year: 2020

Paths in a given network are a generalised form of time-serial chains in many real-world applications, such as trajectories and Internet flows. Differentially private trajectory publishing concerns publishing path information that is usable to the genuine users yet secure against adversaries to reconstruct the path with maximum background knowledge. The exiting studies all assume this knowledge to be all but one vertex on the path. To prevent the adversaries recovering the missing information, they publish a perturbed path where each vertex is sampled from a pre-defined set with differential privacy (DP) to replace the corresponding vertex in the original path. In this paper, we relax this assumption to be all but one edge on the path, and hence consider the scenario of more powerful adversaries with the maximum background knowledge of the entire network topology and the path (including all the vertices) except one (arbitrary) missing edge. Under such an assumption, the perturbed path produced by the existing work is vulnerable, because the adversary can reconstruct the missing edge from the existence of an edge in the perturbed path. To address this vulnerability and effectively protect edge privacy, instead of publishing a perturbed path, we propose a novel scheme of graph-based path publishing to protect the original path by embedding the path in a graph that contains fake edges and replicated vertices applying the differential privacy technique, such that only the legitimate users who have the full knowledge of the network topology are able to recover the exact vertices and edges of the original path with high probability. We theoretically analyse the performance of our algorithm in differential privacy, utility, and execution efficiency. We also conduct extensive experimental evaluations on a high-performance cluster system to validate our analytical results.
Closed loop application of electroadhesion for increased precision in texture rendering
Venue: arXiv cs.HC 2020
Year: 2020

Tactile displays based on friction modulation offer wide-bandwidth forces rendered directly on the fingertip. However, due to a number of touch conditions (e.g., normal force, skin hydration) that result in variations in the friction force and the strength of modulation effect, the precision of the force rendering remains limited. In this paper we demonstrate a closed-loop electroadhesion method for precise playback of friction force profiles on a human finger and we apply this method to the tactile rendering of several textiles encountered in everyday life.
From Open Set to Closed Set: Supervised Spatial Divide-and-Conquer for Object Counting
Venue: arXiv cs.CV 2020
Year: 2020

Visual counting, a task that aims to estimate the number of objects from an image/video, is an open-set problem by nature, i.e., the number of population can vary in [0, inf) in theory. However, collected data and labeled instances are limited in reality, which means that only a small closed set is observed. Existing methods typically model this task in a regression manner, while they are prone to suffer from an unseen scene with counts out of the scope of the closed set. In fact, counting has an interesting and exclusive property---spatially decomposable. A dense region can always be divided until sub-region counts are within the previously observed closed set. We therefore introduce the idea of spatial divide-and-conquer (S-DC) that transforms open-set counting into a closed-set problem. This idea is implemented by a novel Supervised Spatial Divide-and-Conquer Network (SS-DCNet). Thus, SS-DCNet can only learn from a closed set but generalize well to open-set scenarios via S-DC. SS-DCNet is also efficient. To avoid repeatedly computing sub-region convolutional features, S-DC is executed on the feature map instead of on the input image. We provide theoretical analyses as well as a controlled experiment on toy data, demonstrating why closed-set modeling makes sense. Extensive experiments show that SS-DCNet achieves the state-of-the-art performance. Code and models are available at: https://tinyurl.com/SS-DCNet.
Monte Carlo Tree Search for Generating Interactive Data Analysis Interfaces
Venue: arXiv cs.DB 2020
Year: 2020

Interactive tools like user interfaces help democratize data access for end-users by hiding underlying programming details and exposing the necessary widget interface to users. Since customized interfaces are costly to build, automated interface generation is desirable. SQL is the dominant way to analyze data and there already exists logs to analyze data. Previous work proposed a syntactic approach to analyze structural changes in SQL query logs and automatically generates a set of widgets to express the changes. However, they do not consider layout usability and the sequential order of queries in the log. We propose to adopt Monte Carlo Tree Search(MCTS) to search for the optimal interface that accounts for hierarchical layout as well as the usability in terms of how easy to express the query log.
Federated Learning for Localization: A Privacy-Preserving Crowdsourcing Method
Venue: arXiv cs.NI 2020
Year: 2020

Received Signal Strength (RSS) fingerprint-based localization has attracted a lot of research effort and cultivated many commercial applications of location-based services due to its low cost and ease of implementation. Many studies are exploring the use of deep learning (DL) algorithms for localization. DL's ability to extract features and to classify autonomously makes it an attractive solution for fingerprint-based localization. These solutions require frequent retraining of DL models with vast amounts of measurements. Although crowdsourcing is an excellent way to gather immense amounts of data, it jeopardizes the privacy of participants, as it requires to collect labeled data at a centralized server. Recently, federated learning has emerged as a practical concept in solving the privacy preservation issue of crowdsourcing participants by performing model training at the edge devices in a decentralized manner; the participants do not expose their data anymore to a centralized server. This paper presents a novel method utilizing federated learning to improve the accuracy of RSS fingerprint-based localization while preserving the privacy of the crowdsourcing participants. Employing federated learning allows ensuring \emph{preserving the privacy of user data} while enabling an adequate localization performance with experimental data captured in real-world settings. The proposed method improved localization accuracy by 1.8 meters when used as a booster for centralized learning and achieved satisfactory localization accuracy when used standalone.
A fine-grained policy model for Provenance-based Access Control and Policy Algebras.pdf
Venue: arXiv cs.CR 2020
Year: 2020

A fine-grained provenance-based access control policy model is proposed in this paper, in order to improve the express performance of existing model. This method employs provenance as conditions to determine whether a piece of data can be accessed because historical operations performed on data could reveal clues about its sensitivity and vulnerability. Particularly, our proposed work provides a four-valued decision set which allows showing status to match a restriction particularly. This framework consists of target policy, access control policy, and policy algebras. With the complete definition and algebra system construction, a practical fine-grained access control policy model is developed.
Provenance-based Classification Policy based on Encrypted Search
Venue: arXiv cs.CR 2020
Year: 2020

As an important type of cloud data, digital provenance is arousing increasing attention on improving system performance. Currently, provenance has been employed to provide cues regarding access control and to estimate data quality. However, provenance itself might also be sensitive information. Therefore, provenance might be encrypted and stored in the Cloud. In this paper, we provide a mechanism to classify cloud documents by searching specific keywords from their encrypted provenance, and we prove our scheme achieves semantic security. In term of application of the proposed techniques, considering that files are classified to store separately in the cloud, in order to facilitate the regulation and security protection for the files, the classification policies can use provenance as conditions to determine the category of a document. Such as the easiest sample policy goes like: the documents have been reviewed twice can be classified as "public accessible", which can be accessed by the public.