OpenRaiser/Intern-Atlas huggingface dataset:
Recognition of Smoking Gesture Using Smart Watch Technology
Venue: arXiv cs.LG 2020
Year: 2020

Diseases resulting from prolonged smoking are the most common preventable causes of death in the world today. In this report we investigate the success of utilizing accelerometer sensors in smart watches to identify smoking gestures. Early identification of smoking gestures can help to initiate the appropriate intervention method and prevent relapses in smoking. Our experiments indicate 85%-95% success rates in identification of smoking gesture among other similar gestures using Artificial Neural Networks (ANNs). Our investigations concluded that information obtained from the x-dimension of accelerometers is the best means of identifying the smoking gesture, while y and z dimensions are helpful in eliminating other gestures such as: eating, drinking, and scratch of nose. We utilized sensor data from the Apple Watch during the training of the ANN. Using sensor data from another participant collected on Pebble Steel, we obtained a smoking identification accuracy of greater than 90% when using an ANN trained on data previously collected from the Apple Watch. Finally, we have demonstrated the possibility of using smart watches to perform continuous monitoring of daily activities.
Claim Check-Worthiness Detection as Positive Unlabelled Learning
Venue: arXiv cs.CL 2020
Year: 2020

As the first step of automatic fact checking, claim check-worthiness detection is a critical component of fact checking systems. There are multiple lines of research which study this problem: check-worthiness ranking from political speeches and debates, rumour detection on Twitter, and citation needed detection from Wikipedia. To date, there has been no structured comparison of these various tasks to understand their relatedness, and no investigation into whether or not a unified approach to all of them is achievable. In this work, we illuminate a central challenge in claim check-worthiness detection underlying all of these tasks, being that they hinge upon detecting both how factual a sentence is, as well as how likely a sentence is to be believed without verification. As such, annotators only mark those instances they judge to be clear-cut check-worthy. Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning that finds instances which were incorrectly labelled as not check-worthy. In applying this, we out-perform the state of the art in two of the three tasks studied for claim check-worthiness detection in English.
BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Venue: arXiv cs.LG 2020
Year: 2020

Measuring the quality of a generated sequence against a set of references is a central problem in many learning frameworks, be it to compute a score, to assign a reward, or to perform discrimination. Despite great advances in model architectures, metrics that scale independently of the number of references are still based on n-gram estimates. We show that the underlying operations, counting words and comparing counts, can be lifted to embedding words and comparing embeddings. An in-depth analysis of BERT embeddings shows empirically that contextual embeddings can be employed to capture the required dependencies while maintaining the necessary scalability through appropriate pruning and smoothing techniques. We cast unconditional generation as a reinforcement learning problem and show that our reward function indeed provides a more effective learning signal than n-gram reward in this challenging setting.
Balance Between Efficient and Effective Learning: Dense2Sparse Reward Shaping for Robot Manipulation with Environment Uncertainty
Venue: arXiv cs.LG 2020
Year: 2020

Efficient and effective learning is one of the ultimate goals of the deep reinforcement learning (DRL), although the compromise has been made in most of the time, especially for the application of robot manipulations. Learning is always expensive for robot manipulation tasks and the learning effectiveness could be affected by the system uncertainty. In order to solve above challenges, in this study, we proposed a simple but powerful reward shaping method, namely Dense2Sparse. It combines the advantage of fast convergence of dense reward and the noise isolation of the sparse reward, to achieve a balance between learning efficiency and effectiveness, which makes it suitable for robot manipulation tasks. We evaluated our Dense2Sparse method with a series of ablation experiments using the state representation model with system uncertainty. The experiment results show that the Dense2Sparse method obtained higher expected reward compared with the ones using standalone dense reward or sparse reward, and it also has a superior tolerance of system uncertainty.
Path Planning Using Probability Tensor Flows
Venue: arXiv cs.LG 2020
Year: 2020

Probability models have been proposed in the literature to account for "intelligent" behavior in many contexts. In this paper, probability propagation is applied to model agent's motion in potentially complex scenarios that include goals and obstacles. The backward flow provides precious background information to the agent's behavior, viz., inferences coming from the future determine the agent's actions. Probability tensors are layered in time in both directions in a manner similar to convolutional neural networks. The discussion is carried out with reference to a set of simulated grids where, despite the apparent task complexity, a solution, if feasible, is always found. The original model proposed by Attias has been extended to include non-absorbing obstacles, multiple goals and multiple agents. The emerging behaviors are very realistic and demonstrate great potentials of the application of this framework to real environments.
State-of-the-Art Augmented NLP Transformer models for direct and single-step retrosynthesis
Venue: arXiv cs.LG 2020
Year: 2020

We investigated the effect of different training scenarios on predicting the (retro)synthesis of chemical compounds using a text-like representation of chemical reactions (SMILES) and Natural Language Processing neural network Transformer architecture. We showed that data augmentation, which is a powerful method used in image processing, eliminated the effect of data memorization by neural networks, and improved their performance for the prediction of new sequences. This effect was observed when augmentation was used simultaneously for input and the target data simultaneously. The top-5 accuracy was 84.8% for the prediction of the largest fragment (thus identifying principal transformation for classical retro-synthesis) for the USPTO-50k test dataset and was achieved by a combination of SMILES augmentation and a beam search algorithm. The same approach provided significantly better results for the prediction of direct reactions from the single-step USPTO-MIT test set. Our model achieved 90.6% top-1 and 96.1% top-5 accuracy for its challenging mixed set and 97% top-5 accuracy for the USPTO-MIT separated set. It also significantly improved results for USPTO-full set single-step retrosynthesis for both top-1 and top-10 accuracies. The appearance frequency of the most abundantly generated SMILES was well correlated with the prediction outcome and can be used as a measure of the quality of reaction prediction.
Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)
Venue: arXiv cs.CV 2020
Year: 2020

Hyperspectral images provide detailed spectral information through hundreds of (narrow) spectral channels (also known as dimensionality or bands) with continuous spectral information that can accurately classify diverse materials of interest. The increased dimensionality of such data makes it possible to significantly improve data information content but provides a challenge to the conventional techniques (the so-called curse of dimensionality) for accurate analysis of hyperspectral images. Feature extraction, as a vibrant field of research in the hyperspectral community, evolved through decades of research to address this issue and extract informative features suitable for data representation and classification. The advances in feature extraction have been inspired by two fields of research, including the popularization of image and signal processing as well as machine (deep) learning, leading to two types of feature extraction approaches named shallow and deep techniques. This article outlines the advances in feature extraction approaches for hyperspectral imagery by providing a technical overview of the state-of-the-art techniques, providing useful entry points for researchers at different levels, including students, researchers, and senior researchers, willing to explore novel investigations on this challenging topic. In more detail, this paper provides a bird's eye view over shallow (both supervised and unsupervised) and deep feature extraction approaches specifically dedicated to the topic of hyperspectral feature extraction and its application on hyperspectral image classification. Additionally, this paper compares 15 advanced techniques with an emphasis on their methodological foundations in terms of classification accuracies. Furthermore, the codes and libraries are shared at https://github.com/BehnoodRasti/HyFTech-Hyperspectral-Shallow-Deep-Feature-Extraction-Toolbox.
K set-agreement bounds in round-based models through combinatorial topology
Venue: arXiv cs.DC 2020
Year: 2020

Round-based models are very common message-passing models; combinatorial topology applied to distributed computing provides sweeping results like general lower bounds. We combine both to study the computability of k-set agreement. Among all the possible round-based models, we consider oblivious ones, where the constraints are given only round per round by a set of allowed graphs. And among oblivious models, we focus on closed-above ones, that is models where the set of possible graphs contains all graphs with more edges than some starting graphs. These capture intuitively the underlying structure required by some communication model, like containing a ring. We then derive lower bounds and upper bounds in one round for k-set agreement, such that these bounds are proved using combinatorial topology but stated only in terms of graph properties. These bounds extend to multiple rounds when limiting our algorithms to be oblivious -- recalling only pairs of processes and initial value, not who send what and when.
Generalized Policy Elimination: an efficient algorithm for Nonparametric Contextual Bandits
Venue: arXiv cs.LG 2020
Year: 2020

We propose the Generalized Policy Elimination (GPE) algorithm, an oracle-efficient contextual bandit (CB) algorithm inspired by the Policy Elimination algorithm of \cite{dudik2011}. We prove the first regret optimality guarantee theorem for an oracle-efficient CB algorithm competing against a nonparametric class with infinite VC-dimension. Specifically, we show that GPE is regret-optimal (up to logarithmic factors) for policy classes with integrable entropy. For classes with larger entropy, we show that the core techniques used to analyze GPE can be used to design an $\varepsilon$-greedy algorithm with regret bound matching that of the best algorithms to date. We illustrate the applicability of our algorithms and theorems with examples of large nonparametric policy classes, for which the relevant optimization oracles can be efficiently implemented.
Optimizing JPEG Quantization for Classification Networks
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning for computer vision depends on lossy image compression: it reduces the storage required for training and test data and lowers transfer costs in deployment. Mainstream datasets and imaging pipelines all rely on standard JPEG compression. In JPEG, the degree of quantization of frequency coefficients controls the lossiness: an 8 by 8 quantization table (Q-table) decides both the quality of the encoded image and the compression ratio. While a long history of work has sought better Q-tables, existing work either seeks to minimize image distortion or to optimize for models of the human visual system. This work asks whether JPEG Q-tables exist that are "better" for specific vision networks and can offer better quality--size trade-offs than ones designed for human perception or minimal distortion. We reconstruct an ImageNet test set with higher resolution to explore the effect of JPEG compression under novel Q-tables. We attempt several approaches to tune a Q-table for a vision task. We find that a simple sorted random sampling method can exceed the performance of the standard JPEG Q-table. We also use hyper-parameter tuning techniques including bounded random search, Bayesian optimization, and composite heuristic optimization methods. The new Q-tables we obtained can improve the compression rate by 10% to 200% when the accuracy is fixed, or improve accuracy up to $2\%$ at the same compression rate.
Distill, Adapt, Distill: Training Small, In-Domain Models for Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

We explore best practices for training small, memory efficient machine translation models with sequence-level knowledge distillation in the domain adaptation setting. While both domain adaptation and knowledge distillation are widely-used, their interaction remains little understood. Our large-scale empirical results in machine translation (on three language pairs with three domains each) suggest distilling twice for best performance: once using general-domain data and again using in-domain data with an adapted teacher.
SERENIoT: Collaborative Network Security Policy Management and Enforcement for Smart Homes
Venue: arXiv cs.CR 2020
Year: 2020

Network traffic whitelisting has emerged as a dominant approach for securing consumer IoT devices. However, determining what the whitelisted behavior of an IoT device should be remains an open challenge. Proposals to date have relied on manufacturers and trusted parties to provide whitelists, but these proposals require manufacturer involvement or placing trust in an additional stakeholder. Alternatively, locally monitoring devices can allow building whitelists of observed behavior, but devices may not exhaust their functionality set during the observation period, or the behavior may change following a software update which requires re-training. This paper proposes a blockchain-based system for determining whether an IoT device is behaving like other devices of the same type. Our system (SERENIoT, pronounced Serenity) overcomes the challenge of initially determining the correct behavior for a device. Nodes in the SERENIoT public blockchain submit summaries of the network behavior observed for connected IoT devices and build whitelists of behavior observed by the majority of nodes. Changes in behavior through software updates are automatically whitelisted once the update is broadly deployed. Through a proof-of-concept implementation of SERENIoT on a small Raspberry Pi IoT network and a large-scale Amazon EC2 simulation, we evaluate the security, scalability, and performance of our system.
Expected participation and mentality of smart citizen in smart cities
Venue: arXiv cs.CY 2020
Year: 2020

The purpose is to investigate the expected participation and mentality of smart citizens in smart cities. The key question is the role of the human factor in smart environments globally studied through a research corpus of 150 documents including mainstream summaries trend reports, white papers and visions of business, governmental and university research cooperations reaching a wide audience of the subject. Foremost, a short review of the changing scholarly trends is presented as a theoretical framework. Concerning its key ideas, the corpus based findings are recapped and analysed by content networks and the most referred city strategies. Besides, a critical approach reveal further required factors and risks to investigate. The ultimate goal is to understand how the smart city landscape is shaped by citizen-based strategies, open data, empowerment and responsibility. Accordingly, the paper closes with further considerations regarding the importance of anonymous open data, advantages of neighbourhood-based implementations, aspects of permanent and temporary citizen-engagements, interpretation of metaphors or upcoming technologies, and also, privacy and ethical issues. The results provide the policy development and the emerging scholarly interest with a framework study.
EmpTransfo: A Multi-head Transformer Architecture for Creating Empathetic Dialog Systems
Venue: arXiv cs.CL 2020
Year: 2020

Understanding emotions and responding accordingly is one of the biggest challenges of dialog systems. This paper presents EmpTransfo, a multi-head Transformer architecture for creating an empathetic dialog system. EmpTransfo utilizes state-of-the-art pre-trained models (e.g., OpenAI-GPT) for language generation, though models with different sizes can be used. We show that utilizing the history of emotions and other metadata can improve the quality of generated conversations by the dialog system. Our experimental results using a challenging language corpus show that the proposed approach outperforms other models in terms of Hit@1 and PPL (Perplexity).
Designing for Employee Voice
Venue: arXiv cs.HC 2020
Year: 2020

Employee voice and workplace democracy have a positive impact on employee wellbeing and the performance of organizations. In this paper, we conducted interviews with employees to identify facilitators and inhibitors for the voice within the workplace and a corresponding set of appropriate qualities: Civility, Validity, Safety and Egalitarianism. We then operationalised these qualities as a set of design goals - Assured Anonymity, Constructive Moderation, Adequate Slowness and Controlled Access - in the design and development of a secure anonymous employee voice system. Our novel take on the Enterprise Social Network aims to foster good citizenship whilst also promoting frank yet constructive discussion. We reflect on a two-week deployment of our system, the diverse range of candid discussions that emerged around important workplace issues and the potential for change within the host organization. We conclude by reflecting on the ways in which our approach shaped the discourse and supported the creation of a trusted environment for employee voice.
What is the State of Neural Network Pruning?
Venue: arXiv cs.LG 2020
Year: 2020

Neural network pruning---the task of reducing the size of a network by removing parameters---has been the subject of a great deal of work in recent years. We provide a meta-analysis of the literature, including an overview of approaches to pruning and consistent findings in the literature. After aggregating results across 81 papers and pruning hundreds of models in controlled conditions, our clearest finding is that the community suffers from a lack of standardized benchmarks and metrics. This deficiency is substantial enough that it is hard to compare pruning techniques to one another or determine how much progress the field has made over the past three decades. To address this situation, we identify issues with current practices, suggest concrete remedies, and introduce ShrinkBench, an open-source framework to facilitate standardized evaluations of pruning methods. We use ShrinkBench to compare various pruning techniques and show that its comprehensive evaluation can prevent common pitfalls when comparing pruning methods.
Robotic Assembly across Multiple Contact Stiffnesses with Robust Force Controllers
Venue: arXiv cs.RO 2020
Year: 2020

Active Force Control (AFC) is an important scheme for tackling high-precision robotic assembly. Classical force controllers are highly surface-dependent: the controller must be carefully tuned for each type of surface in contact, in order to avoid instabilities and to achieve a reasonable performance level. Here, we build upon the recently-developed Convex Controller Synthesis (CCS) to enable high-precision assembly across a wide range of surface stiffnesses without any surface-dependent tuning. Specifically, we demonstrate peg-in-hole assembly with 100 micron clearance, initial position uncertainties up to 2 cm, and for four types of peg and hole materials -- rubber, plastic, wood, aluminum -- whose stiffnesses range from 10 to 100 N/mm, using a single controller.
Joint Power Control and Passive Beamforming in IRS-Assisted Spectrum Sharing
Venue: arXiv cs.IT 2020
Year: 2020

In cognitive radio (CR) communication systems, achieving high secondary user (SU) rate in the presence of strong cross-link interference with the primary user (PU) is challenging. In this letter, we exploit the emerging intelligent reflecting surface (IRS) technology to tackle this problem. Specifically, we investigate an IRS-assisted CR communication system where an IRS is deployed to assist in the spectrum sharing between a PU link and an SU link. We aim to maximize the achievable SU rate subject to a given signal-to-interference-plus-noise ratio target for the PU link, by jointly optimizing the SU transmit power and IRS reflect beamforming. Since the formulated problem is difficult to solve due to its non-convexity and coupled variables, we propose an efficient algorithm based on alternating optimization and successive convex approximation techniques to solve it sub-optimally, along with some heuristic designs for lower complexity. Simulation results show that IRS is able to significantly improve the SU rate, even for the scenarios deemed most challenging in conventional CR systems without using IRS.
Knowledge graph based methods for record linkage
Venue: arXiv cs.AI 2020
Year: 2020

Nowadays, it is common in Historical Demography the use of individual-level data as a consequence of a predominant life-course approach for the understanding of the demographic behaviour, family transition, mobility, etc. Record linkage advance is key in these disciplines since it allows to increase the volume and the data complexity to be analyzed. However, current methods are constrained to link data coming from the same kind of sources. Knowledge graph are flexible semantic representations, which allow to encode data variability and semantic relations in a structured manner. In this paper we propose the knowledge graph use to tackle record linkage task. The proposed method, named {\bf WERL}, takes advantage of the main knowledge graph properties and learns embedding vectors to encode census information. These embeddings are properly weighted to maximize the record linkage performance. We have evaluated this method on benchmark data sets and we have compared it to related methods with stimulating and satisfactory results.