OpenRaiser/Intern-Atlas huggingface dataset:
Real-time Data Driven Precision Estimator for RAVEN-II Surgical Robot End Effector Position
Venue: arXiv cs.RO 2019
Year: 2019

Surgical robots have been introduced to operating rooms over the past few decades due to their high sensitivity, small size, and remote controllability. The cable-driven nature of many surgical robots allows the systems to be dexterous and lightweight, with diameters as low as 5mm. However, due to the slack and stretch of the cables and the backlash of the gears, inevitable uncertainties are brought into the kinematics calculation. Since the reported end effector position of surgical robots like RAVEN-II is directly calculated using the motor encoder measurements and forward kinematics, it may contain relatively large error up to 10mm, whereas semi-autonomous functions being introduced into abdominal surgeries require position inaccuracy of at most 1mm. To resolve the problem, a cost-effective, real-time and data-driven pipeline for robot end effector position precision estimation is proposed and tested on RAVEN-II. Analysis shows an improved end effector position error of around 1mm RMS traversing through the entire robot workspace without high-resolution motion tracker.
Building Damage Detection in Satellite Imagery Using Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

In all types of disasters, from earthquakes to armed conflicts, aid workers need accurate and timely data such as damage to buildings and population displacement to mount an effective response. Remote sensing provides this data at an unprecedented scale, but extracting operationalizable information from satellite images is slow and labor-intensive. In this work, we use machine learning to automate the detection of building damage in satellite imagery. We compare the performance of four different convolutional neural network models in detecting damaged buildings in the 2010 Haiti earthquake. We also quantify how well the models will generalize to future disasters by training and testing models on different disaster events.
Continuous Phase Modulation With Faster-than-Nyquist Signaling for Channels With 1-bit Quantization and Oversampling at the Receiver
Venue: arXiv cs.IT 2019
Year: 2019

Continuous phase modulation (CPM) with 1-bit quantization at the receiver is promising in terms of energy and spectral efficiency. In this study, CPM waveforms with symbol durations significantly shorter than the inverse of the signal bandwidth are proposed, termed faster-than-Nyquist CPM. This configuration provides a better steering of zero-crossings as compared to conventional CPM. Numerical results confirm a superior performance in terms of BER in comparison with state-of-the-art methods, while having the same spectral efficiency and a lower oversampling factor. Moreover, the new waveform can be detected with low-complexity, which yields almost the same performance as using the BCJR algorithm.
Hierarchical Semantic Correspondence Learning for Post-Discharge Patient Mortality Prediction
Venue: arXiv cs.CL 2019
Year: 2019

Predicting patient mortality is an important and challenging problem in the healthcare domain, especially for intensive care unit (ICU) patients. Electronic health notes serve as a rich source for learning patient representations, that can facilitate effective risk assessment. However, a large portion of clinical notes are unstructured and also contain domain specific terminologies, from which we need to extract structured information. In this paper, we introduce an embedding framework to learn semantically-plausible distributed representations of clinical notes that exploits the semantic correspondence between the unstructured texts and their corresponding structured knowledge, known as semantic frame, in a hierarchical fashion. Our approach integrates text modeling and semantic correspondence learning into a single model that comprises 1) an unstructured embedding module that makes use of self-similarity matrix representations in order to inject structural regularities of different segments inherent in clinical texts to promote local coherence, 2) a structured embedding module to embed the semantic frames (e.g., UMLS semantic types) with deep ConvNet and 3) a hierarchical semantic correspondence module that embeds by enhancing the interactions between text-semantic frame embedding pairs at multiple levels (i.e., words, sentence, note). Evaluations on multiple embedding benchmarks on post discharge intensive care patient mortality prediction tasks demonstrate its effectiveness compared to approaches that do not exploit the semantic interactions between structured and unstructured information present in clinical notes.
Adaptive Step Sizes in Variance Reduction via Regularization
Venue: arXiv cs.LG 2019
Year: 2019

The main goal of this work is equipping convex and nonconvex problems with Barzilai-Borwein (BB) step size. With the adaptivity of BB step sizes granted, they can fail when the objective function is not strongly convex. To overcome this challenge, the key idea here is to bridge (non)convex problems and strongly convex ones via regularization. The proposed regularization schemes are \textit{simple} yet effective. Wedding the BB step size with a variance reduction method, known as SARAH, offers a free lunch compared with vanilla SARAH in convex problems. The convergence of BB step sizes in nonconvex problems is also established and its complexity is no worse than other adaptive step sizes such as AdaGrad. As a byproduct, our regularized SARAH methods for convex functions ensure that the complexity to find $\mathbb{E}[\| \nabla f(\mathbf{x}) \|^2]\leq ε$ is ${\cal O}\big( (n+\frac{1}{\sqrtε})\ln{\frac{1}ε}\big)$, improving $ε$ dependence over existing results. Numerical tests further validate the merits of proposed approaches.
RiWalk: Fast Structural Node Embedding via Role Identification
Venue: arXiv cs.SI 2019
Year: 2019

Nodes performing different functions in a network have different roles, and these roles can be gleaned from the structure of the network. Learning latent representations for the roles of nodes helps to understand the network and to transfer knowledge across networks. However, most existing structural embedding approaches suffer from high computation and space cost or rely on heuristic feature engineering. Here we propose RiWalk, a flexible paradigm for learning structural node representations. It decouples the structural embedding problem into a role identification procedure and a network embedding procedure. Through role identification, rooted kernels with structural dependencies kept are built to better integrate network embedding methods. To demonstrate the effectiveness of RiWalk, we develop two different role identification methods named RiWalk-SP and RiWalk-WL respectively and employ random walk based network embedding methods. Experiments on within-network classification tasks show that our proposed algorithms achieve comparable performance with other baselines while being an order of magnitude more efficient. Besides, we also conduct across-network role classification tasks. The results show potential of structural embeddings in transfer learning. RiWalk is also scalable, making it capable of capturing structural roles in massive networks.
Probability Logic
Venue: arXiv cs.AI 2019
Year: 2019

This chapter presents probability logic as a rationality framework for human reasoning under uncertainty. Selected formal-normative aspects of probability logic are discussed in the light of experimental evidence. Specifically, probability logic is characterized as a generalization of bivalent truth-functional propositional logic (short "logic"), as being connexive, and as being nonmonotonic. The chapter discusses selected argument forms and associated uncertainty propagation rules. Throughout the chapter, the descriptive validity of probability logic is compared to logic, which was used as the gold standard of reference for assessing the rationality of human reasoning in the 20th century.
Seeing and Hearing Egocentric Actions: How Much Can We Learn?
Venue: arXiv cs.CV 2019
Year: 2019

Our interaction with the world is an inherently multimodal experience. However, the understanding of human-to-object interactions has historically been addressed focusing on a single modality. In particular, a limited number of works have considered to integrate the visual and audio modalities for this purpose. In this work, we propose a multimodal approach for egocentric action recognition in a kitchen environment that relies on audio and visual information. Our model combines a sparse temporal sampling strategy with a late fusion of audio, spatial, and temporal streams. Experimental results on the EPIC-Kitchens dataset show that multimodal integration leads to better performance than unimodal approaches. In particular, we achieved a 5.18% improvement over the state of the art on verb classification.
Knowledge-based Biomedical Data Science 2019
Venue: arXiv cs.AI 2019
Year: 2019

Knowledge-based biomedical data science (KBDS) involves the design and implementation of computer systems that act as if they knew about biomedicine. Such systems depend on formally represented knowledge in computer systems, often in the form of knowledge graphs. Here we survey the progress in the last year in systems that use formally represented knowledge to address data science problems in both clinical and biological domains, as well as on approaches for creating knowledge graphs. Major themes include the relationships between knowledge graphs and machine learning, the use of natural language processing, and the expansion of knowledge-based approaches to novel domains, such as Chinese Traditional Medicine and biodiversity.
Byzantine-Tolerant Register in a System with Continuous Churn
Venue: arXiv cs.DC 2019
Year: 2019

A shared read/write register emulation provides the illusion of shared-memory on top of message-passing models. The main hurdle with such emulations is dealing with server faults in the system. Several crash-tolerant register emulations in static systems require algorithms to replicate the value of the shared register onto a majority of servers. Majority correctness is necessary for such emulations. Byzantine faults are considered to be the worst kind of faults that can happen in any distributed system. Emulating a Byzantine-tolerant register requires replicating the register value on to more than two-thirds of the servers. Emulating a register in a dynamic system where servers and clients can enter and leave the system and be faulty is harder than in static systems. There are several crash-tolerant register emulations for dynamic systems. This paper presents the first emulation of a multi-reader multi-writer atomic register in a system that can withstand nodes continually entering and leaving, imposes no upper bound on the system size and can tolerate Byzantine servers. The algorithm works as long as the number of servers entering and leaving during a fixed time interval is at most a constant fraction of the system size at the beginning of the interval, and as long as the number of Byzantine servers in the system is at most f. Although our algorithm requires that there be a constant known upper bound on the number of Byzantine servers, this restriction is unavoidable, as we show that it is impossible to emulate an atomic register if the system size and maximum number of servers that can be Byzantine in the system is unknown to the nodes.
Self Driving RC Car using Behavioral Cloning
Venue: arXiv cs.RO 2019
Year: 2019

Self Driving Car technology is a vehicle that guides itself without human conduction. The first truly autonomous cars appeared in the 1980s with projects funded by DARPA( Defense Advance Research Project Agency ). Since then a lot has changed with the improvements in the fields of Computer Vision and Machine Learning. We have used the concept of behavioral cloning to convert a normal RC model car into an autonomous car using Deep Learning technology
On the Importance of Word Boundaries in Character-level Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Neural Machine Translation (NMT) models generally perform translation using a fixed-size lexical vocabulary, which is an important bottleneck on their generalization capability and overall translation quality. The standard approach to overcome this limitation is to segment words into subword units, typically using some external tools with arbitrary heuristics, resulting in vocabulary units not optimized for the translation task. Recent studies have shown that the same approach can be extended to perform NMT directly at the level of characters, which can deliver translation accuracy on-par with subword-based models, on the other hand, this requires relatively deeper networks. In this paper, we propose a more computationally-efficient solution for character-level NMT which implements a hierarchical decoding architecture where translations are subsequently generated at the level of words and characters. We evaluate different methods for open-vocabulary NMT in the machine translation task from English into five languages with distinct morphological typology, and show that the hierarchical decoding model can reach higher translation accuracy than the subword-level NMT model using significantly fewer parameters, while demonstrating better capacity in learning longer-distance contextual and grammatical dependencies than the standard character-level NMT model.
Stabilizing Transformers for Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown breakthrough success in natural language processing (NLP), achieving state-of-the-art results in domains such as language modeling and machine translation. Harnessing the transformer's ability to process long time horizons of information could provide a similar performance boost in partially observable reinforcement learning (RL) domains, but the large-scale transformers used in NLP have yet to be successfully applied to the RL setting. In this work we demonstrate that the standard transformer architecture is difficult to optimize, which was previously observed in the supervised learning setting but becomes especially pronounced with RL objectives. We propose architectural modifications that substantially improve the stability and learning speed of the original Transformer and XL variant. The proposed architecture, the Gated Transformer-XL (GTrXL), surpasses LSTMs on challenging memory environments and achieves state-of-the-art results on the multi-task DMLab-30 benchmark suite, exceeding the performance of an external memory architecture. We show that the GTrXL, trained using the same losses, has stability and performance that consistently matches or exceeds a competitive LSTM baseline, including on more reactive tasks where memory is less critical. GTrXL offers an easy-to-train, simple-to-implement but substantially more expressive architectural alternative to the standard multi-layer LSTM ubiquitously used for RL agents in partially observable environments.
Computational Psychology to Embed Emotions into News or Advertisements to Increase Reader Affinity
Venue: arXiv cs.IR 2019
Year: 2019

Readers take decisions about going through the complete news based on many factors. The emotional impact of the news title on reader is one of the most important factors. Cognitive ergonomics tries to strike the balance between work, product and environment with human needs and capabilities. The utmost need to integrate emotions in the news as well as advertisements cannot be denied. The idea is that news or advertisement should be able to engage the reader on emotional and behavioral platform. While achieving this objective there is need to learn about reader behavior and use computational psychology while presenting as well as writing news or advertisements. This paper based on Machine Learning, tries to map behavior of the reader with the news/advertisements and also provide inputs for affective value for building personalized news or advertisements presentations. The affective value of the news is determined and news artifacts are mapped to reader. The algorithm suggests the most suitable news for readers while understanding emotional traits required for personalization. This work can be used to improve reader satisfaction through embedding emotions in the reading material and prioritizing news presentations. It can be used to map personal reading material range, personalized programs and ranking programs, advertisements with reference to individuals.
Machine Learning for Paper Grammage Prediction Based on Sensor Measurements in Paper Mills
Venue: arXiv cs.LG 2019
Year: 2019

Automation is at the core of modern industry. It aims to increase production rates, decrease production costs, and reduce human intervention in order to avoid human mistakes and time delays during manufacturing. On the other hand, human assistance is usually required to customize products and reconfigure control systems through a special process interface called Human Machine Interface (HMI). Machine Learning (ML) algorithms can effectively be used to resolve this tradeoff between full automation and human assistance.This paper provides an example of the industrial application of ML algorithms to help human operators save their mental effort and avoid time delays and unintended mistakes for the sake of high production rates. Based on real-time sensor measurements, several ML algorithms have been tried to classify paper rolls according to paper grammage in a white paper mill. The performance evaluation shows that the AdaBoost algorithm is the best ML algorithm for this application with classification accuracy (CA), precision, and recall of 97.1%. The generalization of the proposed approach for achieving cost-effective mills construction will be the subject of our future research.
DP-MAC: The Differentially Private Method of Auxiliary Coordinates for Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

Developing a differentially private deep learning algorithm is challenging, due to the difficulty in analyzing the sensitivity of objective functions that are typically used to train deep neural networks. Many existing methods resort to the stochastic gradient descent algorithm and apply a pre-defined sensitivity to the gradients for privatizing weights. However, their slow convergence typically yields a high cumulative privacy loss. Here, we take a different route by employing the method of auxiliary coordinates, which allows us to independently update the weights per layer by optimizing a per-layer objective function. This objective function can be well approximated by a low-order Taylor's expansion, in which sensitivity analysis becomes tractable. We perturb the coefficients of the expansion for privacy, which we optimize using more advanced optimization routines than SGD for faster convergence. We empirically show that our algorithm provides a decent trained model quality under a modest privacy budget.
Alleviating Bottlenecks for DNN Execution on GPUs via Opportunistic Computing
Venue: arXiv cs.DC 2019
Year: 2019

Edge computing and IoT applications are severely constrained by limited hardware resource. This makes memory consuming DNN frameworks not applicable to edge computing. Simple algorithms such as direct convolution are finding their way in embedded machine learning. As one of the most widely used platforms for DNN acceleration, GPUs face the bottleneck of on-chip bandwidth. This work introduces a GPU DNN execution architecture that targets on relieving the on-chip bandwidth bottleneck by reducing data movement through opportunistic computing. We first investigate data access patterns in the hardware view rather than the software view. Then we propose two opportunistic computing techniques to predictably perform computation when data is available with the help of assistant warps. By moving computation to data, our techniques are able to significantly reduce data movement and relieve the DNN execution bottleneck. Our evaluation results show that the proposed technique can improve DNN application performance as much as 55%.
Health Monitoring in Smart Homes Utilizing Internet of Things
Venue: arXiv cs.HC 2019
Year: 2019

In recent years the concept of the Internet of Things (IoT) has evolved to connect commercial gadgets together with the medical field to facilitate an unprecedented range of accessibility. The development of medical devices connected to internet of things has been praised for the potential of alleviating the strain on the modern healthcare system by giving users the opportunity to reside in the home during treatment or recovery. With the IoT becoming more prevalent and available at a commercial level, there exists room for integration into emerging, intelligent environments such as smart homes. When used in tandem with conventional healthcare, the IoT offers a vast range of custom-tailored treatment options. This paper studies recent state-of-the-art research on the field of IoT for health monitoring and smart homes, examines several potential use-cases of blending the technology, and proposes integration with an existing smart home testbed for further study. Challenges of adoption and future research on the topic are also discussed.
Solving Rubik's Cube with a Robot Hand
Venue: arXiv cs.LG 2019
Year: 2019

We demonstrate that models trained only in simulation can be used to solve a manipulation problem of unprecedented complexity on a real robot. This is made possible by two key components: a novel algorithm, which we call automatic domain randomization (ADR) and a robot platform built for machine learning. ADR automatically generates a distribution over randomized environments of ever-increasing difficulty. Control policies and vision state estimators trained with ADR exhibit vastly improved sim2real transfer. For control policies, memory-augmented models trained on an ADR-generated distribution of environments show clear signs of emergent meta-learning at test time. The combination of ADR with our custom robot platform allows us to solve a Rubik's cube with a humanoid robot hand, which involves both control and state estimation problems. Videos summarizing our results are available: https://openai.com/blog/solving-rubiks-cube/
Covariance Matrix Estimation from Correlated Sub-Gaussian Samples
Venue: arXiv cs.IT 2019
Year: 2019

This paper studies the problem of estimating a covariance matrix from correlated sub-Gaussian samples. We consider using the correlated sample covariance matrix estimator to approximate the true covariance matrix. We establish non-asymptotic error bounds for this estimator in both real and complex cases. Our theoretical results show that the error bounds are determined by the signal dimension $n$, the sample size $m$ and the correlation pattern $\textbf{B}$. In particular, when the correlation pattern $\textbf{B}$ satisfies $tr(\textbf{B})=m$, $||\textbf{B}||_{F}=O(m^{1/2})$, and $||\textbf{B}||=O(1)$, these results reveal that $O(n)$ samples are sufficient to accurately estimate the covariance matrix from correlated sub-Gaussian samples. Numerical simulations are presented to show the correctness of the theoretical results.
Electro-Magnetic Side-Channel Attack Through Learned Denoising and Classification
Venue: arXiv cs.CR 2019
Year: 2019

This paper proposes an upgraded electro-magnetic side-channel attack that automatically reconstructs the intercepted data. A novel system is introduced, running in parallel with leakage signal interception and catching compromising data in real-time. Based on deep learning and character recognition the proposed system retrieves more than 57% of characters present in intercepted signals regardless of signal type: analog or digital. The approach is also extended to a protection system that triggers an alarm if the system is compromised, demonstrating a success rate over 95%. Based on software-defined radio and graphics processing unit architectures, this solution can be easily deployed onto existing information systems where information shall be kept secret.