OpenRaiser/Intern-Atlas huggingface dataset:
Closed ASL Interpreting for Online Videos
Venue: arXiv cs.HC 2019
Year: 2019

Deaf individuals face great challenges in today's society. It can be very difficult to be able to understand different forms of media without a sense of hearing. Many videos and movies found online today are not captioned, and even fewer have a supporting video with an interpreter. Also, even with a supporting interpreter video provided, information is still lost due to the inability to look at both the video and the interpreter simultaneously. To alleviate this issue, we came up with a tool called closed interpreting. Similar to closed captioning, it will be displayed with an online video and can be toggled on and off. However, the closed interpreter is also user-adjustable. Settings, such as interpreter size, transparency, and location, can be adjusted. Our goal with this study is to find out what deaf and hard of hearing viewers like about videos that come with interpreters, and whether the adjustability is beneficial.
TFCheck : A TensorFlow Library for Detecting Training Issues in Neural Network Programs
Venue: arXiv cs.LG 2019
Year: 2019

The increasing inclusion of Machine Learning (ML) models in safety critical systems like autonomous cars have led to the development of multiple model-based ML testing techniques. One common denominator of these testing techniques is their assumption that training programs are adequate and bug-free. These techniques only focus on assessing the performance of the constructed model using manually labeled data or automatically generated data. However, their assumptions about the training program are not always true as training programs can contain inconsistencies and bugs. In this paper, we examine training issues in ML programs and propose a catalog of verification routines that can be used to detect the identified issues, automatically. We implemented the routines in a Tensorflow-based library named TFCheck. Using TFCheck, practitioners can detect the aforementioned issues automatically. To assess the effectiveness of TFCheck, we conducted a case study with real-world, mutants, and synthetic training programs. Results show that TFCheck can successfully detect training issues in ML code implementations.
Investigating BERT's Knowledge of Language: Five Analysis Methods with NPIs
Venue: arXiv cs.CL 2019
Year: 2019

Though state-of-the-art sentence representation models can perform tasks requiring significant knowledge of grammar, it is an open question how best to evaluate their grammatical knowledge. We explore five experimental methods inspired by prior work evaluating pretrained sentence representation models. We use a single linguistic phenomenon, negative polarity item (NPI) licensing in English, as a case study for our experiments. NPIs like "any" are grammatical only if they appear in a licensing environment like negation ("Sue doesn't have any cats" vs. "Sue has any cats"). This phenomenon is challenging because of the variety of NPI licensing environments that exist. We introduce an artificially generated dataset that manipulates key features of NPI licensing for the experiments. We find that BERT has significant knowledge of these features, but its success varies widely across different experimental methods. We conclude that a variety of methods is necessary to reveal all relevant aspects of a model's grammatical knowledge in a given domain.
Multi-layer Domain Adaptation for Deep Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

Despite their success in many computer vision tasks, convolutional networks tend to require large amounts of labeled data to achieve generalization. Furthermore, the performance is not guaranteed on a sample from an unseen domain at test time, if the network was not exposed to similar samples from that domain at training time. This hinders the adoption of these techniques in clinical setting where the imaging data is scarce, and where the intra- and inter-domain variance of the data can be substantial. We propose a domain adaptation technique that is especially suitable for deep networks to alleviate this requirement of labeled data. Our method utilizes gradient reversal layers and Squeezeand-Excite modules to stabilize the training in deep networks. The proposed method was applied to publicly available histopathology and chest X-ray databases and achieved superior performance to existing state-of-the-art networks with and without domain adaptation. Depending on the application, our method can improve multi-class classification accuracy by 5-20% compared to DANN introduced in (Ganin, 2014).
Deep Iterative Frame Interpolation for Full-frame Video Stabilization
Venue: arXiv cs.CV 2019
Year: 2019

Video stabilization is a fundamental and important technique for higher quality videos. Prior works have extensively explored video stabilization, but most of them involve cropping of the frame boundaries and introduce moderate levels of distortion. We present a novel deep approach to video stabilization which can generate video frames without cropping and low distortion. The proposed framework utilizes frame interpolation techniques to generate in between frames, leading to reduced inter-frame jitter. Once applied in an iterative fashion, the stabilization effect becomes stronger. A major advantage is that our framework is end-to-end trainable in an unsupervised manner. In addition, our method is able to run in near real-time (15 fps). To the best of our knowledge, this is the first work to propose an unsupervised deep approach to full-frame video stabilization. We show the advantages of our method through quantitative and qualitative evaluations comparing to the state-of-the-art methods.
Real-time Joint Motion Analysis and Instrument Tracking for Robot-Assisted Orthopaedic Surgery
Venue: arXiv cs.RO 2019
Year: 2019

Robotic-assisted orthopaedic surgeries demand accurate, automated leg manipulation for improved spatial accuracy to reduce iatrogenic damage. In this study, we propose novel rigid body designs and an optical tracking volume setup for tracking of the femur, tibia and surgical instruments. Anatomical points inside the leg are measured using Computed Tomography with an accuracy of 0.3mm. Combined with kinematic modelling, we can express these points relative to any frame and across joints to sub-millimetre accuracy. It enables the setup of vectors on the mechanical axes of the femur and tibia for kinematic analysis. Cadaveric experiments are used to verify the tracking of internal anatomies and joint motion analysis. The proposed integrated solution is a first step in the automation of leg manipulation and can be used as a ground-truth for future robot-assisted orthopaedic research.
iFDK: A Scalable Framework for Instant High-resolution Image Reconstruction
Venue: arXiv cs.DC 2019
Year: 2019

Computed Tomography (CT) is a widely used technology that requires compute-intense algorithms for image reconstruction. We propose a novel back-projection algorithm that reduces the projection computation cost to 1/6 of the standard algorithm. We also propose an efficient implementation that takes advantage of the heterogeneity of GPU-accelerated systems by overlapping the filtering and back-projection stages on CPUs and GPUs, respectively. Finally, we propose a distributed framework for high-resolution image reconstruction on state-of-the-art GPU-accelerated supercomputers. The framework relies on an elaborate interleave of MPI collective communication steps to achieve scalable communication. Evaluation on a single Tesla V100 GPU demonstrates that our back-projection kernel performs up to 1.6x faster than the standard FDK implementation. We also demonstrate the scalability and instantaneous CT capability of the distributed framework by using up to 2,048 V100 GPUs to solve 4K and 8K problems within 30 seconds and 2 minutes, respectively (including I/O).
Video Interpolation and Prediction with Unsupervised Landmarks
Venue: arXiv cs.CV 2019
Year: 2019

Prediction and interpolation for long-range video data involves the complex task of modeling motion trajectories for each visible object, occlusions and dis-occlusions, as well as appearance changes due to viewpoint and lighting. Optical flow based techniques generalize but are suitable only for short temporal ranges. Many methods opt to project the video frames to a low dimensional latent space, achieving long-range predictions. However, these latent representations are often non-interpretable, and therefore difficult to manipulate. This work poses video prediction and interpolation as unsupervised latent structure inference followed by a temporal prediction in this latent space. The latent representations capture foreground semantics without explicit supervision such as keypoints or poses. Further, as each landmark can be mapped to a coordinate indicating where a semantic part is positioned, we can reliably interpolate within the coordinate domain to achieve predictable motion interpolation. Given an image decoder capable of mapping these landmarks back to the image domain, we are able to achieve high-quality long-range video interpolation and extrapolation by operating on the landmark representation space.
Minimal Unique Substrings and Minimal Absent Words in a Sliding Window
Venue: arXiv cs.DS 2019
Year: 2019

A substring $u$ of a string $T$ is called a minimal unique substring (MUS) of $T$ if $u$ occurs exactly once in $T$ and any proper substring of $u$ occurs at least twice in $T$. A string $w$ is called a minimal absent word (MAW) of $T$ if $w$ does not occur in $T$ and any proper substring of $w$ occurs in $T$. In this paper, we study the problems of computing MUSs and MAWs in a sliding window over a given string $T$. We first show how the set of MUSs can change in a sliding window over $T$, and present an $O(n\logσ)$-time and $O(d)$-space algorithm to compute MUSs in a sliding window of width $d$ over $T$, where $σ$ is the maximum number of distinct characters in every window. We then give tight upper and lower bounds on the maximum number of changes in the set of MAWs in a sliding window over $T$. Our bounds improve on the previous results in [Crochemore et al., 2017].
A Comparative Study of Some Central Notions of ASPIC+ and DeLP
Venue: arXiv cs.AI 2019
Year: 2019

This paper formally compares some central notions from two well-known formalisms for rule-based argumentation, DeLP and ASPIC+. The comparisons especially focus on intuitive adequacy and inter-translatability, consistency, and closure properties. As for differences in the definitions of arguments and attack, it turns out that DeLP's definitions are intuitively appealing but that they may not fully comply with Caminada and Amgoud's rationality postulates of strict closure and indirect consistency. For some special cases, the DeLP definitions are shown to fare better than ASPIC+. Next, it is argued that there are reasons to consider a variant of DeLP with grounded semantics, since in some examples its current notion of warrant arguably has counterintuitive consequences and may lead to sets of warranted arguments that are not admissible. Finally, under some minimality and consistency assumptions on ASPIC+ arguments, a one-to-many correspondence between ASPIC+ arguments and DeLP arguments is identified in such a way that if the DeLP warranting procedure is changed to grounded semantics, then DeLP notion of warrant and ASPIC+'s notion of justification are equivalent. This result is proven for three alternative definitions of attack.
Galois Field Arithmetics for Linear Network Coding using AVX512 Instruction Set Extensions
Venue: arXiv cs.DC 2019
Year: 2019

Linear network coding requires arithmetic operations over Galois fields, more specifically over finite extension fields. While coding over GF(2) reduces to simple XOR operations, this field is less preferred for practical applications of random linear network coding due to high chances of linear dependencies and therefore redundant coded packets. Coding over larger fields such as GF(16) and GF(256) does not have that issue, but is significantly slower. SIMD vector extensions of processors such as AVX2 on x86-based systems or NEON on ARM-based devices offer the potential to increase performance by orders of magnitude. In this paper we present an implementation of different algorithms and Galois fields based on the AVX512 instruction set extension and integrate it into the finite field library libmoepgf. We compare the performance of the new implementation to the reference implementation based on AVX2, showing a significant increase in throughput. In addition, we provide a survey of the best possible coding performance offered by a variety of different platforms.
Full-text Search for Verifiable Credential Metadata on Distributed Ledgers
Venue: arXiv cs.CR 2019
Year: 2019

Self-sovereign Identity (SSI) powered by distributed ledger technologies enables more flexible and faster digital identification workflows, while at the same time limiting the control and influence of central authorities. However, a global identity solution must be able to handle myriad credential types from millions of issuing organizations. As metadata about types of digital credentials is readable by everyone on the public permissioned ledger with Hyperledger Indy, anyone could find relevant and trusted credential types for their use cases by looking at the records on the blockchain. To this date, no efficient full-text search mechanism exists that would allow users to search for credential types in a simple and efficient fashion tightly integrated into their applications. In this work, we propose a full-text search framework based on the publicly available metadata on the Hyperledger Indy ledger for retrieving matching credential types. The proposed solution is able to find credential types based on textual input from the user by using a full-text search engine and maintaining a local copy of the ledger. Thus, we do not need to rely on information about credentials coming from a very large candidate pool of third parties we would need to trust, such as the website of a company displaying its own identifier and a list of issued credentials. We have also proven the feasiblity of the concept by implementing and evaluating a prototype of the full-text credential metadata search service.
Real-time stock analysis for blending recipes in industrial plants
Venue: arXiv cs.OH 2019
Year: 2019

Many companies use Excel spreadsheets to keep stock records and to calculate process-specific data. These spreadsheets are often hard to understand and track. And if the user does not protect them, there is a risk that the user randomly changes or erase formulas. The paper focuses on the stocks of products used in a blending process with a known recipe. Developing an application that can bring this data in a centralized form and that can assist the operator in decide is a necessity. When a programmer implements an application that uses data from plants he needs to consider one fundamental aspect as reading real-time data from the process. The real-time stock analysis application takes into account all the above elements. The application is easy to use by an operator in the command room of installation because of the planning algorithms integrated into it. The algorithms proposed and implemented in this paper have well-defined goals: identifying the ingredients needed to achieve the blending process for required quantities, determine the quantities of the finished product that can be made with the existing ingredients and determine the optimum quantities of the finished product. The application implemented in C# intensively uses these algorithms and gives the user the ability to build the result step by step.
Parallel Computation of Graph Embeddings
Venue: arXiv cs.LG 2019
Year: 2019

Graph embedding aims at learning a vector-based representation of vertices that incorporates the structure of the graph. This representation then enables inference of graph properties. Existing graph embedding techniques, however, do not scale well to large graphs. We therefore propose a framework for parallel computation of a graph embedding using a cluster of compute nodes with resource constraints. We show how to distribute any existing embedding technique by first splitting a graph for any given set of constrained compute nodes and then reconciling the embedding spaces derived for these subgraphs. We also propose a new way to evaluate the quality of graph embeddings that is independent of a specific inference task. Based thereon, we give a formal bound on the difference between the embeddings derived by centralised and parallel computation. Experimental results illustrate that our approach for parallel computation scales well, while largely maintaining the embedding quality.
On Learning Disentangled Representations for Gait Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Gait, the walking pattern of individuals, is one of the important biometrics modalities. Most of the existing gait recognition methods take silhouettes or articulated body models as gait features. These methods suffer from degraded recognition performance when handling confounding variables, such as clothing, carrying and viewing angle. To remedy this issue, we propose a novel AutoEncoder framework, GaitNet, to explicitly disentangle appearance, canonical and pose features from RGB imagery. The LSTM integrates pose features over time as a dynamic gait feature while canonical features are averaged as a static gait feature. Both of them are utilized as classification features. In addition, we collect a Frontal-View Gait (FVG) dataset to focus on gait recognition from frontal-view walking, which is a challenging problem since it contains minimal gait cues compared to other views. FVG also includes other important variations, e.g., walking speed, carrying, and clothing. With extensive experiments on CASIA-B, USF, and FVG datasets, our method demonstrates superior performance to the SOTA quantitatively, the ability of feature disentanglement qualitatively, and promising computational efficiency. We further compare our GaitNet with state-of-the-art face recognition to demonstrate the advantages of gait biometrics identification under certain scenarios, e.g., long distance/lower resolutions, cross viewing angles.
Calibrating Wayfinding Decisions in Pedestrian Simulation Models: The Entropy Map
Venue: arXiv cs.HC 2019
Year: 2019

This paper presents entropy maps, an approach to describing and visualising uncertainty among alternative potential movement intentions in pedestrian simulation models. In particular, entropy maps show the instantaneous level of randomness in decisions of a pedestrian agent situated in a specific point of the simulated environment with an heatmap approach. Experimental results highlighting the relevance of this tool supporting modelers are provided and discussed.
"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understanding
Venue: arXiv cs.CL 2019
Year: 2019

Understanding time is crucial for understanding events expressed in natural language. Because people rarely say the obvious, it is often necessary to have commonsense knowledge about various temporal aspects of events, such as duration, frequency, and temporal order. However, this important problem has so far received limited attention. This paper systematically studies this temporal commonsense problem. Specifically, we define five classes of temporal commonsense, and use crowdsourcing to develop a new dataset, MCTACO, that serves as a test set for this task. We find that the best current methods used on MCTACO are still far behind human performance, by about 20%, and discuss several directions for improvement. We hope that the new dataset and our study here can foster more future research on this topic.
To lemmatize or not to lemmatize: how word normalisation affects ELMo performance in word sense disambiguation
Venue: arXiv cs.CL 2019
Year: 2019

We critically evaluate the widespread assumption that deep learning NLP models do not require lemmatized input. To test this, we trained versions of contextualised word embedding ELMo models on raw tokenized corpora and on the corpora with word tokens replaced by their lemmas. Then, these models were evaluated on the word sense disambiguation task. This was done for the English and Russian languages. The experiments showed that while lemmatization is indeed not necessary for English, the situation is different for Russian. It seems that for rich-morphology languages, using lemmatized training and testing data yields small but consistent improvements: at least for word sense disambiguation. This means that the decisions about text pre-processing before training ELMo should consider the linguistic nature of the language in question.
Self Learning from Large Scale Code Corpus to Infer Structure of Method Invocations
Venue: arXiv cs.SE 2019
Year: 2019

Automatically generating code from a textual description of method invocation confronts challenges. There were two current research directions for this problem. One direction focuses on considering a textual description of method invocations as a separate Natural Language query and do not consider the surrounding context of the code. Another direction takes advantage of a practical large scale code corpus for providing a Machine Translation model to generate code. However, this direction got very low accuracy. In this work, we tried to improve these drawbacks by proposing MethodInfoToCode, an approach that embeds context information and optimizes the ability of learning of original Phrase-based Statistical Machine Translation (PBMT) in NLP to infer implementation of method invocation given method name and other context information. We conduct an expression prediction models learned from 2.86 million method invocations from the practical data of high qualities corpus on Github that used 6 popular libraries: JDK, Android, GWT, Joda-Time, Hibernate, and Xstream. By the evaluation, we show that if the developers only write the method name of a method invocation in a body of a method, MethodInfoToCode can predict the generated expression correctly at 73% in F1 score.
Auto-GNN: Neural Architecture Search of Graph Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Graph neural networks (GNN) has been successfully applied to operate on the graph-structured data. Given a specific scenario, rich human expertise and tremendous laborious trials are usually required to identify a suitable GNN architecture. It is because the performance of a GNN architecture is significantly affected by the choice of graph convolution components, such as aggregate function and hidden dimension. Neural architecture search (NAS) has shown its potential in discovering effective deep architectures for learning tasks in image and language modeling. However, existing NAS algorithms cannot be directly applied to the GNN search problem. First, the search space of GNN is different from the ones in existing NAS work. Second, the representation learning capacity of GNN architecture changes obviously with slight architecture modifications. It affects the search efficiency of traditional search methods. Third, widely used techniques in NAS such as parameter sharing might become unstable in GNN. To bridge the gap, we propose the automated graph neural networks (AGNN) framework, which aims to find an optimal GNN architecture within a predefined search space. A reinforcement learning based controller is designed to greedily validate architectures via small steps. AGNN has a novel parameter sharing strategy that enables homogeneous architectures to share parameters, based on a carefully-designed homogeneity definition. Experiments on real-world benchmark datasets demonstrate that the GNN architecture identified by AGNN achieves the best performance, comparing with existing handcrafted models and tradistional search methods.