OpenRaiser/Intern-Atlas huggingface dataset:
Explicit and Efficient Constructions of Coding Schemes for the Binary Deletion Channel and the Poisson Repeat Channel
Venue: arXiv cs.IT 2019
Year: 2019

This work gives an explicit construction of a family of error correcting codes for the binary deletion channel and for the Poisson repeat channel. In the binary deletion channel with parameter $p$ (BDC$_p$) every bit is deleted independently with probability $p$. A lower bound of $(1-p)/9$ is known on the capacity of the BDC$_p$ \cite{mitzenmacher2006simple}, yet no explicit construction is known to achieve this rate. We give an explicit family of codes of rate $(1-p)/16$, for every $p$. This improves upon the work of Guruswami and Li \cite{guruswami2017efficiently} that gave a construction of rate $(1-p)/120$. The codes in our family have polynomial time encoding and decoding algorithms. Another channel considered in this work is the Poisson repeat channel with parameter $λ$ (PRC$_λ$) in which every bit is replaced with a discrete Poisson number of copies of that bit, where the number of copies has mean $λ$. We show that our construction works for this channel as well. As far as we know, this is the first explicit construction of an error correcting code for PRC$_λ$.
Learning Coupled Spatial-temporal Attention for Skeleton-based Action Recognition
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a coupled spatial-temporal attention (CSTA) model for skeleton-based action recognition, which aims to figure out the most discriminative joints and frames in spatial and temporal domains simultaneously. Conventional approaches usually consider all the joints or frames in a skeletal sequence equally important, which are unrobust to ambiguous and redundant information. To address this, we first learn two sets of weights for different joints and frames through two subnetworks respectively, which enable the model to have the ability of "paying attention to" the relatively informative section. Then, we calculate the cross product based on the weights of joints and frames for the coupled spatial-temporal attention. Moreover, our CSTA mechanisms can be easily plugged into existing hierarchical CNN models (CSTA-CNN) to realize their function. Extensive experimental results on the recently collected UESTC dataset and the currently largest NTU dataset have shown the effectiveness of our proposed method for skeleton-based action recognition.
Scheduled Differentiable Architecture Search for Visual Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Convolutional Neural Networks (CNN) have been regarded as a capable class of models for visual recognition problems. Nevertheless, it is not trivial to develop generic and powerful network architectures, which requires significant efforts of human experts. In this paper, we introduce a new idea for automatically exploring architectures on a remould of Differentiable Architecture Search (DAS), which possesses the efficient search via gradient descent. Specifically, we present Scheduled Differentiable Architecture Search (SDAS) for both image and video recognition that nicely integrates the selection of operations during training with a schedule. Technically, an architecture or a cell is represented as a directed graph. Our SDAS gradually fixes the operations on the edges in the graph in a progressive and scheduled manner, as opposed to a one-step decision of operations for all the edges once the training completes in existing DAS, which may make the architecture brittle. Moreover, we enlarge the search space of SDAS particularly for video recognition by devising several unique operations to encode spatio-temporal dynamics and demonstrate the impact in affecting the architecture search of SDAS. Extensive experiments of architecture learning are conducted on CIFAR10, Kinetics10, UCF101 and HMDB51 datasets, and superior results are reported when comparing to DAS method. More remarkably, the search by our SDAS is around 2-fold faster than DAS. When transferring the learnt cells on CIFAR10 and Kinetics10 respectively to large-scale ImageNet and Kinetics400 datasets, the constructed network also outperforms several state-of-the-art hand-crafted structures.
Large Scale Joint Semantic Re-Localisation and Scene Understanding via Globally Unique Instance Coordinate Regression
Venue: arXiv cs.CV 2019
Year: 2019

In this work we present a novel approach to joint semantic localisation and scene understanding. Our work is motivated by the need for localisation algorithms which not only predict 6-DoF camera pose but also simultaneously recognise surrounding objects and estimate 3D geometry. Such capabilities are crucial for computer vision guided systems which interact with the environment: autonomous driving, augmented reality and robotics. In particular, we propose a two step procedure. During the first step we train a convolutional neural network to jointly predict per-pixel globally unique instance labels and corresponding local coordinates for each instance of a static object (e.g. a building). During the second step we obtain scene coordinates by combining object center coordinates and local coordinates and use them to perform 6-DoF camera pose estimation. We evaluate our approach on real world (CamVid-360) and artificial (SceneCity) autonomous driving datasets. We obtain smaller mean distance and angular errors than state-of-the-art 6-DoF pose estimation algorithms based on direct pose regression and pose estimation from scene coordinates on all datasets. Our contributions include: (i) a novel formulation of scene coordinate regression as two separate tasks of object instance recognition and local coordinate regression and a demonstration that our proposed solution allows to predict accurate 3D geometry of static objects and estimate 6-DoF pose of camera on (ii) maps larger by several orders of magnitude than previously attempted by scene coordinate regression methods, as well as on (iii) lightweight, approximate 3D maps built from 3D primitives such as building-aligned cuboids.
A note on the neighbour-distinguishing index of digraphs
Venue: arXiv cs.DM 2019
Year: 2019

In this note, we introduce and study a new version of neighbour-distinguishing arc-colourings of digraphs. An arc-colouring $γ$ of a digraph $D$ is proper if no two arcs with the same head or with the same tail are assigned the same colour. For each vertex $u$ of $D$, we denote by $S_γ^-(u)$ and $S_γ^+(u)$ the sets of colours that appear on the incoming arcs and on the outgoing arcs of $u$, respectively. An arc colouring $γ$ of $D$ is \emph{neighbour-distinguishing} if, for every two adjacent vertices $u$ and $v$ of $D$, the ordered pairs $(S_γ^-(u),S_γ^+(u))$ and $(S_γ^-(v),S_γ^+(v))$ are distinct. The neighbour-distinguishing index of $D$ is then the smallest number of colours needed for a neighbour-distinguishing arc-colouring of $D$.We prove upper bounds on the neighbour-distinguishing index of various classes of digraphs.
The Golden Eras of Graphene Science and Technology: Bibliographic Evidences From Journal and Patent Publications
Venue: arXiv cs.DL 2019
Year: 2019

Today's scientific research is an expensive enterprise funded largely by taxpayers' and corporate groups' monies. It is a critical part in the competition between nations, and all nations want to discover fields of research that promise to create future industries, and dominate these by building up scientific and technological expertise early. However, our understanding of the value chain going from science to technology is still in a relatively infant stage, and the conversion of scientific leadership into market dominance remains very much an alchemy rather than a science. In this paper, we analyze bibliometric records of scientific journal publications and patents related to graphene, at the aggregate level as well as on the temporal and spatial dimensions. We find the present leaders of graphene science and technology emerged rather late in the race, after the initial scientific leaders lost their footings. More importantly, notwithstanding the amount of funding already committed, we find evidences that suggest the 'Golden Eras' of graphene science and technology were in 2010 and 2012 respectively, in spite of the continued growth of journal and patent publications in this area.
Space Diversity-Based Grant-Free Random Access for Critical and Non-Critical IoT Services
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, we study the coexistence of critical and non-critical Internet of Things (IoT) services on a grant-free channel consisting of radio access and backhaul segments. On the radio access segment, IoT devices send packets to access points (APs) over an erasure collision channel using the slotted ALOHA protocol. Then, the APs forward correctly received messages to a base station (BS) over a shared wireless backhaul segment, modeled as an erasure collision channel. The APs hence play the role of uncoordinated relays that provide space diversity and may reduce performance losses caused by collisions. Both non-orthogonal and inter-service orthogonal resource sharing are considered and compared. Throughput and reliability metrics are analyzed, and numerical results are provided to assess the performance trade-offs between critical and non-critical IoT services.
Upper Body Pose Estimation Using Wearable Inertial Sensors and Multiplicative Kalman Filter
Venue: arXiv cs.RO 2019
Year: 2019

Estimating the limbs pose in a wearable way may benefit multiple areas such as rehabilitation, teleoperation, human-robot interaction, gaming, and many more. Several solutions are commercially available, but they are usually expensive or not wearable/portable. We present a wearable pose estimation system (WePosE), based on inertial measurements units (IMUs), for motion analysis and body tracking. Differently from camera-based approaches, the proposed system does not suffer from occlusion problems and lighting conditions, it is cost effective and it can be used in indoor and outdoor environments. Moreover, since only accelerometers and gyroscopes are used to estimate the orientation, the system can be used also in the presence of iron and magnetic disturbances. An experimental validation using a high precision optical tracker has been performed. Results confirmed the effectiveness of the proposed approach.
NLVR2 Visual Bias Analysis
Venue: arXiv cs.CL 2019
Year: 2019

NLVR2 (Suhr et al., 2019) was designed to be robust for language bias through a data collection process that resulted in each natural language sentence appearing with both true and false labels. The process did not provide a similar measure of control for visual bias. This technical report analyzes the potential for visual bias in NLVR2. We show that some amount of visual bias likely exists. Finally, we identify a subset of the test data that allows to test for model performance in a way that is robust to such potential biases. We show that the performance of existing models (Li et al., 2019; Tan and Bansal 2019) is relatively robust to this potential bias. We propose to add the evaluation on this subset of the data to the NLVR2 evaluation protocol, and update the official release to include it. A notebook including an implementation of the code used to replicate this analysis is available at http://nlvr.ai/NLVR2BiasAnalysis.html.
Satisficing Mentalizing: Bayesian Models of Theory of Mind Reasoning in Scenarios with Different Uncertainties
Venue: arXiv cs.AI 2019
Year: 2019

The ability to interpret the mental state of another agent based on its behavior, also called Theory of Mind (ToM), is crucial for humans in any kind of social interaction. Artificial systems, such as intelligent assistants, would also greatly benefit from such mentalizing capabilities. However, humans and systems alike are bound by limitations in their available computational resources. This raises the need for satisficing mentalizing, reconciling accuracy and efficiency in mental state inference that is good enough for a given situation. In this paper, we present different Bayesian models of ToM reasoning and evaluate them based on actual human behavior data that were generated under different kinds of uncertainties. We propose a Switching approach that combines specialized models, embodying simplifying presumptions, in order to achieve a more statisficing mentalizing compared to a Full Bayesian ToM model.
Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings
Venue: arXiv cs.CL 2019
Year: 2019

Contextualized word embeddings (CWE) such as provided by ELMo (Peters et al., 2018), Flair NLP (Akbik et al., 2018), or BERT (Devlin et al., 2019) are a major recent innovation in NLP. CWEs provide semantic vector representations of words depending on their respective context. Their advantage over static word embeddings has been shown for a number of tasks, such as text classification, sequence tagging, or machine translation. Since vectors of the same word type can vary depending on the respective context, they implicitly provide a model for word sense disambiguation (WSD). We introduce a simple but effective approach to WSD using a nearest neighbor classification on CWEs. We compare the performance of different CWE models for the task and can report improvements above the current state of the art for two standard WSD benchmark datasets. We further show that the pre-trained BERT model is able to place polysemic words into distinct 'sense' regions of the embedding space, while ELMo and Flair NLP do not seem to possess this ability.
Model-Agnostic Linear Competitors -- When Interpretable Models Compete and Collaborate with Black-Box Models
Venue: arXiv cs.LG 2019
Year: 2019

Driven by an increasing need for model interpretability, interpretable models have become strong competitors for black-box models in many real applications. In this paper, we propose a novel type of model where interpretable models compete and collaborate with black-box models. We present the Model-Agnostic Linear Competitors (MALC) for partially interpretable classification. MALC is a hybrid model that uses linear models to locally substitute any black-box model, capturing subspaces that are most likely to be in a class while leaving the rest of the data to the black-box. MALC brings together the interpretable power of linear models and good predictive performance of a black-box model. We formulate the training of a MALC model as a convex optimization. The predictive accuracy and transparency (defined as the percentage of data captured by the linear models) balance through a carefully designed objective function and the optimization problem is solved with the accelerated proximal gradient method. Experiments show that MALC can effectively trade prediction accuracy for transparency and provide an efficient frontier that spans the entire spectrum of transparency.
Collaborative Intelligent Cross-Camera Video Analytics at Edge: Opportunities and Challenges
Venue: arXiv cs.NI 2019
Year: 2019

Nowadays, video cameras are deployed in large scale for spatial monitoring of physical places (e.g., surveillance systems in the context of smart cities). The massive camera deployment, however, presents new challenges for analyzing the enormous data, as the cost of high computational overhead of sophisticated deep learning techniques imposes a prohibitive overhead, in terms of energy consumption and processing throughput, on such resource-constrained edge devices. To address these limitations, this paper envisions a collaborative intelligent cross-camera video analytics paradigm at the network edge in which camera nodes adjust their pipelines (e.g., inference) to incorporate correlated observations and shared knowledge from other nodes' contents. By harassing redundant spatio-temporal to reduce the size of the inference search space in one hand, and intelligent collaboration between video nodes on the other, we discuss how such collaborative paradigm can considerably improve accuracy, reduce latency and decrease communication bandwidth compared to non-collaborative baselines. This paper also describes major opportunities and challenges in realizing such a paradigm.
Hydrocephalus verification on brain magnetic resonance images with deep convolutional neural networks and "transfer learning" technique
Venue: arXiv cs.CV 2019
Year: 2019

The hydrocephalus can be either an independent disease or a concomitant symptom of a number of pathologies, therefore representing an urgent issue in the present-day clinical practice. Deep Learning is an evolving technology and the part of a broader field of Machine Learning. Deep learning is currently actively researched in the field of radiology. The aim of this study was to evaluate deep learning applicability to the diagnostics of hydrocephalus with the use of MRI images. We retrospectively collected, annotated, and preprocessed the brain MRI data of 200 patients with and without radiological signs of hydrocephalus. We applied a state-of-the-art deep convolutional neural network in conjunction with transfer learning method to train a hydrocephalus classifier model. Using deep convolutional neural networks, we achieved a high quality of machine learning model. Accuracy, sensitivity, and specificity of hydrocephalus signs identification was 97%, 98%, and 96% respectively. In this study, we demonstrated the capacity of deep neural networks to identify hydrocephalus syndrome using brain MRI images. Applying transfer learning technique, the high quality of classification was achieved although trained on rather limited data.
Cross-Lingual Natural Language Generation via Pre-Training
Venue: arXiv cs.CL 2019
Year: 2019

In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages. We propose to pretrain the encoder and the decoder of a sequence-to-sequence model under both monolingual and cross-lingual settings. The pre-training objective encourages the model to represent different languages in the shared space, so that we can conduct zero-shot cross-lingual transfer. After the pre-training procedure, we use monolingual data to fine-tune the pre-trained model on downstream NLG tasks. Then the sequence-to-sequence model trained in a single language can be directly evaluated beyond that language (i.e., accepting multi-lingual input and producing multi-lingual output). Experimental results on question generation and abstractive summarization show that our model outperforms the machine-translation-based pipeline methods for zero-shot cross-lingual generation. Moreover, cross-lingual transfer improves NLG performance of low-resource languages by leveraging rich-resource language data. Our implementation and data are available at https://github.com/CZWin32768/xnlg.
On the Distance between Timed Automata
Venue: arXiv cs.FL 2019
Year: 2019

The problem of inclusion of the language accepted by timed automaton $A$ (e.g., the implementation) in the language accepted by $B$ (e.g., the specification) is, in general, undecidable in the class of non-deterministic timed automata. In order to tackle this disturbing problem we show how to effectively construct deterministic timed automata $A_d$ and $B_d$ that are discretizations (digitizations) of the non-deterministic timed automata $A$ and $B$ and differ from the original automata by at most $\frac{1}{6}$ time units on each occurrence of an event. Language inclusion in the discretized timed automata is decidable and it is also decidable when instead of $\mathfrak{L}(B)$ we consider $\overline{\mathfrak{L}(B)}$, the closure of ${\mathfrak{L}(B)}$ in the Euclidean topology: if $\mathfrak{L}(A_d) \nsubseteq \mathfrak{L}(B_d)$ then $\mathfrak{L}(A) \nsubseteq \mathfrak{L}(B)$ and if $\mathfrak{L}(A_d) \subseteq \mathfrak{L}(B_d)$ then $\mathfrak{L}(A) \subseteq \overline{\mathfrak{L}(B)}$. Moreover, if $\mathfrak{L}(A_d) \nsubseteq \mathfrak{L}(B_d)$ we would like to know how far away is $\mathfrak{L}(A_d)$ from being included in $\mathfrak{L}(B_d)$. For that matter we define the distance between the languages of timed automata as the limit on how far away a timed trace of one timed automaton can be from the closest timed trace of the other timed automaton. We then show how one can decide under some restriction whether the distance between two timed automata is finite or infinite.
Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Gradient-based methods for optimisation of objectives in stochastic settings with unknown or intractable dynamics require estimators of derivatives. We derive an objective that, under automatic differentiation, produces low-variance unbiased estimators of derivatives at any order. Our objective is compatible with arbitrary advantage estimators, which allows the control of the bias and variance of any-order derivatives when using function approximation. Furthermore, we propose a method to trade off bias and variance of higher order derivatives by discounting the impact of more distant causal dependencies. We demonstrate the correctness and utility of our objective in analytically tractable MDPs and in meta-reinforcement-learning for continuous control.
Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models
Venue: arXiv cs.CL 2019
Year: 2019

Neural language models (LMs) perform well on tasks that require sensitivity to syntactic structure. Drawing on the syntactic priming paradigm from psycholinguistics, we propose a novel technique to analyze the representations that enable such success. By establishing a gradient similarity metric between structures, this technique allows us to reconstruct the organization of the LMs' syntactic representational space. We use this technique to demonstrate that LSTM LMs' representations of different types of sentences with relative clauses are organized hierarchically in a linguistically interpretable manner, suggesting that the LMs track abstract properties of the sentence.
Persuasion and Incentives Through the Lens of Duality
Venue: arXiv cs.GT 2019
Year: 2019

Lagrangian duality underlies both classical and modern mechanism design. In particular, the dual perspective often permits simple and detail-free characterizations of optimal and approximately optimal mechanisms. This paper applies this same methodology to a close cousin of traditional mechanism design, one which shares conceptual and technical elements with its more mature relative: the burgeoning field of persuasion. The dual perspective permits us to analyze optimal persuasion schemes both in settings which have been analyzed in prior work, as well as for natural generalizations which we are the first to explore in depth. Most notably, we permit combining persuasion policies with payments, which serve to augment the persuasion power of the scheme. In both single and multi-receiver settings, as well as under a variety of constraints on payments, we employ duality to obtain structural insights, as well as tractable and simple characterizations of optimal policies.