OpenRaiser/Intern-Atlas huggingface dataset:
Sensory Optimization: Neural Networks as a Model for Understanding and Creating Art
Venue: arXiv cs.AI 2019
Year: 2019

This article is about the cognitive science of visual art. Artists create physical artifacts (such as sculptures or paintings) which depict people, objects, and events. These depictions are usually stylized rather than photo-realistic. How is it that humans are able to understand and create stylized representations? Does this ability depend on general cognitive capacities or an evolutionary adaptation for art? What role is played by learning and culture? Machine Learning can shed light on these questions. It's possible to train convolutional neural networks (CNNs) to recognize objects without training them on any visual art. If such CNNs can generalize to visual art (by creating and understanding stylized representations), then CNNs provide a model for how humans could understand art without innate adaptations or cultural learning. I argue that Deep Dream and Style Transfer show that CNNs can create a basic form of visual art, and that humans could create art by similar processes. This suggests that artists make art by optimizing for effects on the human object-recognition system. Physical artifacts are optimized to evoke real-world objects for this system (e.g. to evoke people or landscapes) and to serve as superstimuli for this system.
Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we study Reinforcement Learning from Demonstrations (RLfD) that improves the exploration efficiency of Reinforcement Learning (RL) by providing expert demonstrations. Most of existing RLfD methods require demonstrations to be perfect and sufficient, which yet is unrealistic to meet in practice. To work on imperfect demonstrations, we first define an imperfect expert setting for RLfD in a formal way, and then point out that previous methods suffer from two issues in terms of optimality and convergence, respectively. Upon the theoretical findings we have derived, we tackle these two issues by regarding the expert guidance as a soft constraint on regulating the policy exploration of the agent, which eventually leads to a constrained optimization problem. We further demonstrate that such problem is able to be addressed efficiently by performing a local linear search on its dual form. Considerable empirical evaluations on a comprehensive collection of benchmarks indicate our method attains consistent improvement over other RLfD counterparts.
Study of Non-Uniform Channel Polarization and Design of Polar Codes with Arbitrary Rates
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we present the concept of non-uniform channel polarization and a scheme for rate-compatible polar code construction for any codeword length using additive white Gaussian noise (AWGN) channels and the successive cancellation (SC) decoder. A Non-Uniform Polarization technique based on the Gaussian Approximation (NUPGA) is developed and an efficient rate-compatible design technique is devised to choose the best channels for transmission by a process of re-polarization of the codeword with the desired length. Simulations illustrate the proposed NUPGA design against existing rate-compatible techniques.
Stochastic Magnetoelectric Neuron for Temporal Information Encoding
Venue: arXiv cs.ET 2019
Year: 2019

Emulating various facets of computing principles of the brain can potentially lead to the development of neuro-computers that are able to exhibit brain-like cognitive capabilities. In this letter, we propose a magnetoelectronic neuron that utilizes noise as a computing resource and is able to encode information over time through the independent control of external voltage signals. We extensively characterize the device operation using simulations and demonstrate its suitability for neuromorphic computing platforms performing temporal information encoding.
Multiple-Source Ellipsoidal Localization Using Acoustic Energy Measurements
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, the multiple-source ellipsoidal localization problem based on acoustic energy measurements is investigated via set-membership estimation theory. When the probability density function of measurement noise is unknown-but-bounded, multiple-source localization is a difficult problem since not only the acoustic energy measurements are complicated nonlinear functions of multiple sources, but also the multiple sources bring about a high-dimensional state estimation problem. First, when the energy parameter and the position of the source are bounded in an interval and a ball respectively, the nonlinear remainder bound of the Taylor series expansion is obtained analytically on-line. Next, based on the separability of the nonlinear measurement function, an efficient estimation procedure is developed. It solves the multiple-source localization problem by using an alternating optimization iterative algorithm, in which the remainder bound needs to be known on-line. For this reason, we first derive the remainder bound analytically. When the energy decay factor is unknown but bounded, an efficient estimation procedure is developed based on interval mathematics. Finally, numerical examples demonstrate the effectiveness of the ellipsoidal localization algorithms for multiple-source localization. In particular, our results show that when the noise is non-Gaussian, the set-membership localization algorithm performs better than the EM localization algorithm.
Towards the Automation of Deep Image Prior
Venue: arXiv cs.CV 2019
Year: 2019

Single image inverse problem is a notoriously challenging ill-posed problem that aims to restore the original image from one of its corrupted versions. Recently, this field has been immensely influenced by the emergence of deep-learning techniques. Deep Image Prior (DIP) offers a new approach that forces the recovered image to be synthesized from a given deep architecture. While DIP is quite an effective unsupervised approach, it is deprecated in real-world applications because of the requirement of human assistance. In this work, we aim to find the best-recovered image without the assistance of humans by adding a stopping criterion, which will reach maximum when the iteration no longer improves the image quality. More specifically, we propose to add a pseudo noise to the corrupted image and measure the pseudo-noise component in the recovered image by the orthogonality between signal and noise. The accuracy of the orthogonal stopping criterion has been demonstrated for several tested problems such as denoising, super-resolution, and inpainting, in which 38 out of 40 experiments are higher than 95%.
Rumor Detection on Social Media: Datasets, Methods and Opportunities
Venue: arXiv cs.IR 2019
Year: 2019

Social media platforms have been used for information and news gathering, and they are very valuable in many applications. However, they also lead to the spreading of rumors and fake news. Many efforts have been taken to detect and debunk rumors on social media by analyzing their content and social context using machine learning techniques. This paper gives an overview of the recent studies in the rumor detection field. It provides a comprehensive list of datasets used for rumor detection, and reviews the important studies based on what types of information they exploit and the approaches they take. And more importantly, we also present several new directions for future research.
Real-Time Semantic Segmentation via Multiply Spatial Fusion Network
Venue: arXiv cs.CV 2019
Year: 2019

Real-time semantic segmentation plays a significant role in industry applications, such as autonomous driving, robotics and so on. It is a challenging task as both efficiency and performance need to be considered simultaneously. To address such a complex task, this paper proposes an efficient CNN called Multiply Spatial Fusion Network (MSFNet) to achieve fast and accurate perception. The proposed MSFNet uses Class Boundary Supervision to process the relevant boundary information based on our proposed Multi-features Fusion Module which can obtain spatial information and enlarge receptive field. Therefore, the final upsampling of the feature maps of 1/8 original image size can achieve impressive results while maintaining a high speed. Experiments on Cityscapes and Camvid datasets show an obvious advantage of the proposed approach compared with the existing approaches. Specifically, it achieves 77.1% Mean IOU on the Cityscapes test dataset with the speed of 41 FPS for a 1024*2048 input, and 75.4% Mean IOU with the speed of 91 FPS on the Camvid test dataset.
Hebbian Synaptic Modifications in Spiking Neurons that Learn
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we derive a new model of synaptic plasticity, based on recent algorithms for reinforcement learning (in which an agent attempts to learn appropriate actions to maximize its long-term average reward). We show that these direct reinforcement learning algorithms also give locally optimal performance for the problem of reinforcement learning with multiple agents, without any explicit communication between agents. By considering a network of spiking neurons as a collection of agents attempting to maximize the long-term average of a reward signal, we derive a synaptic update rule that is qualitatively similar to Hebb's postulate. This rule requires only simple computations, such as addition and leaky integration, and involves only quantities that are available in the vicinity of the synapse. Furthermore, it leads to synaptic connection strengths that give locally optimal values of the long term average reward. The reinforcement learning paradigm is sufficiently broad to encompass many learning problems that are solved by the brain. We illustrate, with simulations, that the approach is effective for simple pattern classification and motor learning tasks.
Learning with Hierarchical Complement Objective
Venue: arXiv cs.CV 2019
Year: 2019

Label hierarchies widely exist in many vision-related problems, ranging from explicit label hierarchies existed in image classification to latent label hierarchies existed in semantic segmentation. Nevertheless, state-of-the-art methods often deploy cross-entropy loss that implicitly assumes class labels to be exclusive and thus independence from each other. Motivated by the fact that classes from the same parental category usually share certain similarity, we design a new training diagram called Hierarchical Complement Objective Training (HCOT) that leverages the information from label hierarchy. HCOT maximizes the probability of the ground truth class, and at the same time, neutralizes the probabilities of rest of the classes in a hierarchical fashion, making the model take advantage of the label hierarchy explicitly. The proposed HCOT is evaluated on both image classification and semantic segmentation tasks. Experimental results confirm that HCOT outperforms state-of-the-art models in CIFAR-100, ImageNet-2012, and PASCAL-Context. The study further demonstrates that HCOT can be applied on tasks with latent label hierarchies, which is a common characteristic in many machine learning tasks.
Unsupervised Visual Representation Learning with Increasing Object Shape Bias
Venue: arXiv cs.CV 2019
Year: 2019

(Very early draft)Traditional supervised learning keeps pushing convolution neural network(CNN) achieving state-of-art performance. However, lack of large-scale annotation data is always a big problem due to the high cost of it, even ImageNet dataset is over-fitted by complex models now. The success of unsupervised learning method represented by the Bert model in natural language processing(NLP) field shows its great potential. And it makes that unlimited training samples becomes possible and the great universal generalization ability changes NLP research direction directly. In this article, we purpose a novel unsupervised learning method based on contrastive predictive coding. Under that, we are able to train model with any non-annotation images and improve model's performance to reach state-of-art performance at the same level of model complexity. Beside that, since the number of training images could be unlimited amplification, an universal large-scale pre-trained computer vision model is possible in the future.
Distribution Context Aware Loss for Person Re-identification
Venue: arXiv cs.CV 2019
Year: 2019

To learn the optimal similarity function between probe and gallery images in Person re-identification, effective deep metric learning methods have been extensively explored to obtain discriminative feature embedding. However, existing metric loss like triplet loss and its variants always emphasize pair-wise relations but ignore the distribution context in feature space, leading to inconsistency and sub-optimal. In fact, the similarity of one pair not only decides the match of this pair, but also has potential impacts on other sample pairs. In this paper, we propose a novel Distribution Context Aware (DCA) loss based on triplet loss to combine both numerical similarity and relation similarity in feature space for better clustering. Extensive experiments on three benchmarks including Market-1501, DukeMTMC-reID and MSMT17, evidence the favorable performance of our method against the corresponding baseline and other state-of-the-art methods.
Fast 3D Pose Refinement with RGB Images
Venue: arXiv cs.CV 2019
Year: 2019

Pose estimation is a vital step in many robotics and perception tasks such as robotic manipulation, autonomous vehicle navigation, etc. Current state-of-the-art pose estimation methods rely on deep neural networks with complicated structures and long inference times. While highly robust, they require computing power often unavailable on mobile robots. We propose a CNN-based pose refinement system which takes a coarsely estimated 3D pose from a computationally cheaper algorithm along with a bounding box image of the object, and returns a highly refined pose. Our experiments on the YCB-Video dataset show that our system can refine 3D poses to an extremely high precision with minimal training data.
Rebalancing Learning on Evolving Data Streams
Venue: arXiv cs.LG 2019
Year: 2019

Nowadays, every device connected to the Internet generates an ever-growing stream of data (formally, unbounded). Machine Learning on unbounded data streams is a grand challenge due to its resource constraints. In fact, standard machine learning techniques are not able to deal with data whose statistics is subject to gradual or sudden changes without any warning. Massive Online Analysis (MOA) is the collective name, as well as a software library, for new learners that are able to manage data streams. In this paper, we present a research study on streaming rebalancing. Indeed, data streams can be imbalanced as static data, but there is not a method to rebalance them incrementally, one element at a time. For this reason we propose a new streaming approach able to rebalance data streams online. Our new methodology is evaluated against some synthetically generated datasets using prequential evaluation in order to demonstrate that it outperforms the existing approaches.
Multi-task Sentence Encoding Model for Semantic Retrieval in Question Answering Systems
Venue: arXiv cs.CL 2019
Year: 2019

Question Answering (QA) systems are used to provide proper responses to users' questions automatically. Sentence matching is an essential task in the QA systems and is usually reformulated as a Paraphrase Identification (PI) problem. Given a question, the aim of the task is to find the most similar question from a QA knowledge base. In this paper, we propose a Multi-task Sentence Encoding Model (MSEM) for the PI problem, wherein a connected graph is employed to depict the relation between sentences, and a multi-task learning model is applied to address both the sentence matching and sentence intent classification problem. In addition, we implement a general semantic retrieval framework that combines our proposed model and the Approximate Nearest Neighbor (ANN) technology, which enables us to find the most similar question from all available candidates very quickly during online serving. The experiments show the superiority of our proposed method as compared with the existing sentence matching models.
Algorithmic Discrepancy Minimization
Venue: arXiv cs.DS 2019
Year: 2019

This report will be a literature review on a result in algorithmic discrepancy theory. We will begin by providing a quick overview on discrepancy theory and some major results in the field, and then focus on an important result by Shachar Lovett and Raghu Meka. We restate the main algorithm and ideas of the paper, and rewrite proofs for some of the major results in the paper.
A Graph Autoencoder Approach to Causal Structure Learning
Venue: arXiv cs.LG 2019
Year: 2019

Causal structure learning has been a challenging task in the past decades and several mainstream approaches such as constraint- and score-based methods have been studied with theoretical guarantees. Recently, a new approach has transformed the combinatorial structure learning problem into a continuous one and then solved it using gradient-based optimization methods. Following the recent state-of-the-arts, we propose a new gradient-based method to learn causal structures from observational data. The proposed method generalizes the recent gradient-based methods to a graph autoencoder framework that allows nonlinear structural equation models and is easily applicable to vector-valued variables. We demonstrate that on synthetic datasets, our proposed method outperforms other gradient-based methods significantly, especially on large causal graphs. We further investigate the scalability and efficiency of our method, and observe a near linear training time when scaling up the graph size.
RIP constants for deterministic compressed sensing matrices-beyond Gershgorin
Venue: arXiv cs.IT 2019
Year: 2019

Compressed sensing (CS) is a signal acquisition paradigm to simultaneously acquire and reduce dimension of signals that admit sparse representations. This is achieved by collecting linear, non-adaptive measurements of a signal, which can be formalized as multiplying the signal with a "measurement matrix". If the measurement satisfies the so-called restricted isometry property (RIP), then it will be appropriate to be used in compressed sensing. While a wide class of random matrices provably satisfy the RIP with high probability, explicit and deterministic constructions have been shown (so far) to satisfy the RIP only in a significantly suboptimal regime. In this paper, we propose two novel approaches for improving the RIP constant estimates based on Gershgorin circle theorem for a specific deterministic construction based on Paley tight frames, obtaining an improvement over the Gershgorin bound by a multiplicative constant. In one approach we use a recent result on the spectra of the skew-adjacency matrices of oriented graphs. In the other approach, we use the so-called Dembo bounds on the extreme eigenvalues of a positive semidefinite Hermitian matrix. We also generalize these bounds and we combine the new bounds with a conjecture we make regarding the distribution of quadratic residues in a finite field to provide a potential path to break the so-called "square-root barrier"-we provide a proof based on the assumption that the conjecture holds.
Subspace Shapes: Enhancing High-Dimensional Subspace Structures via Ambient Occlusion Shading
Venue: arXiv cs.HC 2019
Year: 2019

We test the hypothesis whether transforming a data matrix into a 3D shaded surface or even a volumetric display can be more appealing to humans than a scatterplot since it makes direct use of the innate 3D scene understanding capabilities of the human visual system. We also test whether 3D shaded displays can add a significant amount of information to the visualization of high-dimensional data, especially when enhanced with proper tools to navigate the various 3D subspaces. Our experiments suggest that mainstream users prefer shaded displays over scatterplots for visual cluster analysis tasks after receiving training for both. Our experiments also provide evidence that 3D displays can better communicate spatial relationships, size, and shape of clusters.
Deep and Dense Sarcasm Detection
Venue: arXiv cs.CL 2019
Year: 2019

Recent work in automated sarcasm detection has placed a heavy focus on context and meta-data. Whilst certain utterances indeed require background knowledge and commonsense reasoning, previous works have only explored shallow models for capturing the lexical, syntactic and semantic cues present within a text. In this paper, we propose a deep 56 layer network, implemented with dense connectivity to model the isolated utterance and extract richer features therein. We compare our approach against recent state-of-the-art architectures which make considerable use of extrinsic information, and demonstrate competitive results whilst using only the local features of the text. Further, we provide an analysis of the dependency of prior convolution outputs in generating the final feature maps. Finally a case study is presented, supporting that our approach accurately classifies additional uses of clear sarcasm, which a standard CNN misclassifies.
Zero-Interaction Security -- Towards Sound Experimental Validation
Venue: arXiv cs.CR 2019
Year: 2019

Reproducibility and realistic datasets are crucial for advancing research. Unfortunately, they are often neglected as valid scientific contributions in many young disciplines, with computer science being no exception. In this article, we show the challenges encountered when reproducing the work of others, collecting realistic data in the wild, and ensuring that our own work is reproducible in turn. The presented findings are based on our study investigating the limits of zero-interaction security (ZIS) -- a novel concept, leveraging sensor data collected by Internet of Things (IoT) devices to pair or authenticate devices. In particular, we share our experiences in reproducing five state-of-the-art ZIS schemes, collecting a comprehensive dataset of sensor data from the real world, evaluating these schemes on the collected data, and releasing the data, code, and documentation to facilitate reproducibility of our results. In our discussion, we outline general considerations when conducting similar studies and give specific examples of technical and methodological issues that we experienced. We hope that our findings will raise awareness about the importance of reproducibility and realistic datasets in computer science and inform future research.
A Hierarchical Framework to Generate Robust Biped Locomotion Based on Divergent Component of Motion
Venue: arXiv cs.RO 2019
Year: 2019

Keeping the stability can be counted as the essential ability of a humanoid robot to step out of the laboratory to work in our real environment. Since humanoid robots have similar kinematic to a human, humans expect these robots to be robustly capable of stabilizing even in a challenging situation like while a severe push is applied. This paper presents a robust walking framework which not only takes into account the traditional push recovery approaches (e.g., ankle, hip and step strategies) but also uses the concept of Divergent Component of the Motion (DCM) to adjust next step timing and location. The control core of the proposed framework is composed of a Linear-Quadratic-Gaussian (LQG) controller and two proportional controllers. In this framework, the LQG controller tries to track the reference trajectories and the proportional controllers are designed to adjust the next step timing and location that allow the robot to recover from a severe push. The robustness and the performance of the proposed framework have been validated by performing a set of simulations, including walking and push recovery using MATLAB. The simulation results verified that the proposed framework is capable of providing a robust walking even in very challenging situations.