OpenRaiser/Intern-Atlas huggingface dataset:
Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data
Venue: arXiv cs.LG 2019
Year: 2019

Data augmentation has been widely applied as an effective methodology to improve generalization in particular when training deep neural networks. Recently, researchers proposed a few intensive data augmentation techniques, which indeed improved accuracy, yet we notice that these methods augment data have also caused a considerable gap between clean and augmented data. In this paper, we revisit this problem from an analytical perspective, for which we estimate the upper-bound of expected risk using two terms, namely, empirical risk and generalization error, respectively. We develop an understanding of data augmentation as regularization, which highlights the major features. As a result, data augmentation significantly reduces the generalization error, but meanwhile leads to a slightly higher empirical risk. On the assumption that data augmentation helps models converge to a better region, the model can benefit from a lower empirical risk achieved by a simple method, i.e., using less-augmented data to refine the model trained on fully-augmented data. Our approach achieves consistent accuracy gain on a few standard image classification benchmarks, and the gain transfers to object detection.
How Much Do Unstated Problem Constraints Limit Deep Robotic Reinforcement Learning?
Venue: arXiv cs.RO 2019
Year: 2019

Deep Reinforcement Learning is a promising paradigm for robotic control which has been shown to be capable of learning policies for high-dimensional, continuous control of unmodeled systems. However, RoboticReinforcement Learning currently lacks clearly defined benchmark tasks, which makes it difficult for researchers to reproduce and compare against prior work. ``Reacher'' tasks, which are fundamental to robotic manipulation, are commonly used as benchmarks, but the lack of a formal specification elides details that are crucial to replication. In this paper we present a novel empirical analysis which shows that the unstated spatial constraints in commonly used implementations of Reacher tasks make it dramatically easier to learn a successful control policy with DeepDeterministic Policy Gradients (DDPG), a state-of-the-art Deep RL algorithm. Our analysis suggests that less constrained Reacher tasks are significantly more difficult to learn, and hence that existing de facto benchmarks are not representative of the difficulty of general robotic manipulation.
Analysis and Optimization of Successful Symbol Transmission Rate for Grant-free Massive Access with Massive MIMO
Venue: arXiv cs.IT 2019
Year: 2019

Grant-free massive access is an important technique for supporting massive machine-type communications (mMTC) for Internet-of-Things (IoT). Two important features in grant-free massive access are low-complexity devices and short-packet data transmission, making the traditional performance metric, achievable rate, unsuitable in this case. In this letter, we investigate grant-free massive access in a massive multiple-input multiple-output (MIMO) system. We consider random access control, and adopt approximate message passing (AMP) for user activity detection and channel estimation in the pilot transmission phase and small phase-shift-keying (PSK) modulation in the data transmission phase. We propose a more reasonable performance metric, namely successful symbol transmission rate (SSTR), for grant-free massive access. We obtain closed-form approximate expressions for the asymptotic SSTR in the cases of maximal ratio combining (MRC) and zero forcing (ZF) beamforming at the base station (BS), respectively. We also maximize the asymptotic SSTR with respect to the access parameter and pilot length.
BERT Meets Chinese Word Segmentation
Venue: arXiv cs.CL 2019
Year: 2019

Chinese word segmentation (CWS) is a fundamental task for Chinese language understanding. Recently, neural network-based models have attained superior performance in solving the in-domain CWS task. Last year, Bidirectional Encoder Representation from Transformers (BERT), a new language representation model, has been proposed as a backbone model for many natural language tasks and redefined the corresponding performance. The excellent performance of BERT motivates us to apply it to solve the CWS task. By conducting intensive experiments in the benchmark datasets from the second International Chinese Word Segmentation Bake-off, we obtain several keen observations. BERT can slightly improve the performance even when the datasets contain the issue of labeling inconsistency. When applying sufficiently learned features, Softmax, a simpler classifier, can attain the same performance as that of a more complicated classifier, e.g., Conditional Random Field (CRF). The performance of BERT usually increases as the model size increases. The features extracted by BERT can be also applied as good candidates for other neural network models.
Learning Lightweight Pedestrian Detector with Hierarchical Knowledge Distillation
Venue: arXiv cs.CV 2019
Year: 2019

It remains very challenging to build a pedestrian detection system for real world applications, which demand for both accuracy and speed. This work presents a novel hierarchical knowledge distillation framework to learn a lightweight pedestrian detector, which significantly reduces the computational cost and still holds the high accuracy at the same time. Following the `teacher--student' diagram that a stronger, deeper neural network can teach a lightweight network to learn better representations, we explore multiple knowledge distillation architectures and reframe this approach as a unified, hierarchical distillation framework. In particular, the proposed distillation is performed at multiple hierarchies, multiple stages in a modern detector, which empowers the student detector to learn both low-level details and high-level abstractions simultaneously. Experiment result shows that a student model trained by our framework, with 6 times compression in number of parameters, still achieves competitive performance as the teacher model on the widely used pedestrian detection benchmark.
A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
Venue: arXiv cs.LG 2019
Year: 2019

Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance. To mitigate this issue, we provide a simple but effective baseline method that is robust to noisy labels, even with severe noise. Our objective involves a variance regularization term that implicitly penalizes the Jacobian norm of the neural network on the whole training set (including the noisy-labeled data), which encourages generalization and prevents overfitting to the corrupted labels. Experiments on both synthetically generated incorrect labels and realistic large-scale noisy datasets demonstrate that our approach achieves state-of-the-art performance with a high tolerance to severe noise.
Corrigendum to: "Linear time algorithm to cover and hit a set of line segments optimally by two axis-parallel squares", Theoretical Computer Science 769 (2019) 63--74
Venue: arXiv cs.CG 2019
Year: 2019

In the paper "Linear time algorithm to cover and hit a set of line segments optimally by two axis-parallel squares", TCS Volume 769 (2019), pages 63--74, the LHIT problem is proposed as follows: For a given set of non-intersecting line segments ${\cal L} = \{\ell_1, \ell_2, \ldots, \ell_n\}$ in $I\!\!R^2$, compute two axis-parallel congruent squares ${\cal S}_1$ and ${\cal S}_2$ of minimum size whose union hits all the line segments in $\cal L$, and a linear time algorithm was proposed. Later it was observed that the algorithm has a bug. In this corrigendum, we corrected the algorithm. The time complexity of the corrected algorithm is $O(n^2)$.
MCTS-based Automated Negotiation Agent
Venue: arXiv cs.MA 2019
Year: 2019

This paper introduces a new negotiating agent model for automated negotiation. We focus on applications without time pressure with multidi-mensional negotiation on both continuous and discrete domains. The agent bidding strategy relies on Monte Carlo Tree Search, which is a trendy method since it has been used with success on games with high branching factor such as Go. It also exploits opponent modeling techniques thanks to Gaussian process regression and Bayesian learning. Evaluation is done by confronting the existing agents that are able to negotiate in such context: Random Walker, Tit-for-tat and Nice Tit-for-Tat. None of those agents succeeds in beating our agent. Also, the modular and adaptive nature of our approach is a huge advantage when it comes to optimize it in specific applicative contexts.
Multi-sense Definition Modeling using Word Sense Decompositions
Venue: arXiv cs.CL 2019
Year: 2019

Word embeddings capture syntactic and semantic information about words. Definition modeling aims to make the semantic content in each embedding explicit, by outputting a natural language definition based on the embedding. However, existing definition models are limited in their ability to generate accurate definitions for different senses of the same word. In this paper, we introduce a new method that enables definition modeling for multiple senses. We show how a Gumble-Softmax approach outperforms baselines at matching sense-specific embeddings to definitions during training. In experiments, our multi-sense definition model improves recall over a state-of-the-art single-sense definition model by a factor of three, without harming precision.
Fault-Diagnosing SLAM for Varying Scale Change Detection
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we present a new fault diagnosis (FD) -based approach for detection of imagery changes that can detect significant changes as inconsistencies between different sub-modules (e.g., self-localizaiton) of visual SLAM. Unlike classical change detection approaches such as pairwise image comparison (PC) and anomaly detection (AD), neither the memorization of each map image nor the maintenance of up-to-date place-specific anomaly detectors are required in this FD approach. A significant challenge that is encountered when incorporating different SLAM sub-modules into FD involves dealing with the varying scales of objects that have changed (e.g., the appearance of small dangerous obstacles on the floor). To address this issue, we reconsider the bag-of-words (BoW) image representation, by exploiting its recent advances in terms of self-localization and change detection. As a key advantage, BoW image representation can be reorganized into any different scaling by simply cropping the original BoW image. Furthermore, we propose to combine different self-localization modules with strong and weak BoW features with different discriminativity, and to treat inconsistency between strong and weak self-localization as an indicator of change. The efficacy of the proposed approach for FD with/without AD and/or PC was experimentally validated.
Mining Minimal Map-Segments for Visual Place Classifiers
Venue: arXiv cs.CV 2019
Year: 2019

In visual place recognition (VPR), map segmentation (MS) is a preprocessing technique used to partition a given view-sequence map into place classes (i.e., map segments) so that each class has good place-specific training images for a visual place classifier (VPC). Existing approaches to MS implicitly/explicitly suppose that map segments have a certain size, or individual map segments are balanced in size. However, recent VPR systems showed that very small important map segments (minimal map segments) often suffice for VPC, and the remaining large unimportant portion of the map should be discarded to minimize map maintenance cost. Here, a new MS algorithm that can mine minimal map segments from a large view-sequence map is presented. To solve the inherently NP hard problem, MS is formulated as a video-segmentation problem and the efficient point-trajectory based paradigm of video segmentation is used. The proposed map representation was implemented with three types of VPC: deep convolutional neural network, bag-of-words, and object class detector, and each was integrated into a Monte Carlo localization algorithm (MCL) within a topometric VPR framework. Experiments using the publicly available NCLT dataset thoroughly investigate the efficacy of MS in terms of VPR performance.
SANVis: Visual Analytics for Understanding Self-Attention Networks
Venue: arXiv cs.CL 2019
Year: 2019

Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org
Multiprocessor Real-Time Locking Protocols: A Systematic Review
Venue: arXiv cs.DC 2019
Year: 2019

We systematically survey the literature on analytically sound multiprocessor real-time locking protocols from 1988 until 2018, covering the following topics: progress mechanisms that prevent the lock-holder preemption problem, spin-lock protocols, binary semaphore protocols, independence-preserving (or fully preemptive) locking protocols, reader-writer and k-exclusion synchronization, support for nested critical sections, and implementation and system-integration aspects. A special focus is placed on the suspension-oblivious and suspension-aware analysis approaches for semaphore protocols, their respective notions of priority inversion, optimality criteria, lower bounds on maximum priority-inversion blocking, and matching asymptotically optimal locking protocols.
Building a Library of Tactile Skills Based on FingerVision
Venue: arXiv cs.RO 2019
Year: 2019

Camera-based tactile sensors are emerging as a promising inexpensive solution for tactile-enhanced manipulation tasks. A recently introduced FingerVision sensor was shown capable of generating reliable signals for force estimation, object pose estimation, and slip detection. In this paper, we build upon the FingerVision design, improving already existing control algorithms, and, more importantly, expanding its range of applicability to more challenging tasks by utilizing raw skin deformation data for control. In contrast to previous approaches that rely on the average deformation of the whole sensor surface, we directly employ local deviations of each spherical marker immersed in the silicone body of the sensor for feedback control and as input to learning tasks. We show that with such input, substances of varying texture and viscosity can be distinguished on the basis of tactile sensations evoked while stirring them. As another application, we learn a mapping between skin deformation and force applied to an object. To demonstrate the full range of capabilities of the proposed controllers, we deploy them in a challenging architectural assembly task that involves inserting a load-bearing element underneath a bendable plate at the point of maximum load.
Persian Signature Verification using Fully Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

Fully convolutional networks (FCNs) have been recently used for feature extraction and classification in image and speech recognition, where their inputs have been raw signal or other complicated features. Persian signature verification is done using conventional convolutional neural networks (CNNs). In this paper, we propose to use FCN for learning a robust feature extraction from the raw signature images. FCN can be considered as a variant of CNN where its fully connected layers are replaced with a global pooling layer. In the proposed manner, FCN inputs are raw signature images and convolution filter size is fixed. Recognition accuracy on UTSig database, shows that FCN with a global average pooling outperforms CNN.
Dependency-based Text Graphs for Keyphrase and Summary Extraction with Applications to Interactive Content Retrieval
Venue: arXiv cs.AI 2019
Year: 2019

We build a bridge between neural network-based machine learning and graph-based natural language processing and introduce a unified approach to keyphrase, summary and relation extraction by aggregating dependency graphs from links provided by a deep-learning based dependency parser. We reorganize dependency graphs to focus on the most relevant content elements of a sentence, integrate sentence identifiers as graph nodes and after ranking the graph, we extract our keyphrases and summaries from its largest strongly-connected component. We take advantage of the implicit structural information that dependency links bring to extract subject-verb-object, is-a and part-of relations. We put it all together into a proof-of-concept dialog engine that specializes the text graph with respect to a query and reveals interactively the document's most relevant content elements. The open-source code of the integrated system is available at https://github.com/ptarau/DeepRank . Keywords: graph-based natural language processing, dependency graphs, keyphrase, summary and relation extraction, query-driven salient sentence extraction, logic-based dialog engine, synergies between neural and symbolic processing.
Invasiveness Prediction of Pulmonary Adenocarcinomas Using Deep Feature Fusion Networks
Venue: arXiv cs.CV 2019
Year: 2019

Early diagnosis of pathological invasiveness of pulmonary adenocarcinomas using computed tomography (CT) imaging would alter the course of treatment of adenocarcinomas and subsequently improve the prognosis. Most of the existing systems use either conventional radiomics features or deep-learning features alone to predict the invasiveness. In this study, we explore the fusion of the two kinds of features and claim that radiomics features can be complementary to deep-learning features. An effective deep feature fusion network is proposed to exploit the complementarity between the two kinds of features, which improves the invasiveness prediction results. We collected a private dataset that contains lung CT scans of 676 patients categorized into four invasiveness types from a collaborating hospital. Evaluations on this dataset demonstrate the effectiveness of our proposal.
Low-Rank Approximation of Matrices for PMI-based Word Embeddings
Venue: arXiv cs.CL 2019
Year: 2019

We perform an empirical evaluation of several methods of low-rank approximation in the problem of obtaining PMI-based word embeddings. All word vectors were trained on parts of a large corpus extracted from English Wikipedia (enwik9) which was divided into two equal-sized datasets, from which PMI matrices were obtained. A repeated measures design was used in assigning a method of low-rank approximation (SVD, NMF, QR) and dimensionality of the vectors (250, 500) to each of the PMI matrix replicates. Our experiments show that word vectors obtained from the truncated SVD achieve the best performance on two downstream tasks, similarity and analogy, compare to the other two low-rank approximation methods.
Hybrid Beamformer Codebook Design and Ordering for Compressive mmWave Channel Estimation
Venue: arXiv cs.IT 2019
Year: 2019

In millimeter wave (mmWave) communication systems, beamforming with large antenna arrays is critical to overcome high path losses. Separating all-digital beamforming into analog and digital stages can provide the large reduction in power consumption and small loss in spectral efficiency needed for practical implementations. Developing algorithms with this favorable tradeoff is challenging due to the additional degrees of freedom in the analog stage and its accompanying hardware constraints. In hybrid beamforming systems, for example, channel estimation algorithms do not directly observe the channels, face a high channel count, and operate at low SNR before transmit-receive beam alignment. Since mmWave channels are sparse in time and beam domains, many compressed sensing (CS) channel estimation algorithms have been developed that randomly configure the analog beamformers, digital beamformers, and/or pilot symbols. In this paper, we propose to design deterministic beamformers and pilot symbols for open-loop channel estimation. We use CS approaches that rely on low coherence for their recovery guarantees, and hence seek to minimize the mutual coherence of the compressed sensing matrix. We also propose a precoder column ordering to design the pilot symbols. Simulation results show that our beamformer designs reduce channel estimation error over competing methods.
Deep Message Passing on Sets
Venue: arXiv cs.LG 2019
Year: 2019

Modern methods for learning over graph input data have shown the fruitfulness of accounting for relationships among elements in a collection. However, most methods that learn over set input data use only rudimentary approaches to exploit intra-collection relationships. In this work we introduce Deep Message Passing on Sets (DMPS), a novel method that incorporates relational learning for sets. DMPS not only connects learning on graphs with learning on sets via deep kernel learning, but it also bridges message passing on sets and traditional diffusion dynamics commonly used in denoising models. Based on these connections, we develop two new blocks for relational learning on sets: the set-denoising block and the set-residual block. The former is motivated by the connection between message passing on general graphs and diffusion-based denoising models, whereas the latter is inspired by the well-known residual network. In addition to demonstrating the interpretability of our model by learning the true underlying relational structure experimentally, we also show the effectiveness of our approach on both synthetic and real-world datasets by achieving results that are competitive with or outperform the state-of-the-art.
Levantamento de Requisitos para Jogos Educativos Infantis
Venue: arXiv cs.HC 2019
Year: 2019

The use of games within the classroom is in constant growing, however, there are still demands for better solutions to problems such as reconciling leisure with class or the lack of attention of children. This work demonstrates methodologies used to survey and analyze requirements for the development of children's educational games, with the target audience of children from 5 to 9 years old. A list of basic requirements that can improve the development of educational games and provide a basis for further development of a game is raised. It is possible to continue the work by addressing theories of education, the process of gamification of education and other requirements gathering methods.
Stochastic model of business process decomposition
Venue: arXiv cs.OH 2019
Year: 2019

Decomposition is the basis of works dedicated to business process modelling at the stage of information and management systems analysis and design. The article shows that the business process decomposition can be represented as a Galton Watson branching stochastic process. This representation allows estimating the decomposition tree depth and the total amount of its elements, as well as explaining the empirical requirement for the business function decomposition (not more than 7 elements). The problem is deemed relevant as the obtained results allow objectively estimating the labor input in business process modelling.