OpenRaiser/Intern-Atlas huggingface dataset:
A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects
Venue: arXiv cs.RO 2020
Year: 2020

We present a framework for solving long-horizon planning problems involving manipulation of rigid objects that operates directly from a point-cloud observation, i.e. without prior object models. Our method plans in the space of object subgoals and frees the planner from reasoning about robot-object interaction dynamics by relying on a set of generalizable manipulation primitives. We show that for rigid bodies, this abstraction can be realized using low-level manipulation skills that maintain sticking contact with the object and represent subgoals as 3D transformations. To enable generalization to unseen objects and improve planning performance, we propose a novel way of representing subgoals for rigid-body manipulation and a graph-attention based neural network architecture for processing point-cloud inputs. We experimentally validate these choices using simulated and real-world experiments on the YuMi robot. Results demonstrate that our method can successfully manipulate new objects into target configurations requiring long-term planning. Overall, our framework realizes the best of the worlds of task-and-motion planning (TAMP) and learning-based approaches. Project website: https://anthonysimeonov.github.io/rpo-planning-framework/.
Obstacle avoidance-driven controller for safety-critical aerial robots
Venue: arXiv cs.RO 2020
Year: 2020

The goal of this thesis is to propose the combination of Control-Barrier-Functions (CBF) with Model-Predictive-Control (MPC) resulting in the novel Model-Predictive-Control-Barrier-Function (MPCBF). It can be shown, that the performance of the MPCBF surpasses the performance of the CBF due to the increased time horizon of the MPC. Moreover, the MPCBF was applied to a quadrotor, a system strongly in need of fast and predictive control. Using the MPCBF, the quadrotor was able to avoid obstacles, which the CBF failed to avoid due to the relative speed of the obstacle. The results of this work are experimentally validated.
Higher order hesitant fuzzy Choquet integral operator and its application to multiple criteria decision making
Venue: arXiv cs.AI 2020
Year: 2020

Generally, the criteria involved in a decision making problem are interactive or inter-dependent, and therefore aggregating them by the use of traditional operators which are based on additive measures is not logical. This verifies that we have to implement fuzzy measures for modelling the interaction phenomena among the criteria.On the other hand, based on the recent extension of hesitant fuzzy set, called higher order hesitant fuzzy set (HOHFS) which allows the membership of a given element to be defined in forms of several possible generalized types of fuzzy set, we encourage to propose the higher order hesitant fuzzy (HOHF) Choquet integral operator. This concept not only considers the importance of the higher order hesitant fuzzy arguments, but also it can reflect the correlations among those arguments. Then,a detailed discussion on the aggregation properties of the HOHF Choquet integral operator will be presented.To enhance the application of HOHF Choquet integral operator in decision making, we first assess the appropriate energy policy for the socio-economic development. Then, the efficiency of the proposed HOHF Choquet integral operator-based technique over a number of exiting techniques is further verified by employing another decision making problem associated with the technique of TODIM (an acronym in Portuguese of Interactive and Multicriteria Decision Making).
End-to-end spoken language understanding using transformer networks and self-supervised pre-trained features
Venue: arXiv cs.CL 2020
Year: 2020

Transformer networks and self-supervised pre-training have consistently delivered state-of-art results in the field of natural language processing (NLP); however, their merits in the field of spoken language understanding (SLU) still need further investigation. In this paper we introduce a modular End-to-End (E2E) SLU transformer network based architecture which allows the use of self-supervised pre-trained acoustic features, pre-trained model initialization and multi-task training. Several SLU experiments for predicting intent and entity labels/values using the ATIS dataset are performed. These experiments investigate the interaction of pre-trained model initialization and multi-task training with either traditional filterbank or self-supervised pre-trained acoustic features. Results show not only that self-supervised pre-trained acoustic features outperform filterbank features in almost all the experiments, but also that when these features are used in combination with multi-task training, they almost eliminate the necessity of pre-trained model initialization.
Avoiding Communication in Logistic Regression
Venue: arXiv cs.LG 2020
Year: 2020

Stochastic gradient descent (SGD) is one of the most widely used optimization methods for solving various machine learning problems. SGD solves an optimization problem by iteratively sampling a few data points from the input data, computing gradients for the selected data points, and updating the solution. However, in a parallel setting, SGD requires interprocess communication at every iteration. We introduce a new communication-avoiding technique for solving the logistic regression problem using SGD. This technique re-organizes the SGD computations into a form that communicates every $s$ iterations instead of every iteration, where $s$ is a tuning parameter. We prove theoretical flops, bandwidth, and latency upper bounds for SGD and its new communication-avoiding variant. Furthermore, we show experimental results that illustrate that the new Communication-Avoiding SGD (CA-SGD) method can achieve speedups of up to $4.97\times$ on a high-performance Infiniband cluster without altering the convergence behavior or accuracy.
Facebook AI's WMT20 News Translation Task Submission
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes Facebook AI's submission to WMT20 shared news translation task. We focus on the low resource setting and participate in two language pairs, Tamil <-> English and Inuktitut <-> English, where there are limited out-of-domain bitext and monolingual data. We approach the low resource problem using two main strategies, leveraging all available data and adapting the system to the target news domain. We explore techniques that leverage bitext and monolingual data from all languages, such as self-supervised model pretraining, multilingual models, data augmentation, and reranking. To better adapt the translation system to the test domain, we explore dataset tagging and fine-tuning on in-domain data. We observe that different techniques provide varied improvements based on the available data of the language pair. Based on the finding, we integrate these techniques into one training pipeline. For En->Ta, we explore an unconstrained setup with additional Tamil bitext and monolingual data and show that further improvement can be obtained. On the test set, our best submitted systems achieve 21.5 and 13.7 BLEU for Ta->En and En->Ta respectively, and 27.9 and 13.0 for Iu->En and En->Iu respectively.
Feature Sharing and Integration for Cooperative Cognition and Perception with Volumetric Sensors
Venue: arXiv cs.CV 2020
Year: 2020

The recent advancement in computational and communication systems has led to the introduction of high-performing neural networks and high-speed wireless vehicular communication networks. As a result, new technologies such as cooperative perception and cognition have emerged, addressing the inherent limitations of sensory devices by providing solutions for the detection of partially occluded targets and expanding the sensing range. However, designing a reliable cooperative cognition or perception system requires addressing the challenges caused by limited network resources and discrepancies between the data shared by different sources. In this paper, we examine the requirements, limitations, and performance of different cooperative perception techniques, and present an in-depth analysis of the notion of Deep Feature Sharing (DFS). We explore different cooperative object detection designs and evaluate their performance in terms of average precision. We use the Volony dataset for our experimental study. The results confirm that the DFS methods are significantly less sensitive to the localization error caused by GPS noise. Furthermore, the results attest that detection gain of DFS methods caused by adding more cooperative participants in the scenes is comparable to raw information sharing technique while DFS enables flexibility in design toward satisfying communication requirements.
Uniform Bipartition in the Population Protocol Model with Arbitrary Communication Graphs
Venue: arXiv cs.DC 2020
Year: 2020

In this paper, we focus on the uniform bipartition problem in the population protocol model. This problem aims to divide a population into two groups of equal size. In particular, we consider the problem in the context of \emph{arbitrary} communication graphs. As a result, we clarify the solvability of the uniform bipartition problem with arbitrary communication graphs when agents in the population have designated initial states, under various assumptions such as the existence of a base station, symmetry of the protocol, and fairness of the execution. When the problem is solvable, we present protocols for uniform bipartition. When global fairness is assumed, the space complexity of our solutions is tight.
Data-aided Sensing for Distributed Detection
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we study data-aided sensing (DAS) for distributed detection in wireless sensor networks (WSNs) when sensors' measurements are correlated. In particular, we derive a node selection criterion based on the J-divergence in DAS for reliable decision subject to a decision delay constraint. Based on the proposed J-divergence based DAS, the nodes can be selected to rapidly increase the log-likelihood ratio (LLR), which leads to a reliable decision with a smaller number of the sensors that upload measurements for a shorter decision delay. From simulation results, it is confirmed that the J-divergence based DAS can provide a reliable decision with a smaller number of sensors compared to other approaches.
Deep Learning Framework From Scratch Using Numpy
Venue: arXiv cs.MS 2020
Year: 2020

This work is a rigorous development of a complete and general-purpose deep learning framework from the ground up. The fundamental components of deep learning - automatic differentiation and gradient methods of optimizing multivariable scalar functions - are developed from elementary calculus and implemented in a sensible object-oriented approach using only Python and the Numpy library. Demonstrations of solved problems using the framework, named ArrayFlow, include a computer vision classification task, solving for the shape of a catenary, and a 2nd order differential equation.
Cascade RNN-Transducer: Syllable Based Streaming On-device Mandarin Speech Recognition with a Syllable-to-Character Converter
Venue: arXiv cs.SD 2020
Year: 2020

End-to-end models are favored in automatic speech recognition (ASR) because of its simplified system structure and superior performance. Among these models, recurrent neural network transducer (RNN-T) has achieved significant progress in streaming on-device speech recognition because of its high-accuracy and low-latency. RNN-T adopts a prediction network to enhance language information, but its language modeling ability is limited because it still needs paired speech-text data to train. Further strengthening the language modeling ability through extra text data, such as shallow fusion with an external language model, only brings a small performance gain. In view of the fact that Mandarin Chinese is a character-based language and each character is pronounced as a tonal syllable, this paper proposes a novel cascade RNN-T approach to improve the language modeling ability of RNN-T. Our approach firstly uses an RNN-T to transform acoustic feature into syllable sequence, and then converts the syllable sequence into character sequence through an RNN-T-based syllable-to-character converter. Thus a rich text repository can be easily used to strengthen the language model ability. By introducing several important tricks, the cascade RNN-T approach surpasses the character-based RNN-T by a large margin on several Mandarin test sets, with much higher recognition quality and similar latency.
Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis
Venue: arXiv cs.SD 2020
Year: 2020

This paper proposes a unified model to conduct emotion transfer, control and prediction for sequence-to-sequence based fine-grained emotional speech synthesis. Conventional emotional speech synthesis often needs manual labels or reference audio to determine the emotional expressions of synthesized speech. Such coarse labels cannot control the details of speech emotion, often resulting in an averaged emotion expression delivery, and it is also hard to choose suitable reference audio during inference. To conduct fine-grained emotion expression generation, we introduce phoneme-level emotion strength representations through a learned ranking function to describe the local emotion details, and the sentence-level emotion category is adopted to render the global emotions of synthesized speech. With the global render and local descriptors of emotions, we can obtain fine-grained emotion expressions from reference audio via its emotion descriptors (for transfer) or directly from phoneme-level manual labels (for control). As for the emotional speech synthesis with arbitrary text inputs, the proposed model can also predict phoneme-level emotion expressions from texts, which does not require any reference audio or manual label.
A Divide et Impera Approach for 3D Shape Reconstruction from Multiple Views
Venue: arXiv cs.CV 2020
Year: 2020

Estimating the 3D shape of an object from a single or multiple images has gained popularity thanks to the recent breakthroughs powered by deep learning. Most approaches regress the full object shape in a canonical pose, possibly extrapolating the occluded parts based on the learned priors. However, their viewpoint invariant technique often discards the unique structures visible from the input images. In contrast, this paper proposes to rely on viewpoint variant reconstructions by merging the visible information from the given views. Our approach is divided into three steps. Starting from the sparse views of the object, we first align them into a common coordinate system by estimating the relative pose between all the pairs. Then, inspired by the traditional voxel carving, we generate an occupancy grid of the object taken from the silhouette on the images and their relative poses. Finally, we refine the initial reconstruction to build a clean 3D model which preserves the details from each viewpoint. To validate the proposed method, we perform a comprehensive evaluation on the ShapeNet reference benchmark in terms of relative pose estimation and 3D shape reconstruction.
Audience Creation for Consumables -- Simple and Scalable Precision Merchandising for a Growing Marketplace
Venue: arXiv cs.LG 2020
Year: 2020

Consumable categories, such as grocery and fast-moving consumer goods, are quintessential to the growth of e-commerce marketplaces in developing countries. In this work, we present the design and implementation of a precision merchandising system, which creates audience sets from over 10 million consumers and is deployed at Flipkart Supermart, one of the largest online grocery stores in India. We employ temporal point process to model the latent periodicity and mutual-excitation in the purchase dynamics of consumables. Further, we develop a likelihood-free estimation procedure that is robust against data sparsity, censure and noise typical of a growing marketplace. Lastly, we scale the inference by quantizing the triggering kernels and exploiting sparse matrix-vector multiplication primitive available on a commercial distributed linear algebra backend. In operation spanning more than a year, we have witnessed a consistent increase in click-through rate in the range of 25-70% for banner-based merchandising in the storefront, and in the range of 12-26% for push notification-based campaigns.
Feasibility Study on CCTV-aware Routing and Navigation for Privacy, Anonymity, and Safety. Jyvaskyla -- Case-study of the First City to Benefit from CCTV-aware Technology. (Preprint)
Venue: arXiv cs.CR 2020
Year: 2020

In order to withstand the ever-increasing invasion of privacy by CCTV cameras and technologies, on par CCTV-aware solutions must exist that provide privacy, safety, and cybersecurity features. We argue that a first important step towards such CCTV-aware solutions must be a mapping system that provides both privacy and safety routing and navigation options. To the best of our knowledge, there are no mapping nor navigation systems that support privacy and safety routing options. In this paper, we explore the feasibility of a CCTV-aware routing and navigation solution. The aim of this feasibility exploration is to understand what are the main impacts of CCTV on privacy, and what are the challenges and benefits to building such technology. We evaluate our approach on seven (7) pedestrian walking routes within the downtown area of the city of Jyvaskyla, Finland. We first map a total of 450 CCTV cameras, and then experiment with routing and navigation under several different configurations to coarsely model the possible cameras' parameters and coverage from the real-world. We report two main results. First, our preliminary findings support the overall feasibility of our approach. Second, the results also reveal a data-driven worrying reality for persons wishing to preserve their privacy/anonymity as their main living choice. When modelling cameras at their low performance end, a privacy-preserving route has on average a 1.5x distance increase when compared to generic routing. When modelling cameras at their medium-to-high performance end, a privacy-preserving route has on average a 5.0x distance increase, while in some cases there are no privacy-preserving routes possible at all. These results further support and encourage both global mapping of CCTV cameras and refinements to camera modelling and underlying technology.
Accent and Speaker Disentanglement in Many-to-many Voice Conversion
Venue: arXiv cs.SD 2020
Year: 2020

This paper proposes an interesting voice and accent joint conversion approach, which can convert an arbitrary source speaker's voice to a target speaker with non-native accent. This problem is challenging as each target speaker only has training data in native accent and we need to disentangle accent and speaker information in the conversion model training and re-combine them in the conversion stage. In our recognition-synthesis conversion framework, we manage to solve this problem by two proposed tricks. First, we use accent-dependent speech recognizers to obtain bottleneck features for different accented speakers. This aims to wipe out other factors beyond the linguistic information in the BN features for conversion model training. Second, we propose to use adversarial training to better disentangle the speaker and accent information in our encoder-decoder based conversion model. Specifically, we plug an auxiliary speaker classifier to the encoder, trained with an adversarial loss to wipe out speaker information from the encoder output. Experiments show that our approach is superior to the baseline. The proposed tricks are quite effective in improving accentedness and audio quality and speaker similarity are well maintained.
Can Semantic Labels Assist Self-Supervised Visual Representation Learning?
Venue: arXiv cs.CV 2020
Year: 2020

Recently, contrastive learning has largely advanced the progress of unsupervised visual representation learning. Pre-trained on ImageNet, some self-supervised algorithms reported higher transfer learning performance compared to fully-supervised methods, seeming to deliver the message that human labels hardly contribute to learning transferrable visual features. In this paper, we defend the usefulness of semantic labels but point out that fully-supervised and self-supervised methods are pursuing different kinds of features. To alleviate this issue, we present a new algorithm named Supervised Contrastive Adjustment in Neighborhood (SCAN) that maximally prevents the semantic guidance from damaging the appearance feature embedding. In a series of downstream tasks, SCAN achieves superior performance compared to previous fully-supervised and self-supervised methods, and sometimes the gain is significant. More importantly, our study reveals that semantic labels are useful in assisting self-supervised methods, opening a new direction for the community.
Topology-Based Feature Design and Tracking for Multi-Center Cyclones
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a concept to design, track, and compare application-specific feature definitions expressed as sets of critical points. Our work has been inspired by the observation that in many applications a large variety of different feature definitions for the same concept are used. Often, these definitions compete with each other and it is unclear which definition should be used in which context. A prominent example is the definition of cyclones in climate research. Despite the differences, frequently these feature definitions can be related to topological concepts. In our approach, we provide a cyclone tracking framework that supports interactive feature definition and comparison based on a precomputed tracking graph that stores all extremal points as well as their temporal correspondents. The framework combines a set of independent building blocks: critical point extraction, critical point tracking, feature definition, and track exploration. One of the major advantages of such an approach is the flexibility it provides, that is, each block is exchangeable. Moreover, it also enables us to perform the most expensive analysis, the construction of a full tracking graph, as a prepossessing step, while keeping the feature definition interactive. Different feature definitions can be explored and compared interactively based on this tracking graph. Features are specified by rules for grouping critical points, while feature tracking corresponds to filtering and querying the full tracking graph by specific requests. We demonstrate this method for cyclone identification and tracking in the context of climate research.
Pyramid Point: A Multi-Level Focusing Network for Revisiting Feature Layers
Venue: arXiv cs.CV 2020
Year: 2020

We present a method to learn a diverse group of object categories from an unordered point set. We propose our Pyramid Point network, which uses a dense pyramid structure instead of the traditional 'U' shape, typically seen in semantic segmentation networks. This pyramid structure gives a second look, allowing the network to revisit different layers simultaneously, increasing the contextual information by creating additional layers with less noise. We introduce a Focused Kernel Point convolution (FKP Conv), which expands on the traditional point convolutions by adding an attention mechanism to the kernel outputs. This FKP Conv increases our feature quality and allows us to weigh the kernel outputs dynamically. These FKP Convs are the central part of our Recurrent FKP Bottleneck block, which makes up the backbone of our encoder. With this distinct network, we demonstrate competitive performance on three benchmark data sets. We also perform an ablation study to show the positive effects of each element in our FKP Conv.