OpenRaiser/Intern-Atlas huggingface dataset:
DropClass and DropAdapt: Dropping classes for deep speaker representation learning
Venue: arXiv cs.SD 2020
Year: 2020

Many recent works on deep speaker embeddings train their feature extraction networks on large classification tasks, distinguishing between all speakers in a training set. Empirically, this has been shown to produce speaker-discriminative embeddings, even for unseen speakers. However, it is not clear that this is the optimal means of training embeddings that generalize well. This work proposes two approaches to learning embeddings, based on the notion of dropping classes during training. We demonstrate that both approaches can yield performance gains in speaker verification tasks. The first proposed method, DropClass, works via periodically dropping a random subset of classes from the training data and the output layer throughout training, resulting in a feature extractor trained on many different classification tasks. Combined with an additive angular margin loss, this method can yield a 7.9% relative improvement in equal error rate (EER) over a strong baseline on VoxCeleb. The second proposed method, DropAdapt, is a means of adapting a trained model to a set of enrolment speakers in an unsupervised manner. This is performed by fine-tuning a model on only those classes which produce high probability predictions when the enrolment speakers are used as input, again also dropping the relevant rows from the output layer. This method yields a large 13.2% relative improvement in EER on VoxCeleb. The code for this paper has been made publicly available.
Effect of Analysis Window and Feature Selection on Classification of Hand Movements Using EMG Signal
Venue: arXiv cs.CV 2020
Year: 2020

Electromyography (EMG) signals have been successfully employed for driving prosthetic limbs of a single or double degree of freedom. This principle works by using the amplitude of the EMG signals to decide between one or two simpler movements. This method underperforms as compare to the contemporary advances done at the mechanical, electronics, and robotics end, and it lacks intuition. Recently, research on myoelectric control based on pattern recognition (PR) shows promising results with the aid of machine learning classifiers. Using the approach termed as, EMG-PR, EMG signals are divided into analysis windows, and features are extracted for each window. These features are then fed to the machine learning classifiers as input. By offering multiple class movements and intuitive control, this method has the potential to power an amputated subject to perform everyday life movements. In this paper, we investigate the effect of the analysis window and feature selection on classification accuracy of different hand and wrist movements using time-domain features. We show that effective data preprocessing and optimum feature selection helps to improve the classification accuracy of hand movements. We use publicly available hand and wrist gesture dataset of $40$ intact subjects for experimentation. Results computed using different classification algorithms show that the proposed preprocessing and features selection outperforms the baseline and achieve up to $98\%$ classification accuracy.
Novelty Detection via Non-Adversarial Generative Network
Venue: arXiv cs.CV 2020
Year: 2020

One-class novelty detection is the process of determining if a query example differs from the training examples (the target class). Most of previous strategies attempt to learn the real characteristics of target sample by using generative adversarial networks (GANs) methods. However, the training process of GANs remains challenging, suffering from instability issues such as mode collapse and vanishing gradients. In this paper, by adopting non-adversarial generative networks, a novel decoder-encoder framework is proposed for novelty detection task, insteading of classical encoder-decoder style. Under the non-adversarial framework, both latent space and image reconstruction space are jointly optimized, leading to a more stable training process with super fast convergence and lower training losses. During inference, inspired by cycleGAN, we design a new testing scheme to conduct image reconstruction, which is the reverse way of training sequence. Experiments show that our model has the clear superiority over cutting-edge novelty detectors and achieves the state-of-the-art results on the datasets.
Study of Cloud-Aided Multi-Way Multiple-Antenna Relaying with Best-User Link Selection and Joint ML Detection
Venue: arXiv cs.IT 2020
Year: 2020

In this work, we present a cloud-aided uplink framework for multi-way multiple-antenna relay systems which facilitates joint linear Maximum Likelihood (ML) symbol detection in the cloud and where users are selected to simultaneously transmit to each other aided by relays. We also investigate relay selection techniques for the proposed cloud-aided uplink framework that uses cloud-based buffers and physical-layer network coding. In particular, we develop a novel multi-way relay selection protocol based on the selection of the best link, denoted as Multi-Way Cloud-Aided Best-User-Link (MWC-Best-User-Link). We then devise the maximum minimum distance relay selection criterion along with the algorithm that is incorporated into the proposed MWC-Best-User-Link protocol. Simulations show that MWC-Best-User-Link outperforms previous works in terms of average delay, sum-rate and bit error rate.
Evolving Neural Networks through a Reverse Encoding Tree
Venue: arXiv cs.NE 2020
Year: 2020

NeuroEvolution is one of the most competitive evolutionary learning frameworks for designing novel neural networks for use in specific tasks, such as logic circuit design and digital gaming. However, the application of benchmark methods such as the NeuroEvolution of Augmenting Topologies (NEAT) remains a challenge, in terms of their computational cost and search time inefficiency. This paper advances a method which incorporates a type of topological edge coding, named Reverse Encoding Tree (RET), for evolving scalable neural networks efficiently. Using RET, two types of approaches -- NEAT with Binary search encoding (Bi-NEAT) and NEAT with Golden-Section search encoding (GS-NEAT) -- have been designed to solve problems in benchmark continuous learning environments such as logic gates, Cartpole, and Lunar Lander, and tested against classical NEAT and FS-NEAT as baselines. Additionally, we conduct a robustness test to evaluate the resilience of the proposed NEAT algorithms. The results show that the two proposed strategies deliver improved performance, characterized by (1) a higher accumulated reward within a finite number of time steps; (2) using fewer episodes to solve problems in targeted environments, and (3) maintaining adaptive robustness under noisy perturbations, which outperform the baselines in all tested cases. Our analysis also demonstrates that RET expends potential future research directions in dynamic environments. Code is available from https://github.com/HaolingZHANG/ReverseEncodingTree.
Revisiting Meta-Learning as Supervised Learning
Venue: arXiv cs.LG 2020
Year: 2020

Recent years have witnessed an abundance of new publications and approaches on meta-learning. This community-wide enthusiasm has sparked great insights but has also created a plethora of seemingly different frameworks, which can be hard to compare and evaluate. In this paper, we aim to provide a principled, unifying framework by revisiting and strengthening the connection between meta-learning and traditional supervised learning. By treating pairs of task-specific data sets and target models as (feature, label) samples, we can reduce many meta-learning algorithms to instances of supervised learning. This view not only unifies meta-learning into an intuitive and practical framework but also allows us to transfer insights from supervised learning directly to improve meta-learning. For example, we obtain a better understanding of generalization properties, and we can readily transfer well-understood techniques, such as model ensemble, pre-training, joint training, data augmentation, and even nearest neighbor based methods. We provide an intuitive analogy of these methods in the context of meta-learning and show that they give rise to significant improvements in model performance on few-shot learning.
Parameterized Algorithms for Matrix Completion With Radius Constraints
Venue: arXiv cs.DM 2020
Year: 2020

Considering matrices with missing entries, we study NP-hard matrix completion problems where the resulting completed matrix shall have limited (local) radius. In the pure radius version, this means that the goal is to fill in the entries such that there exists a 'center string' which has Hamming distance to all matrix rows as small as possible. In stringology, this problem is also known as Closest String with Wildcards. In the local radius version, the requested center string must be one of the rows of the completed matrix. Hermelin and Rozenberg [CPM 2014, TCS 2016] performed parameterized complexity studies for Closest String with Wildcards. We answer one of their open questions, fix a bug concerning a fixed-parameter tractability result in their work, and improve some upper running time bounds. For the local radius case, we reveal a computational complexity dichotomy. In general, our results indicate that, although being NP-hard as well, this variant often allows for faster (fixed-parameter) algorithms.
Single-Stage Object Detection from Top-View Grid Maps on Custom Sensor Setups
Venue: arXiv cs.CV 2020
Year: 2020

We present our approach to unsupervised domain adaptation for single-stage object detectors on top-view grid maps in automated driving scenarios. Our goal is to train a robust object detector on grid maps generated from custom sensor data and setups. We first introduce a single-stage object detector for grid maps based on RetinaNet. We then extend our model by image- and instance-level domain classifiers at different feature pyramid levels which are trained in an adversarial manner. This allows us to train robust object detectors for unlabeled domains. We evaluate our approach quantitatively on the nuScenes and KITTI benchmarks and present qualitative domain adaptation results for unlabeled measurements recorded by our experimental vehicle. Our results demonstrate that object detection accuracy for unlabeled domains can be improved by applying our domain adaptation strategy.
A Novel Incremental Cross-Modal Hashing Approach
Venue: arXiv cs.CV 2020
Year: 2020

Cross-modal retrieval deals with retrieving relevant items from one modality, when provided with a search query from another modality. Hashing techniques, where the data is represented as binary bits have specifically gained importance due to the ease of storage, fast computations and high accuracy. In real world, the number of data categories is continuously increasing, which requires algorithms capable of handling this dynamic scenario. In this work, we propose a novel incremental cross-modal hashing algorithm termed "iCMH", which can adapt itself to handle incoming data of new categories. The proposed approach consists of two sequential stages, namely, learning the hash codes and training the hash functions. At every stage, a small amount of old category data termed "exemplars" is is used so as not to forget the old data while trying to learn for the new incoming data, i.e. to avoid catastrophic forgetting. In the first stage, the hash codes for the exemplars is used, and simultaneously, hash codes for the new data is computed such that it maintains the semantic relations with the existing data. For the second stage, we propose both a non-deep and deep architectures to learn the hash functions effectively. Extensive experiments across a variety of cross-modal datasets and comparisons with state-of-the-art cross-modal algorithms shows the usefulness of our approach.
Traduction des Grammaires Catégorielles de Lambek dans les Grammaires Catégorielles Abstraites
Venue: arXiv cs.CL 2020
Year: 2020

Lambek Grammars (LG) are a computational modelling of natural language, based on non-commutative compositional types. It has been widely studied, especially for languages where the syntax plays a major role (like English). The goal of this internship report is to demonstrate that every Lambek Grammar can be, not entirely but efficiently, expressed in Abstract Categorial Grammars (ACG). The latter is a novel modelling based on higher-order signature homomorphisms (using $λ$-calculus), aiming at uniting the currently used models. The main idea is to transform the type rewriting system of LGs into that of Context-Free Grammars (CFG) by erasing introduction and elimination rules and generating enough axioms so that the cut rule suffices. This iterative approach preserves the derivations and enables us to stop the possible infinite generative process at any step. Although the underlying algorithm was not fully implemented, this proof provides another argument in favour of the relevance of ACGs in Natural Language Processing.
Structural-Aware Sentence Similarity with Recursive Optimal Transport
Venue: arXiv cs.CL 2020
Year: 2020

Measuring sentence similarity is a classic topic in natural language processing. Light-weighted similarities are still of particular practical significance even when deep learning models have succeeded in many other tasks. Some light-weighted similarities with more theoretical insights have been demonstrated to be even stronger than supervised deep learning approaches. However, the successful light-weighted models such as Word Mover's Distance [Kusner et al., 2015] or Smooth Inverse Frequency [Arora et al., 2017] failed to detect the difference from the structure of sentences, i.e. order of words. To address this issue, we present Recursive Optimal Transport (ROT) framework to incorporate the structural information with the classic OT. Moreover, we further develop Recursive Optimal Similarity (ROTS) for sentences with the valuable semantic insights from the connections between cosine similarity of weighted average of word vectors and optimal transport. ROTS is structural-aware and with low time complexity compared to optimal transport. Our experiments over 20 sentence textural similarity (STS) datasets show the clear advantage of ROTS over all weakly supervised approaches. Detailed ablation study demonstrate the effectiveness of ROT and the semantic insights.
Comparison Between Traditional Machine Learning Models And Neural Network Models For Vietnamese Hate Speech Detection
Venue: arXiv cs.CL 2020
Year: 2020

Hate-speech detection on social network language has become one of the main researching fields recently due to the spreading of social networks like Facebook and Twitter. In Vietnam, the threat of offensive and harassment cause bad impacts for online user. The VLSP - Shared task about Hate Speech Detection on social networks showed many proposed approaches for detecting whatever comment is clean or not. However, this problem still needs further researching. Consequently, we compare traditional machine learning and deep learning on a large dataset about the user's comments on social network in Vietnamese and find out what is the advantage and disadvantage of each model by comparing their accuracy on F1-score, then we pick two models in which has highest accuracy in traditional machine learning models and deep neural models respectively. Next, we compare these two models capable of predicting the right label by referencing their confusion matrices and considering the advantages and disadvantages of each model. Finally, from the comparison result, we propose our ensemble method that concentrates the abilities of traditional methods and deep learning methods.
Project CLAI: Instrumenting the Command Line as a New Environment for AI Agents
Venue: arXiv cs.HC 2020
Year: 2020

This whitepaper reports on Project CLAI (Command Line AI), which aims to bring the power of AI to the command line interface (CLI). The CLAI platform sets up the CLI as a new environment for AI researchers to conquer by surfacing the command line as a generic environment that researchers can interface to using a simple sense-act API, much like the traditional AI agent architecture. In this paper, we discuss the design and implementation of the platform in detail, through illustrative use cases of new end user interaction patterns enabled by this design, and through quantitative evaluation of the system footprint of a CLAI-enabled terminal. We also report on some early user feedback on CLAI's features from an internal survey.
Towards an Operational Definition of Group Network Codes
Venue: arXiv cs.IT 2020
Year: 2020

Group network codes are a generalization of linear codes that have seen several studies over the last decade. When studying network codes, operations performed at internal network nodes called local encoding functions, are of significant interest. While local encoding functions of linear codes are well understood (and of operational significance), no similar operational definition exists for group network codes. To bridge this gap, we study the connections between group network codes and a family of codes called Coordinate-Wise-Linear (CWL) codes. CWL codes generalize linear codes and, in addition, can be defined locally (i.e., operationally). In this work, we study the connection between CWL codes and group codes from both a local and global encoding perspective. We show that Abelian group codes can be expressed as CWL codes and, as a result, they inherit an operational definition.
cube2net: Efficient Query-Specific Network Construction with Data Cube Organization
Venue: arXiv cs.SI 2020
Year: 2020

Networks are widely used to model objects with interactions and have enabled various downstream applications. However, in the real world, network mining is often done on particular query sets of objects, which does not require the construction and computation of networks including all objects in the datasets. In this work, for the first time, we propose to address the problem of query-specific network construction, to break the efficiency bottlenecks of existing network mining algorithms and facilitate various downstream tasks. To deal with real-world massive networks with complex attributes, we propose to leverage the well-developed data cube technology to organize network objects w.r.t. their essential attributes. An efficient reinforcement learning algorithm is then developed to automatically explore the data cube structures and construct the optimal query-specific networks. With extensive experiments of two classic network mining tasks on different real-world large datasets, we show that our proposed cube2net pipeline is general, and much more effective and efficient in query-specific network construction, compared with other methods without the leverage of data cube or reinforcement learning.
On the Submodularity of Diffusion Models: Equivalent Conditions and Applications
Venue: arXiv cs.SI 2020
Year: 2020

The diffusion model has been a crucial component in studies about social networks. Many studies, especially these about influence maximization concern the proof of the submodularity of particular diffusion models. Such proofs have been model-dependent and are somewhat ad hoc. In this paper, we prove a theorem that provides a necessary and sufficient condition for a diffusion model to be submodular. This theorem can be used to justify the submodularity of an arbitrary diffusion model. We also apply this theorem to build a projection operator that maps an arbitrary diffusion model into a submodular one. Moreover, we use the established theorem to propose a diffusion model of multiple heterogeneous pieces of information that partially features submodularity.
Quid Pro Quo allocations in Production-Inventory games
Venue: arXiv cs.GT 2020
Year: 2020

The concept of Owen point, introduced in Guardiola et al. (2009), is an appealing solution concept that for Production-Inventory games (PI-games) always belongs to their core. The Owen point allows all the players in the game to operate at minimum cost but it does not take into account the cost reduction induced by essential players over their followers (fans). Thus, it may be seen as an altruistic allocation for essential players what can be criticized. The aim this paper is two-fold: to study the structure and complexity of the core of PI-games and to introduce new core allocations for PI-games improving the weaknesses of the Owen point. Regarding the first goal, we advance further on the analysis of PI-games and we analyze its core structure and algorithmic complexity. Specifically, we prove that the number of extreme points of the core of PI-games is exponential on the number of players. On the other hand, we propose and characterize a new core-allocation, the Omega point, which compensates the essential players for their role on reducing the costs of their fans. Moreover, we define another solution concept, the Quid Pro Quo set (QPQ-set) of allocations, which is based on the Owen and Omega points. Among all the allocations in this set, we emphasize what we call the Solomonic QPQ allocation and we provide some necessary conditions for the coincidence of that allocation with the Shapley value and the Nucleolus.
Local Nash Equilibria are Isolated, Strict Local Nash Equilibria in `Almost All' Zero-Sum Continuous Games
Venue: arXiv cs.GT 2020
Year: 2020

We prove that differential Nash equilibria are generic amongst local Nash equilibria in continuous zero-sum games. That is, there exists an open-dense subset of zero-sum games for which local Nash equilibria are non-degenerate differential Nash equilibria. The result extends previous results to the zero-sum setting, where we obtain even stronger results; in particular, we show that local Nash equilibria are generically hyperbolic critical points. We further show that differential Nash equilibria of zero-sum games are structurally stable. The purpose for presenting these extensions is the recent renewed interest in zero-sum games within machine learning and optimization. Adversarial learning and generative adversarial network approaches are touted to be more robust than the alternative. Zero-sum games are at the heart of such approaches. Many works proceed under the assumption of hyperbolicity of critical points. Our results justify this assumption by showing `almost all' zero-sum games admit local Nash equilibria that are hyperbolic.
Careful synchronization of partial deterministic finite automata
Venue: arXiv cs.FL 2020
Year: 2020

We approach the task of computing a carefully synchronizing word of optimum length for a given partial deterministic automaton, encoding the problem as an instance of SAT and invoking a SAT solver. Our experiments demonstrate that this approach gives satisfactory results for automata with up to 100 states even if very modest computational resources are used. We compare our results with the ones obtained by the first author for exact synchronization, which is another version of synchronization studied in the literature, and draw some theoretical conclusions.