OpenRaiser/Intern-Atlas huggingface dataset:
Utterance-level Permutation Invariant Training with Latency-controlled BLSTM for Single-channel Multi-talker Speech Separation
Venue: arXiv cs.SD 2019
Year: 2019

Utterance-level permutation invariant training (uPIT) has achieved promising progress on single-channel multi-talker speech separation task. Long short-term memory (LSTM) and bidirectional LSTM (BLSTM) are widely used as the separation networks of uPIT, i.e. uPIT-LSTM and uPIT-BLSTM. uPIT-LSTM has lower latency but worse performance, while uPIT-BLSTM has better performance but higher latency. In this paper, we propose using latency-controlled BLSTM (LC-BLSTM) during inference to fulfill low-latency and good-performance speech separation. To find a better training strategy for BLSTM-based separation network, chunk-level PIT (cPIT) and uPIT are compared. The experimental results show that uPIT outperforms cPIT when LC-BLSTM is used during inference. It is also found that the inter-chunk speaker tracing (ST) can further improve the separation performance of uPIT-LC-BLSTM. Evaluated on the WSJ0 two-talker mixed-speech separation task, the absolute gap of signal-to-distortion ratio (SDR) between uPIT-BLSTM and uPIT-LC-BLSTM is reduced to within 0.7 dB.
A Study of Multilingual Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Multilingual neural machine translation (NMT) has recently been investigated from different aspects (e.g., pivot translation, zero-shot translation, fine-tuning, or training from scratch) and in different settings (e.g., rich resource and low resource, one-to-many, and many-to-one translation). This paper concentrates on a deep understanding of multilingual NMT and conducts a comprehensive study on a multilingual dataset with more than 20 languages. Our results show that (1) low-resource language pairs benefit much from multilingual training, while rich-resource language pairs may get hurt under limited model capacity and training with similar languages benefits more than dissimilar languages; (2) fine-tuning performs better than training from scratch in the one-to-many setting while training from scratch performs better in the many-to-one setting; (3) the bottom layers of the encoder and top layers of the decoder capture more language-specific information, and just fine-tuning these parts can achieve good accuracy for low-resource language pairs; (4) direct translation is better than pivot translation when the source language is similar to the target language (e.g., in the same language branch), even when the size of direct training data is much smaller; (5) given a fixed training data budget, it is better to introduce more languages into multilingual training for zero-shot translation.
Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Venue: arXiv cs.CL 2019
Year: 2019

Self-attention based Transformer has demonstrated the state-of-the-art performances in a number of natural language processing tasks. Self-attention is able to model long-term dependencies, but it may suffer from the extraction of irrelevant information in the context. To tackle the problem, we propose a novel model called \textbf{Explicit Sparse Transformer}. Explicit Sparse Transformer is able to improve the concentration of attention on the global context through an explicit selection of the most relevant segments. Extensive experimental results on a series of natural language processing and computer vision tasks, including neural machine translation, image captioning, and language modeling, all demonstrate the advantages of Explicit Sparse Transformer in model performance. We also show that our proposed sparse attention method achieves comparable or better results than the previous sparse attention method, but significantly reduces training and testing time. For example, the inference speed is twice that of sparsemax in Transformer model. Code will be available at \url{https://github.com/lancopku/Explicit-Sparse-Transformer}
Improving Visual Recognition using Ambient Sound for Supervision
Venue: arXiv cs.CV 2019
Year: 2019

Our brains combine vision and hearing to create a more elaborate interpretation of the world. When the visual input is insufficient, a rich panoply of sounds can be used to describe our surroundings. Since more than 1,000 hours of videos are uploaded to the internet everyday, it is arduous, if not impossible, to manually annotate these videos. Therefore, incorporating audio along with visual data without annotations is crucial for leveraging this explosion of data for recognizing and understanding objects and scenes. Owens,et.al suggest that a rich representation of the physical world can be learned by using a convolutional neural network to predict sound textures associated with a given video frame. We attempt to reproduce the claims from their experiments, of which the code is not publicly available. In addition, we propose improvements in the pretext task that result in better performance in other downstream computer vision tasks.
Utility Mining Across Multi-Sequences with Individualized Thresholds
Venue: arXiv cs.DB 2019
Year: 2019

Utility-oriented pattern mining has become an emerging topic since it can reveal high-utility patterns (e.g., itemsets, rules, sequences) from different types of data, which provides more information than the traditional frequent/confident-based pattern mining models. The utilities of various items are not exactly equal in realistic situations; each item has its own utility or importance. In general, user considers a uniform minimum utility (minutil) threshold to identify the set of high-utility sequential patterns (HUSPs). This is unable to find the interesting patterns while the minutil is set extremely high or low. We first design a new utility mining framework namely USPT for mining high-Utility Sequential Patterns across multi-sequences with individualized Thresholds. Each item in the designed framework has its own specified minimum utility threshold. Based on the lexicographic-sequential tree and the utility-array structure, the USPT framework is presented to efficiently discover the HUSPs. With the upper-bounds on utility, several pruning strategies are developed to prune the unpromising candidates early in the search space. Several experiments are conducted on both real-life and synthetic datasets to show the performance of the designed USPT algorithm, and the results showed that USPT could achieve good effectiveness and efficiency for mining HUSPs with individualized minimum utility thresholds.
Learning Controllable Disentangled Representations with Decorrelation Regularization
Venue: arXiv cs.LG 2019
Year: 2019

A crucial problem in learning disentangled image representations is controlling the degree of disentanglement during image editing, while preserving the identity of objects. In this work, we propose a simple yet effective model with the encoder-decoder architecture to address this challenge. To encourage disentanglement, we devise a distance covariance based decorrelation regularization. Further, for the reconstruction step, our model leverages a soft target representation combined with the latent image code. By exploiting the real-valued space of the soft target representations, we are able to synthesize novel images with the designated properties. We also design a classification based protocol to quantitatively evaluate the disentanglement strength of our model. Experimental results show that the proposed model competently disentangles factors of variation, and is able to manipulate face images to synthesize the desired attributes.
Joint Annotator-and-Spectrum Allocation in Wireless Networks for Crowd Labelling
Venue: arXiv cs.IT 2019
Year: 2019

The massive sensing data generated by Internet-of-Things will provide fuel for ubiquitous artificial intelligence (AI), automating the operations of our society ranging from transportation to healthcare. The realistic adoption of this technique however entails labelling of the enormous data prior to the training of AI models via supervised learning. To tackle this challenge, we explore a new perspective of wireless crowd labelling that is capable of downloading data to many imperfect mobile annotators for repetition labelling by exploiting multicasting in wireless networks. In this cross-disciplinary area, the integration of the rate-distortion theory and the principle of repetition labelling for accuracy improvement gives rise to a new tradeoff between radio-and-annotator resources under a constraint on labelling accuracy. Building on the tradeoff and aiming at maximizing the labelling throughput, this work focuses on the joint optimization of encoding rate, annotator clustering, and sub-channel allocation, which results in an NP-hard integer programming problem. To devise an efficient solution approach, we establish an optimal sequential annotator-clustering scheme based on the order of decreasing signal-to-noise ratios. Thereby, the optimal solution can be found by an efficient tree search. Next, the solution is simplified by applying truncated channel inversion. Alternatively, the optimization problem can be recognized as a knapsack problem, which can be efficiently solved in pseudo-polynomial time by means of dynamic programming. In addition, exact polices are derived for the annotators constrained and spectrum constrained cases. Last, simulation results demonstrate the significant throughput gains based on the optimal solution compared with decoupled allocation of the two types of resources.
Discussion of Intelligent Electric Wheelchairs for Caregivers and Care Recipients
Venue: arXiv cs.HC 2019
Year: 2019

In order to reduce the burden on caregivers, we developed an intelligent electric wheelchair. We held workshops with caregivers, asked then regarding the problems in caregiving, and developed problem-solving methods. In the workshop, caregivers' physical fitness and psychology of the older adults were found to be problems and a solution was proposed. We implemented a cooperative operation function for multiple electric wheelchairs based on the workshop and demonstrated it at a nursing home. By listening to older adults, we obtained feedback on the automatic driving electric wheelchair. From the results of this study, we discovered the issues and solutions to be applied to the intelligent electric wheelchair.
Proof of Federated Learning: A Novel Energy-recycling Consensus Algorithm
Venue: arXiv cs.CR 2019
Year: 2019

Proof of work (PoW), the most popular consensus mechanism for Blockchain, requires ridiculously large amounts of energy but without any useful outcome beyond determining accounting rights among miners. To tackle the drawback of PoW, we propose a novel energy-recycling consensus algorithm, namely proof of federated learning (PoFL), where the energy originally wasted to solve difficult but meaningless puzzles in PoW is reinvested to federated learning. Federated learning and pooled-ming, a trend of PoW, have a natural fit in terms of organization structure. However, the separation between the data usufruct and ownership in Blockchain lead to data privacy leakage in model training and verification, deviating from the original intention of federal learning. To address the challenge, a reverse game-based data trading mechanism and a privacy-preserving model verification mechanism are proposed. The former can guard against training data leakage while the latter verifies the accuracy of a trained model with privacy preservation of the task requester's test data as well as the pool's submitted model. To the best of our knowledge, our paper is the first work to employ federal learning as the proof of work for Blockchain. Extensive simulations based on synthetic and real-world data demonstrate the effectiveness and efficiency of our proposed mechanisms.
Learning Hybrid Representation by Robust Dictionary Learning in Factorized Compressed Space
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we investigate the robust dictionary learning (DL) to discover the hybrid salient low-rank and sparse representation in a factorized compressed space. A Joint Robust Factorization and Projective Dictionary Learning (J-RFDL) model is presented. The setting of J-RFDL aims at improving the data representations by enhancing the robustness to outliers and noise in data, encoding the reconstruction error more accurately and obtaining hybrid salient coefficients with accurate reconstruction ability. Specifically, J-RFDL performs the robust representation by DL in a factorized compressed space to eliminate the negative effects of noise and outliers on the results, which can also make the DL process efficient. To make the encoding process robust to noise in data, J-RFDL clearly uses sparse L2, 1-norm that can potentially minimize the factorization and reconstruction errors jointly by forcing rows of the reconstruction errors to be zeros. To deliver salient coefficients with good structures to reconstruct given data well, J-RFDL imposes the joint low-rank and sparse constraints on the embedded coefficients with a synthesis dictionary. Based on the hybrid salient coefficients, we also extend J-RFDL for the joint classification and propose a discriminative J-RFDL model, which can improve the discriminating abilities of learnt coeffi-cients by minimizing the classification error jointly. Extensive experiments on public datasets demonstrate that our formulations can deliver superior performance over other state-of-the-art methods.
Nonlinear systems for unconventional computing
Venue: arXiv cs.ET 2019
Year: 2019

The search for new computational machines beyond the traditional von Neumann architecture has given rise to a modern area of nonlinear science -- development of unconventional computing -- requiring the efforts of mathematicians, physicists and engineers. Many analogue physical systems including nonlinear oscillator networks, lasers, and condensates were proposed and realised to address hard computational problems from various areas of social and physical sciences and technology. The analogue systems emulate spin Hamiltonians with continuous or discrete degrees of freedom to which actual optimisation problems can be mapped. Understanding the underlying physical process by which the system finds the ground state often leads to new classes of system-inspired or quantum-inspired algorithms for hard optimisation. Together physical platforms and related algorithms can be combined to form a hybrid architecture that may one day compete with conventional computing. In this Chapter, we review some of the systems and physically-inspired algorithms that show such promise.
Edge Caching with Real-Time Guarantees
Venue: arXiv cs.IT 2019
Year: 2019

In recent years, optimization of the success transmission probability in wireless cache-enabled networks has been studied extensively. However, few works have concerned about the real-time performance of the cache-enabled networks. In this paper, we investigate the performance of the cache-enabled with real-time guarantees by adopting the age of information (AoI) as the metric to characterize the timeliness of the delivered information. We establish a spatial-temporal model by utilizing the stochastic geometry and queuing theory which captures both the temporal traffic dynamics and the interferers' geographic distribution. Under the random caching framework, we achieve the closed-form expression of AoI by adopting the maximum average received power criterion for the user association. Finally, we formulate a convex optimization problem for the minimization of the Peak AoI (PAoI) and obtain the optimal caching probabilities by utilizing the Karush-Kuhn-Tucker (KKT) conditions. Numerical results demonstrate that the random caching strategy is a better choice rather than the most popular content and uniform caching strategy to improve the real-time performance for the cached file as well as maintaining the file diversity at a relatively high level.
Fault Tolerance in SDN Data Plane Considering Network and Application Based Metrics
Venue: arXiv cs.NI 2019
Year: 2019

Failures in networks result in service disruptions which may cause deteriorated Quality of Service (QoS) for the end users. Since SDN is becoming the mainstream paradigm for networks, implementation of a robust fault tolerance scheme for SDN-based networks is crucial. Existing SDN data plane fault tolerance approaches can be classified as reactive and proactive which may or may not rely on the controller, respectively. However, none of them qualifies as a complete solution, providing only partial remedies. In this work, we propose Dynamic Protection with Quality of Alternative Paths (DPQoAP) that considers not only the existing faults within the network but also the quality of alternative paths. As a result, we can sustain the QoS throughout the network after the recovery. We also investigate how application based parameters are affected by link failures. To this end, we explore the change in Quality of Experience (QoE) caused by link failures under different cases using Dynamic Adaptive Streaming over HTTP (DASH) for video streaming. On the other hand, even though DASH is proposed as a solution to improve the QoE affected by the dynamic conditions of the networks, it remains insufficient to handle the congested links that show the symptoms of a link failure. Thus, we apply the data plane fault tolerance approach in SDN to improve the QoE of DASH clients in the case of congestion as well as the failure. The performance of the proposed solutions are evaluated through various experiments considering the QoS and QoE parameters. It is observed that DPQoAP enhances the efficiency of the networking operations and adaptability of the applications.
Analysis of Reference and Citation Copying in Evolving Bibliographic Networks
Venue: arXiv cs.SI 2019
Year: 2019

Extensive literature demonstrates how the copying of references (links) can lead to the emergence of various structural properties (e.g., power-law degree distribution and bipartite cores) in bibliographic and other similar directed networks. However, it is also well known that the copying process is incapable of mimicking the number of directed triangles in such networks; neither does it have the power to explain the obsolescence of older papers. In this paper, we propose RefOrCite, a new model that allows for copying of both the references from (i.e., out-neighbors of) as well as the citations to (i.e., in-neighbors of) an existing node. In contrast, the standard copying model (CP) only copies references. While retaining its spirit, RefOrCite differs from the Forest Fire (FF) model in ways that makes RefOrCite amenable to mean-field analysis for degree distribution, triangle count, and densification. Empirically, RefOrCite gives the best overall agreement with observed degree distribution, triangle count, diameter, h-index, and the growth of citations to newer papers.
A simple baseline for domain adaptation using rotation prediction
Venue: arXiv cs.CV 2019
Year: 2019

Recently, domain adaptation has become a hot research area with lots of applications. The goal is to adapt a model trained in one domain to another domain with scarce annotated data. We propose a simple yet effective method based on self-supervised learning that outperforms or is on par with most state-of-the-art algorithms, e.g. adversarial domain adaptation. Our method involves two phases: predicting random rotations (self-supervised) on the target domain along with correct labels for the source domain (supervised), and then using self-distillation on the target domain. Our simple method achieves state-of-the-art results on semi-supervised domain adaptation on DomainNet dataset. Further, we observe that the unlabeled target datasets of popular domain adaptation benchmarks do not contain any categories apart from testing categories. We believe this introduces a bias that does not exist in many real applications. We show that removing this bias from the unlabeled data results in a large drop in performance of state-of-the-art methods, while our simple method is relatively robust.
MoEVC: A Mixture-of-experts Voice Conversion System with Sparse Gating Mechanism for Accelerating Online Computation
Venue: arXiv cs.SD 2019
Year: 2019

With the recent advancements of deep learning technologies, the performance of voice conversion (VC) in terms of quality and similarity has been significantly improved. However, heavy computations are generally required for deep-learning-based VC systems, which can cause notable latency and thus confine their deployments in real-world applications. Therefore, increasing online computation efficiency has become an important task. In this study, we propose a novel mixture-of-experts (MoE) based VC system. The MoE model uses a gating mechanism to specify optimal weights to feature maps to increase VC performance. In addition, assigning sparse constraints on the gating mechanism can accelerate online computation by skipping the convolution process by zeroing out redundant feature maps. Experimental results show that by specifying suitable sparse constraints, we can effectively increase the online computation efficiency with a notable 70% FLOPs (floating-point operations per second) reduction while improving the VC performance in both objective evaluations and human listening tests.
Visual Agreement Regularized Training for Multi-Modal Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Multi-modal machine translation aims at translating the source sentence into a different language in the presence of the paired image. Previous work suggests that additional visual information only provides dispensable help to translation, which is needed in several very special cases such as translating ambiguous words. To make better use of visual information, this work presents visual agreement regularized training. The proposed approach jointly trains the source-to-target and target-to-source translation models and encourages them to share the same focus on the visual information when generating semantically equivalent visual words (e.g. "ball" in English and "ballon" in French). Besides, a simple yet effective multi-head co-attention model is also introduced to capture interactions between visual and textual features. The results show that our approaches can outperform competitive baselines by a large margin on the Multi30k dataset. Further analysis demonstrates that the proposed regularized training can effectively improve the agreement of attention on the image, leading to better use of visual information.
Two families of Entanglement-assisted quantum MDS codes from constacyclic codes
Venue: arXiv cs.IT 2019
Year: 2019

Entanglement-assisted quantum error correcting codes (EAQECCs) can be derived from arbitrary classical linear codes. However, it is a very difficult task to determine the number of entangled states required. In this work, using the method of the decomposition of the defining set of constacyclic codes, we construct two families of q-ary entanglement-assisted quantum MDS (EAQMDS) codes based on classical constacyclic MDS codes by exploiting less pre-shared maximally entangled states. We show that a class of q-ary EAQMDS have minimum distance upper bound greater than q. Some of them have much larger minimum distance than the known quantum MDS (QMDS) codes of the same length. Most of these q-ary EAQMDS codes are new in the sense that their parameters are not covered by the codes available in the literature.
Derandomized compressed sensing with nonuniform guarantees for $\ell_1$ recovery
Venue: arXiv cs.IT 2019
Year: 2019

We extend the techniques of Hügel, Rauhut and Strohmer (Found. Comput. Math., 2014) to show that for every $δ\in(0,1]$, there exists an explicit random $m\times N$ partial Fourier matrix $A$ with $m=s\operatorname{polylog}(N/ε)$ and entropy $s^δ\operatorname{polylog}(N/ε)$ such that for every $s$-sparse signal $x\in\mathbb{C}^N$, there exists an event of probability at least $1-ε$ over which $x$ is the unique minimizer of $\|z\|_1$ subject to $Az=Ax$. The bulk of our analysis uses tools from decoupling to estimate the extreme singular values of the submatrix of $A$ whose columns correspond to the support of $x$.