OpenRaiser/Intern-Atlas huggingface dataset:
Complexity-Adaptive Maximum-Likelihood Decoding of Modified $\boldsymbol{G}_N$-Coset Codes
Venue: arXiv cs.IT 2021
Year: 2021

A complexity-adaptive tree search algorithm is proposed for $\boldsymbol{G}_N$-coset codes that implements maximum-likelihood (ML) decoding by using a successive decoding schedule. The average complexity is close to that of the successive cancellation (SC) decoding for practical error rates when applied to polar codes and short Reed-Muller (RM) codes, e.g., block lengths up to $N=128$. By modifying the algorithm to limit the worst-case complexity, one obtains a near-ML decoder for longer RM codes and their subcodes. Unlike other bit-flip decoders, no outer code is needed to terminate decoding. The algorithm can thus be applied to modified $\boldsymbol{G}_N$-coset code constructions with dynamic frozen bits. One advantage over sequential decoders is that there is no need to optimize a separate parameter.
Wiki-Reliability: A Large Scale Dataset for Content Reliability on Wikipedia
Venue: arXiv cs.IR 2021
Year: 2021

Wikipedia is the largest online encyclopedia, used by algorithms and web users as a central hub of reliable information on the web. The quality and reliability of Wikipedia content is maintained by a community of volunteer editors. Machine learning and information retrieval algorithms could help scale up editors' manual efforts around Wikipedia content reliability. However, there is a lack of large-scale data to support the development of such research. To fill this gap, in this paper, we propose Wiki-Reliability, the first dataset of English Wikipedia articles annotated with a wide set of content reliability issues. To build this dataset, we rely on Wikipedia "templates". Templates are tags used by expert Wikipedia editors to indicate content issues, such as the presence of "non-neutral point of view" or "contradictory articles", and serve as a strong signal for detecting reliability issues in a revision. We select the 10 most popular reliability-related templates on Wikipedia, and propose an effective method to label almost 1M samples of Wikipedia article revisions as positive or negative with respect to each template. Each positive/negative example in the dataset comes with the full article text and 20 features from the revision's metadata. We provide an overview of the possible downstream tasks enabled by such data, and show that Wiki-Reliability can be used to train large-scale models for content reliability prediction. We release all data and code for public use.
MASS: Multi-task Anthropomorphic Speech Synthesis Framework
Venue: arXiv cs.SD 2021
Year: 2021

Text-to-Speech (TTS) synthesis plays an important role in human-computer interaction. Currently, most TTS technologies focus on the naturalness of speech, namely,making the speeches sound like humans. However, the key tasks of the expression of emotion and the speaker identity are ignored, which limits the application scenarios of TTS synthesis technology. To make the synthesized speech more realistic and expand the application scenarios, we propose a multi-task anthropomorphic speech synthesis framework (MASS), which can synthesize speeches from text with specified emotion and speaker identity. The MASS framework consists of a base TTS module and two novel voice conversion modules: the emotional voice conversion module and the speaker voice conversion module. We propose deep emotion voice conversion model (DEVC) and deep speaker voice conversion model (DSVC) based on convolution residual networks. It solves the problem of feature loss during voice conversion. The model trainings are independent of parallel datasets, and are capable of many-to-many voice conversion. In the emotional voice conversion, speaker voice conversion experiments, as well as the multi-task speech synthesis experiments, experimental results show DEVC and DSVC convert speech effectively. The quantitative and qualitative evaluation results of multi-task speech synthesis experiments show MASS can effectively synthesis speech with specified text, emotion and speaker identity.
Examining and Mitigating Kernel Saturation in Convolutional Neural Networks using Negative Images
Venue: arXiv cs.CV 2021
Year: 2021

Neural saturation in Deep Neural Networks (DNNs) has been studied extensively, but remains relatively unexplored in Convolutional Neural Networks (CNNs). Understanding and alleviating the effects of convolutional kernel saturation is critical for enhancing CNN models classification accuracies. In this paper, we analyze the effect of convolutional kernel saturation in CNNs and propose a simple data augmentation technique to mitigate saturation and increase classification accuracy, by supplementing negative images to the training dataset. We hypothesize that greater semantic feature information can be extracted using negative images since they have the same structural information as standard images but differ in their data representations. Varied data representations decrease the probability of kernel saturation and thus increase the effectiveness of kernel weight updates. The two datasets selected to evaluate our hypothesis were CIFAR- 10 and STL-10 as they have similar image classes but differ in image resolutions thus making for a better understanding of the saturation phenomenon. MNIST dataset was used to highlight the ineffectiveness of the technique for linearly separable data. The ResNet CNN architecture was chosen since the skip connections in the network ensure the most important features contributing the most to classification accuracy are retained. Our results show that CNNs are indeed susceptible to convolutional kernel saturation and that supplementing negative images to the training dataset can offer a statistically significant increase in classification accuracies when compared against models trained on the original datasets. Our results present accuracy increases of 6.98% and 3.16% on the STL-10 and CIFAR-10 datasets respectively.
REPT: Bridging Language Models and Machine Reading Comprehension via Retrieval-Based Pre-training
Venue: arXiv cs.CL 2021
Year: 2021

Pre-trained Language Models (PLMs) have achieved great success on Machine Reading Comprehension (MRC) over the past few years. Although the general language representation learned from large-scale corpora does benefit MRC, the poor support in evidence extraction which requires reasoning across multiple sentences hinders PLMs from further advancing MRC. To bridge the gap between general PLMs and MRC, we present REPT, a REtrieval-based Pre-Training approach. In particular, we introduce two self-supervised tasks to strengthen evidence extraction during pre-training, which is further inherited by downstream MRC tasks through the consistent retrieval operation and model architecture. To evaluate our proposed method, we conduct extensive experiments on five MRC datasets that require collecting evidence from and reasoning across multiple sentences. Experimental results demonstrate the effectiveness of our pre-training approach. Moreover, further analysis shows that our approach is able to enhance the capacity of evidence extraction without explicit supervision.
Efficient Error-Correcting-Code Mechanism for High-Throughput Memristive Processing-in-Memory
Venue: arXiv cs.AR 2021
Year: 2021

Inefficient data transfer between computation and memory inspired emerging processing-in-memory (PIM) technologies. Many PIM solutions enable storage and processing using memristors in a crossbar-array structure, with techniques such as memristor-aided logic (MAGIC) used for computation. This approach provides highly-paralleled logic computation with minimal data movement. However, memristors are vulnerable to soft errors and standard error-correcting-code (ECC) techniques are difficult to implement without moving data outside the memory. We propose a novel technique for efficient ECC implementation along diagonals to support reliable computation inside the memory without explicitly reading the data. Our evaluation demonstrates an improvement of over eight orders of magnitude in reliability (mean time to failure) for an increase of about 26% in computation latency.
In-Hindsight Quantization Range Estimation for Quantized Training
Venue: arXiv cs.LG 2021
Year: 2021

Quantization techniques applied to the inference of deep neural networks have enabled fast and efficient execution on resource-constraint devices. The success of quantization during inference has motivated the academic community to explore fully quantized training, i.e. quantizing back-propagation as well. However, effective gradient quantization is still an open problem. Gradients are unbounded and their distribution changes significantly during training, which leads to the need for dynamic quantization. As we show, dynamic quantization can lead to significant memory overhead and additional data traffic slowing down training. We propose a simple alternative to dynamic quantization, in-hindsight range estimation, that uses the quantization ranges estimated on previous iterations to quantize the present. Our approach enables fast static quantization of gradients and activations while requiring only minimal hardware support from the neural network accelerator to keep track of output statistics in an online fashion. It is intended as a drop-in replacement for estimating quantization ranges and can be used in conjunction with other advances in quantized training. We compare our method to existing methods for range estimation from the quantized training literature and demonstrate its effectiveness with a range of architectures, including MobileNetV2, on image classification benchmarks (Tiny ImageNet & ImageNet).
Expressing and Exploiting the Common Subgoal Structure of Classical Planning Domains Using Sketches: Extended Version
Venue: arXiv cs.AI 2021
Year: 2021

Width-based planning methods deal with conjunctive goals by decomposing problems into subproblems of low width. Algorithms like SIW thus fail when the goal is not easily serializable in this way or when some of the subproblems have a high width. In this work, we address these limitations by using a simple but powerful language for expressing finer problem decompositions introduced recently by Bonet and Geffner, called policy sketches. A policy sketch over a set of Boolean and numerical features is a set of sketch rules that express how the values of these features are supposed to change. Like general policies, policy sketches are domain general, but unlike policies, the changes captured by sketch rules do not need to be achieved in a single step. We show that many planning domains that cannot be solved by SIW are provably solvable in low polynomial time with the SIW_R algorithm, the version of SIW that employs user-provided policy sketches. Policy sketches are thus shown to be a powerful language for expressing domain-specific knowledge in a simple and compact way and a convenient alternative to languages such as HTNs or temporal logics. Furthermore, they make it easy to express general problem decompositions and prove key properties of them like their width and complexity.
Boomerang uniformity of a class of power maps
Venue: arXiv cs.IT 2021
Year: 2021

We consider the boomerang uniformity of an infinite class of (locally-APN) power maps and show that its boomerang uniformity over the finite field $\F_{2^n}$ is $2$ and $4$, when $n \equiv 0 \pmod 4$ and $n \equiv 2 \pmod 4$, respectively. As a consequence, we show that for this class of power maps, the differential uniformity is strictly greater than its boomerang uniformity.
Overcoming Complexity Catastrophe: An Algorithm for Beneficial Far-Reaching Adaptation under High Complexity
Venue: arXiv cs.NE 2021
Year: 2021

In his seminal work with NK algorithms, Kauffman noted that fitness outcomes from algorithms navigating an NK landscape show a sharp decline at high complexity arising from pervasive interdependence among problem dimensions. This phenomenon - where complexity effects dominate (Darwinian) adaptation efforts - is called complexity catastrophe. We present an algorithm - incremental change taking turns (ICTT) - that finds distant configurations having fitness superior to that reported in extant research, under high complexity. Thus, complexity catastrophe is not inevitable: a series of incremental changes can lead to excellent outcomes.
RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation
Venue: arXiv cs.CV 2021
Year: 2021

Existing online multiple object tracking (MOT) algorithms often consist of two subtasks, detection and re-identification (ReID). In order to enhance the inference speed and reduce the complexity, current methods commonly integrate these double subtasks into a unified framework. Nevertheless, detection and ReID demand diverse features. This issue would result in an optimization contradiction during the training procedure. With the target of alleviating this contradiction, we devise a module named Global Context Disentangling (GCD) that decouples the learned representation into detection-specific and ReID-specific embeddings. As such, this module provides an implicit manner to balance the different requirements of these two subtasks. Moreover, we observe that preceding MOT methods typically leverage local information to associate the detected targets and neglect to consider the global semantic relation. To resolve this restriction, we develop a module, referred to as Guided Transformer Encoder (GTE), by combining the powerful reasoning ability of Transformer encoder and deformable attention. Unlike previous works, GTE avoids analyzing all the pixels and only attends to capture the relation between query nodes and a few self-adaptively selected key samples. Therefore, it is computationally efficient. Extensive experiments have been conducted on the MOT16, MOT17 and MOT20 benchmarks to demonstrate the superiority of the proposed MOT framework, namely RelationTrack. The experimental results indicate that RelationTrack has surpassed preceding methods significantly and established a new state-of-the-art performance, e.g., IDF1 of 70.5% and MOTA of 67.2% on MOT20.
Exploring open-ended gameplay features with Micro RollerCoaster Tycoon
Venue: arXiv cs.AI 2021
Year: 2021

This paper introduces MicroRCT, a novel open source simulator inspired by the theme park sandbox game RollerCoaster Tycoon. The goal in MicroRCT is to place rides and shops in an amusement park to maximize profit earned from park guests. Thus, the challenges for game AI include both selecting high-earning attractions and placing them in locations that are convenient to guests. In this paper, the MAP-Elites algorithm is used to generate a diversity of park layouts, exploring two theoretical questions about evolutionary algorithms and game design: 1) Is there a benefit to starting from a minimal starting point for evolution and complexifying incrementally? and 2) What are the effects of resource limitations on creativity and optimization? Results indicate that building from scratch with no costs results in the widest diversity of high-performing designs.
Trials and Tribulations of Developing Hybrid Quantum-Classical Microservices Systems
Venue: arXiv cs.SE 2021
Year: 2021

Quantum computing holds great promise to solve to problems where classical computers cannot reach. To the point where it already arouses the interest of both scientific and industrial communities. Thus, it is expected that hybrid systems will start to appear where quantum software interacts with classical systems. Such coexistence can be fostered by service computing. Unfortunately, the way in which quantum code can be offered as a service still misses out on many of the potential benefits of service computing. This paper takes the traveling salesman problem, and tackles the challenge of giving it an implementation in the form of a quantum microservice. Then it is used to detect which of the benefits of service computing are lost in the process. The conclusions help to measure the distance between the current state of technology and the state that would be desirable in order to have a real quantum service engineering.
Highly Efficient Memory Failure Prediction using Mcelog-based Data Mining and Machine Learning
Venue: arXiv cs.DB 2021
Year: 2021

In the data center, unexpected downtime caused by memory failures can lead to a decline in the stability of the server and even the entire information technology infrastructure, which harms the business. Therefore, whether the memory failure can be accurately predicted in advance has become one of the most important issues to be studied in the data center. However, for the memory failure prediction in the production system, it is necessary to solve technical problems such as huge data noise and extreme imbalance between positive and negative samples, and at the same time ensure the long-term stability of the algorithm. This paper compares and summarizes some commonly used skills and the improvement they can bring. The single model we proposed won the top 14th in the 2nd Alibaba Cloud AIOps Competition belonging to the 25th PAKDD conference. It takes only 30 minutes to pass the online test, while most of the other contestants' solution need more than 3 hours. Codes has been open source to https://www.github.com/ycd2016/acaioc2.
Learning Contact-aware CPG-based Locomotion in a Soft Snake Robot
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we present a model-free learning-based control scheme for the soft snake robot to improve its contact-aware locomotion performance in a cluttered environment. The control scheme includes two cooperative controllers: A bio-inspired controller (C1) that controls both the steering and velocity of the soft snake robot, and an event-triggered regulator (R2) that controls the steering of the snake in anticipation of obstacle contacts and during contact. The inputs from the two controllers are composed as the input to a Matsuoka CPG network to generate smooth and rhythmic actuation inputs to the soft snake. To enable stable and efficient learning with two controllers, we develop a game-theoretic process, fictitious play, to train C1 and R2 with a shared potential-field-based reward function for goal tracking tasks. The proposed approach is tested and evaluated in the simulator and shows significant improvement of locomotion performance in the obstacle-based environment comparing to two baseline controllers.
Asymptotic Behavior and Typicality Properties of Runlength-Limited Sequences
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies properties of binary runlength-limited sequences with additional constraints on their Hamming weight and/or their number of runs of identical symbols. An algebraic and a probabilistic (entropic) characterization of the exponential growth rate of the number of such sequences, i.e., their information capacity, are obtained by using the methods of multivariate analytic combinatorics, and properties of the capacity as a function of its parameters are stated. The second-order term in the asymptotic expansion of the rate of these sequences is also given, and the typical values of the relevant quantities are derived. Several applications of the results are illustrated, including bounds on codes for weight-preserving and run-preserving channels (e.g., the run-preserving insertion-deletion channel), a sphere-packing bound for channels with sparse error patterns, and the asymptotics of constant-weight sub-block constrained sequences. In addition, the asymptotics of a closely related notion -- $ q $-ary sequences with fixed Manhattan weight -- is briefly discussed, and an application in coding for molecular timing channels is illustrated.
Enhancing Photorealism Enhancement
Venue: arXiv cs.CV 2021
Year: 2021

We present an approach to enhancing the realism of synthetic images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We hypothesize that this is one of the causes of strong artifacts that can be observed in the results of many prior methods. To address this we propose a new strategy for sampling image patches during training. We also introduce multiple architectural improvements in the deep network modules used for photorealism enhancement. We confirm the benefits of our contributions in controlled experiments and report substantial gains in stability and realism in comparison to recent image-to-image translation methods and a variety of other baselines.
Rise of QAnon: A Mental Model of Good and Evil Stews in an Echochamber
Venue: arXiv cs.SI 2021
Year: 2021

The QAnon conspiracy posits that Satan-worshiping Democrats operate a covert child sex-trafficking operation, which Donald Trump is destined to expose and annihilate. Emblematic of the ease with which political misconceptions can spread through social media, QAnon originated in late 2017 and rapidly grew to shape the political beliefs of millions. To illuminate the process by which a conspiracy theory spreads, we report two computational studies examining the social network structure and semantic content of tweets produced by users central to the early QAnon network on Twitter. Using data mined in the summer of 2018, we examined over 800,000 tweets about QAnon made by about 100,000 users. The majority of users disseminated rather than produced information, serving to create an online echochamber. Users appeared to hold a simplistic mental model in which political events are viewed as a struggle between antithetical forces-both observed and unobserved-of Good and Evil.
What shall we do with an hour of data? Speech recognition for the un- and under-served languages of Common Voice
Venue: arXiv cs.CL 2021
Year: 2021

This technical report describes the methods and results of a three-week sprint to produce deployable speech recognition models for 31 under-served languages of the Common Voice project. We outline the preprocessing steps, hyperparameter selection, and resulting accuracy on official testing sets. In addition to this we evaluate the models on multiple tasks: closed-vocabulary speech recognition, pre-transcription, forced alignment, and key-word spotting. The following experiments use Coqui STT, a toolkit for training and deployment of neural Speech-to-Text models.
Unsupervised domain adaptation via double classifiers based on high confidence pseudo label
Venue: arXiv cs.CV 2021
Year: 2021

Unsupervised domain adaptation (UDA) aims to solve the problem of knowledge transfer from labeled source domain to unlabeled target domain. Recently, many domain adaptation (DA) methods use centroid to align the local distribution of different domains, that is, to align different classes. This improves the effect of domain adaptation, but domain differences exist not only between classes, but also between samples. This work rethinks what is the alignment between different domains, and studies how to achieve the real alignment between different domains. Previous DA methods only considered one distribution feature of aligned samples, such as full distribution or local distribution. In addition to aligning the global distribution, the real domain adaptation should also align the meso distribution and the micro distribution. Therefore, this study propose a double classifier method based on high confidence label (DCP). By aligning the centroid and the distribution between centroid and sample of different classifiers, the meso and micro distribution alignment of different domains is realized. In addition, in order to reduce the chain error caused by error marking, This study propose a high confidence marking method to reduce the marking error. To verify its versatility, this study evaluates DCP on digital recognition and target recognition data sets. The results show that our method achieves state-of-the-art results on most of the current domain adaptation benchmark datasets.