Reviewerly/exHarmony huggingface dataset:
Quantum Algorithm for Estimating Volumes of Convex Bodies

Estimating the volume of a convex body is a central problem in convex geometry and can be viewed as a continuous version of counting. We present a quantum algorithm that estimates the volume of an n -dimensional convex body within multiplicative error ε using Õ(n 3 + n 2.5 /ε ) queries to a membership oracle and Õ(n 5 +n 4.5 /ε) additional arithmetic operations. For comparison, the best known classical algorithm uses Õ(n 3.5 +n 3 /ε 2 ) queries and Õ(n 5.5 +n 5 /ε 2 ) additional arithmetic operations. To the best of our knowledge, this is the first quantum speedup for volume estimation. Our algorithm is based on a refined framework for speeding up simulated annealing algorithms that might be of independent interest. This framework applies in the setting of “Chebyshev cooling,” where the solution is expressed as a telescoping product of ratios, each having bounded variance. We develop several novel techniques when implementing our framework, including a theory of continuous-space quantum walks with rigorous bounds on discretization error. To complement our quantum algorithms, we also prove that volume estimation requires Ω (√ n+1/ε) quantum membership queries, which rules out the possibility of exponential quantum speedup in n and shows optimality of our algorithm in 1/ε up to poly-logarithmic factors.
An efficient privacy-preserving blockchain storage method for internet of things environment

Blockchain is a key technology to realize decentralized trust management. In recent studies, sharding-based blockchain models are proposed and applied to the resource-constrained Internet of Things (IoT) scenario, and machine learning-based models are presented to improve the query efficiency of the sharding-based blockchains by classifying hot data and storing them locally. However, in some scenarios, these presented blockchain models cannot be deployed because the block features used as input in the learning method are privacy. In this paper, we propose an efficient privacy-preserving blockchain storage method for the IoT environment. The new method classifies hot blocks based on the federated extreme learning machine method and saves the hot blocks through one of the sharded blockchain models called ElasticChain. The features of hot blocks will not be read by other nodes in this method, and user privacy is effectively protected. Meanwhile, hot blocks are saved locally, and data query speed is improved. Furthermore, in order to comprehensively evaluate a hot block, five features of hot blocks are defined, including objective feature, historical popularity, potential popularity, storage requirements and training value. Finally, the experimental results on synthetic data demonstrate the accuracy and efficiency of the proposed blockchain storage model.
Multi-Feature Fusion Method for Chinese Shipping Companies Credit Named Entity Recognition

Shipping Enterprise Credit Named Entity Recognition (NER) aims to recognize shipping enterprise credit entities from unstructured shipping enterprise credit texts. Aiming at the problem of low entity recognition rate caused by complex and diverse entities and nesting phenomenon in the field of shipping enterprise credit, a deep learning method based on multi-feature fusion is proposed to improve the recognition effect of shipping enterprise credit entities. In this study, the shipping enterprise credit dataset is manually labeled using the BIO labeling model, combining the pre-trained model Bidirectional Encoder Representations from Transformers (BERT) and bidirectional gated recurrent unit (BiGRU) with conditional random field (CRF) to form the BERT-BiGRU-CRF model, and changing the input of the model from a single feature vector to a multi-feature vector (MF) after stitching character vector features, word vector features, word length features, and part-of-speech (pos) features; BiGRU is introduced to extract the contextual features of shipping enterprise credit texts. Finally, CRF completes the sequence annotation task. According to the experimental results, using the BERT-MF-BiGRU-CRF model for NER of shipping enterprise credit text data, the F1 Score (F1) reaches 91.7%, which is 8.37% higher than the traditional BERT-BiGRU-CRF model. The experimental results show that the BERT-MF-BiGRU-CRF model can effectively perform NER for shipping enterprise credit text data, which is helpful to construct a credit knowledge graph for shipping enterprises, while the research results can provide references for complex entities and nested entities recognition in other fields.
HeMGNN: Heterogeneous Network Embedding Based on a Mixed Graph Neural Network

Network embedding is an effective way to realize the quantitative analysis of large-scale networks. However, mainstream network embedding models are limited by the manually pre-set metapaths, which leads to the unstable performance of the model. At the same time, the information from homogeneous neighbors is mostly focused in encoding the target node, while ignoring the role of heterogeneous neighbors in the node embedding. This paper proposes a new embedding model, HeMGNN, for heterogeneous networks. The framework of the HeMGNN model is divided into two modules: the metapath subgraph extraction module and the node embedding mixing module. In the metapath subgraph extraction module, HeMGNN automatically generates and filters out the metapaths related to domain mining tasks, so as to effectively avoid the excessive dependence of network embedding on artificial prior knowledge. In the node embedding mixing module, HeMGNN integrates the information of homogeneous and heterogeneous neighbors when learning the embedding of the target nodes. This makes the node vectors generated according to the HeMGNN model contain more abundant topological and semantic information provided by the heterogeneous networks. The Rich semantic information makes the node vectors achieve good performance in downstream domain mining tasks. The experimental results show that, compared to the baseline models, the average classification and clustering performance of HeMGNN has improved by up to 0.3141 and 0.2235, respectively.
Accurate defocusing fringe projection profilometry in a large depth-of-field

Defocusing fringe projection profilometry (DFPP) has been one of the most popular 3-D measurement techniques. The measurement error caused by the low-contrast patterns becomes non-ignorable in large depth-of-field (DoF) DFPP. Traditional methods sacrifice the measurement speed, 3-D details or ignore the influence of the projector defocusing, which limit the performance of extending the system DoF. In this paper, a deep learning-based fringe-enhancing method (DFEM) is proposed, which transforms three patterns with different phase shifts captured at a fixed focal length into the desired phase. DFEM divides multiple sub-DoFs for reducing the difficulty of pattern transformation in the training. In the testing, the geometric constraint is introduced for determining the sub-DoF in which the object is located. DFEM can achieve accurate 3-D reconstruction in a DoF large to 1420 mm, which improve the DoF of DFPP up to 4.7 times of the traditional one. The provided experiments demonstrate the accurateness of the resulted large-DoF DFPP.
Front-End Adapter: Adapting Front-End Input of Speech Based Self-Supervised Learning for Speech Recognition

Recent years have witnessed a boom in self-supervised learning (SSL) in various areas including speech processing. Speech based SSL models present promising performance in a range of speech related tasks. However, the training of SSL models is computationally expensive and a common practice is to fine-tune a released SSL model on the specific task. It is essential to use consistent front-end input during pre-training and fine-tuning. This consistency may introduce potential issues when the optimal front-end is not the same as that used in pre-training. In this paper, we propose a simple but effective front-end adapter to address this front-end discrepancy. By minimizing the distance between the outputs of different front-ends, the filterbank feature (Fbank) can be compatible with SSL models which are pre-trained with waveform. The experiment results demonstrate the effectiveness of our proposed front-end adapter on several popular SSL models for the speech recognition task.
Customized Automatic Face Beautification

In the age of social media, posting attractive mugshots is commonplace, leading to an urgent need for automatic facial beautification techniques. To better meet the esthetic preferences of users, we devise a customized automatic face beautification task that can retouch the face adaptively to match the user-entered target score whilst preserving the ID information as much as possible. To accomplish this task, we propose a Human Esthetics Guided StyleGAN Inversion method to retouch each face in the embedding space using StyleGAN inversion. This process is guided by a pre-trained facial beauty prediction model that measures the difference between the target score and the predicted score of the retouched face. We conduct extensive experiments on various faces with different attributes, where the experimental results show that our method achieves the competitive performance, both in terms of visual effect and the proposed criterion.
Inductive Relation Prediction from Relational Paths and Context with Hierarchical Transformers

Relation prediction on knowledge graphs (KGs) is a key research topic. Dominant embedding-based methods mainly focus on the transductive setting and lack the inductive ability to generalize to new entities for inference. Existing methods for inductive reasoning mostly mine the connections between entities, i.e., relational paths, without considering the nature of head and tail entities contained in the relational context. This paper proposes a novel method that captures both connections between entities and the intrinsic nature of entities, by simultaneously aggregating RElational Paths and cOntext with a unified hieRarchical Transformer framework, namely REPORT. REPORT relies solely on relation semantics and can naturally generalize to the fully-inductive setting, where KGs for training and inference have no common entities. In the experiments, REPORT performs consistently better than all baselines on almost all the eight version subsets of two fully-inductive datasets. Moreover. REPORT is interpretable by providing each element’s contribution to the prediction results.
Fine-Grained Textual Knowledge Transfer to Improve RNN Transducers for Speech Recognition and Understanding

RNN Tranducer (RNN-T) technology is very popular for building deployable models for end-to-end (E2E) automatic speech recognition (ASR) and spoken language understanding (SLU). Since these are E2E models operating on speech directly, there remains a potential to improve their performance using purely text based models like BERT, which have strong language understanding capabilities. In this paper, we propose a new training criteria for RNN-T based E2E ASR and SLU to transfer BERT’s knowledge into these systems. In the first stage of our proposed mechanism, we improve ASR performance by using a fine-grained, tokenwise knowledge transfer from BERT. In the second stage, we fine-tune the ASR model for SLU such that the above knowledge is explicitly utilized by the RNN-T model for improved performance. Our techniques improve ASR performance on the Switchboard and CallHome test sets of the NIST Hub5 2000 evaluation and on the recently released SLURP dataset on which we achieve a new state-of-the-art performance. For SLU, we show significant improvements on the SLURP slot filling task, outperforming HuBERT-base and reaching a performance close to HuBERTlarge. Compared to large transformer based speech models like HuBERT, our model is significantly more compact and uses only 300 hours of speech pretraining data.
Byzantine-Robust and Communication-Efficient Personalized Federated Learning

This paper investigates personalized federated learning, in which a group of workers are coordinated by a server to train correlated local models, in addition to a common global model. This distributed statistical learning problem faces two challenges: efficiency of information exchange between the workers and the server, and robustness to potential malicious messages from the so-called Byzantine workers. We propose a projected stochastic block gradient descent method to address the robustness issue. Therein, each regular worker learns in a personalized manner with the aid of the global model, and the server judiciously aggregates the local models via a Huber function-based descent step. To improve communication efficiency, we allow the regular workers to perform multi-steps of local update per communication round. Convergence of the proposed method is established for non-convex personalized federated learning. Numerical experiments on neural network training validate advantages of the proposed method over the existing ones.
Self-Supervised Adversarial Training for Contrastive Sentence Embedding

The defense against adversarial attacks was originally proposed for computer vision, and recently such an adversarial training (AT) has been emerging for natural language understanding. In an AT process, the adversarial perturbations are added on the input word embeddings as the noisy data which are included to allow the trained model to be noise invariant and accordingly improve the model generalization. However, the performance of existing works was bounded under the supervised or semi-supervised setting. In addition, the contrastive learning (CL) has obtained a significant performance in a self-supervised pre-training for language models. This paper presents a novel method to re-formulate CL to meet a self-supervised classification objective. Using this new formula, a self-supervised AT method is proposed for training an efficient sentence encoder. Experiments show that the pro-posed CL can improve the previous methods to find unsupervised sentence embeddings. With the help of AT, this method further surpasses the previous supervised methods.
Multi-Head Feature Pyramid Networks for Breast Mass Detection

Analysis of X-ray images is one of the main tools to diagnose breast cancer. The ability to quickly and accurately detect the location of masses from the huge amount of image data is the key to reducing the morbidity and mortality of breast cancer. Currently, the main factor limiting the accuracy of breast mass detection is the unequal focus on the mass boxes, leading the network to focus too much on larger masses at the expense of smaller ones. In the paper, we propose the multi-head feature pyramid module (MHFPN) to solve the problem of unbalanced focus of target boxes during feature map fusion and design a multi-head breast mass detection network (MBMDnet). Experimental studies show that, comparing to the SOTA detection baselines, our method improves by 6.58% (in AP@50) and 5.4% (in TPR@50) on the commonly used IN-breast dataset, while about 6-8% improvements (in AP@20) are also observed on the public MIAS and BCS-DBT datasets.
ACE-VC: Adaptive and Controllable Voice Conversion Using Explicitly Disentangled Self-Supervised Speech Representations

In this work, we propose a zero-shot voice conversion method using speech representations trained with self-supervised learning. First, we develop a multi-task model to decompose a speech utterance into features such as linguistic content, speaker characteristics, and speaking style. To disentangle content and speaker representations, we propose a training strategy based on Siamese networks that encourages similarity between the content representations of the original and pitch-shifted audio. Next, we develop a synthesis model with pitch and duration predictors that can effectively reconstruct the speech signal from its decomposed representation. Our framework allows controllable and speaker-adaptive synthesis to perform zero-shot any-to-any voice conversion achieving state-of-the-art results on metrics evaluating speaker similarity, intelligibility, and naturalness. Using just 10 seconds of data for a target speaker, our framework can perform voice swapping and achieves a speaker verification EER of 5.5% for seen speakers and 8.4% for unseen speakers. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Improving Contextual Biasing with Text Injection

In this work, we present a model-based approach to improving contextual biasing that improves quality without drastically increasing model computation during inference. Specifically, we look at injecting text data during training which is representative of contextually-relevant context that will be seen at inference, using a modality-matching text injection method known as JOIST. As JOIST injects text data directly into the E2E model, there is no additional model computation during inference, which is a big difference compared to most model-based biasing techniques. We find that our proposed approach, when combined with an FST-based context model, improves recognition of contacts between 5–15% relative.
Learning Speech Representations with Flexible Hidden Feature Dimensions

Non-parallel many-to-many voice conversion is a kind of style transfer task in speech. Recently, AutoVC has been applied in this field as a popular solution, as it can achieve distribution-matching style transfer by training only the re- construction loss. However, in order to strike a good balance between timbre disentanglement and sound quality, AutoVC requires imposing very strict constraints on the dimensionality of the latent representation. This constraint affects the quality of the converted speech while making it challenging to apply to other datasets directly. This paper proposes a new voice conversion framework that uses only one encoder to obtain timbre and content information by partitioning the latent space in the channel dimension. Furthermore, two different types of classifiers and two additional reconstruction losses are proposed to ensure that different parts of the latent space contain only separated content and timbre information, respectively. Experiments on the VCTK dataset show that the proposed model achieves state-of-the-art results in terms of the naturalness and similarity of converted speech. In addition, we experimentally show that for different division proportions of latent space, the content and timbre information will always be well separated.
ACF: Aligned Contrastive Finetuning For Language and Vision Tasks

Contrastive learning (CL) has achieved great success in various fields with self-supervised learning. However, CL under the supervised setting is not fully explored, especially how to utilize the class labels in CL. We propose a novel aligned contrastive finetuning (ACF) approach in this work. Specifically, we consider the label embeddings as labeled instances and put them in an InfoNCE loss objective together with the instance representations, thus aligning the label embeddings and instance representation in the same semantic space. In addition, we design a correlation-based regularization term to alleviate the anisotropy problem. Extensive experiments are conducted on language understanding and image classification tasks, demonstrating our ACF method’s competitiveness. ACF is off-the-shelf and can be plugged into any pre-trained models without additional network architectures or computation overhead.
Discriminative Speaker Representation Via Contrastive Learning with Class-Aware Attention in Angular Space

The challenges in applying contrastive learning to speaker verification (SV) are that the softmax-based contrastive loss lacks discriminative power and that the hard negative pairs can easily influence learning. To overcome the first challenge, we propose a contrastive learning SV framework incorporating an additive angular margin into the supervised contrastive loss in which the margin improves the speaker representation’s discrimination ability. For the second challenge, we introduce a class-aware attention mechanism through which hard negative samples contribute less significantly to the supervised contrastive loss. We also employed gradient-based multi-objective optimization to balance the classification and contrastive loss. Experimental results on CN-Celeb and Voxceleb1 show that this new learning objective can cause the encoder to find an embedding space that exhibits great speaker discrimination across languages.
Multi-Blank Transducers for Speech Recognition

This paper proposes a modification to RNN-Transducer (RNN-T) models for automatic speech recognition (ASR). In standard RNN-T, the emission of a blank symbol consumes exactly one input frame; in our proposed method, we introduce additional blank symbols, which consume two or more input frames when emitted. We refer to the added symbols as big blanks, and the method multi-blank RNN-T. For training multi-blank RNN-Ts, we propose a novel logit under-normalization method in order to prioritize emissions of big blanks. With experiments on multiple languages and datasets, we show that multi-blank RNN-T methods could bring relative speedups of over +90%/+139% to model inference for English Librispeech and German Multilingual Librispeech datasets, respectively. The multi-blank RNN-T method also improves ASR accuracy consistently. We will release our implementation of the method in the NeMo (https://github.com/NVIDIA/NeMo) toolkit.
Robust Time Series Recovery and Classification Using Test-Time Noise Simulator Networks

Time-series are commonly susceptible to various types of corruption due to sensor-level changes and defects which can result in missing samples, sensor and quantization noise, unknown calibration, unknown phase shifts etc. These corruptions cannot be easily corrected as the noise model may be unknown at the time of deployment. This also results in the inability to employ pre-trained classifiers, trained on (clean) source data. In this paper, we present a general framework and models for time-series that can make use of (unlabeled) test samples to estimate the noise model-entirely at test time. To this end, we use a coupled decoder model and an additional neural network which acts as a learned noise model simulator. We show that the framework is able to "clean" the data so as to match the source training data statistics and the cleaned data can be directly used with a pre-trained classifier for robust predictions. We perform empirical studies on diverse application domains with different types of sensors, clearly demonstrating the effectiveness and generality of this method.
Local Feature Enhanced Adversarial Network for the Blind Image Quality Assessment

As a hot research topic in the field of computer vision, blind image quality assessment (BIQA) can provide high-quality images for end-users and promote the development of other fields of computer vision. Although the existing BIQA based on convolution neural networks has made significant progress in synthetic distortion evaluation, it still cannot be well extended to authentic distortion and algorithm-related distortion. Therefore, this paper proposes a BIQA adversarial network with local feature enhancement to deal with this challenge. First, the ResNeSt50 network with local feature enhancement is used to extract the features of images, which effectively combines the overall semantic information and the local features of the images. Then, the mapping of distorted images to their quality scores is learned by the adversarial network. Extensive experiments demonstrate that the proposed method performs best on three categories of distorted scenario databases (nine databases) compared with state-of-the-art BIQA methods.