Reviewerly/exHarmony huggingface dataset:
ASR Error Correction with Dual-Channel Self-Supervised Learning

To improve the performance of Automatic Speech Recognition (ASR), it is common to deploy an error correction module at the post-processing stage to correct recognition errors. In this paper, we propose 1) an error correction model, which takes account of both contextual information and phonetic information by dual-channel; 2) a self-supervised learning method for the model. Firstly, an error region detection model is used to detect the error regions of ASR output. Then, we perform dual-channel feature extraction for the error regions, where one channel extracts their contextual information with a pre-trained language model, while the other channel builds their phonetic information. At the training stage, we construct error patterns at the phoneme level, which simplifies the data annotation procedure, thus allowing us to leverage a large scale of unlabeled data to train our model in a self-supervised learning manner. Experimental results on different test sets demonstrate the effectiveness and robustness of our model.
NER-based military simulation scenario development process

For combat simulation, the simulation scenario serves as the foundation and data source. It is not, however, easy to develop military simulation scenario because the developing process of these texts was time-consuming. To solve this problem, in this paper, we propose a distant supervised method for developing military simulation scenarios based on named entity recognition (NER) method. This method consists of three phases: extracting the key elements of simulation scenario, recognizing named entities of the text, and generating an executable simulation scenario. First, we analyze the two types of scenarios involved in the development process of military simulation scenarios: operational scenario and executable scenario. Second, we train a NER model on operational scenario corpus. Then, we compare our distant supervised-based NER method with the other NER methods, and we achieve an overall improvement of F1 score of 9.01%. Finally, to demonstrate the feasibility of our approach, we use a case study to implement a combat simulation scenario development progress.
DHWP: Learning High-Quality Short Hash Codes Via Weight Pruning

Hashing is widely used in large-scale image retrieval because of its efficiency in storage and computation. Although longer hash codes can lead to higher search accuracy, the retrieval cost increases linearly with the increase of the number of hash bits. Most deep hashing methods suffer from the problem of trivial solutions and usually result in highly correlated redundant hash bits in practice, which limits the performance. To obtain short hash codes with high quality for fast and accurate image retrieval, we propose a novel framework named Deep Hashing via Weight Pruning (DHWP). DHWP first trains the model with relatively long hash codes. Then it obtains shorter codes gradually by weight pruning based on four different criteria. The framework of DHWP can be applied to most deep supervised hashing models, which helps remove those redundant hash bits while retaining the representation ability of long hash codes. Extensive experimental results on two widely used benchmark datasets show that DHWP outperforms the existing state-of-the-art methods, especially for short hash codes.
Learning self-supervised task progression metrics: a case of cloth folding

An important challenge for smart manufacturing systems is finding relevant metrics that capture task quality and progression for process monitoring to ensure process reliability and safety. Data-driven process metrics construct features and labels from abundant raw process data, which incurs costs and inaccuracies due to the labelling process. In this work, we circumvent expensive process data labelling by distilling the task intent from video demonstrations. We present a method to express the task intent in the form of a scalar value by aligning a self-supervised learned embedding to a small set of high-quality task demonstrations. We evaluate our method on the challenging case of monitoring the progress of people folding clothing. We demonstrate that our approach effectively learns to represent task progression without manually labelling sub-steps or progress in the videos. Using case-based experiments, we find that our method learns task-relevant features and useful invariances, making it robust to noise, distractors and variations in the task and shirts. The experimental results show that the proposed method can monitor processes in domains where state representation is inherently challenging.
Data Augmentation for Long-Tailed and Imbalanced Polyphone Disambiguation in Mandarin

Polyphone disambiguation is an important module in Mandarin Chinese text-to-speech (TTS). Recently, neural-network-based (NN-based) models have achieved a great improvement on poly-phone disambiguation. However, a long-tailed and imbalanced distribution is usually observed in the training data of polyphone disambiguation, resulting in an unsatisfying performance on the low-frequent polyphone in the imbalanced pinyin set, and the least-frequent polyphonic characters and polyphones. In this paper, we proposed a simple data-augmentation method based on the pre-trained mask language model BERT to mitigate the long-tailed and imbalanced distribution problem. We incorporate a weighted sampling technique in the data augmentation method to balance the data distribution, and a useful filtering strategy to remove some noisy augmented data. Experimental results show that the proposed data-augmentation method can improve the prediction accuracy, especially for those low-frequent polyphone in the imbalanced pinyin set, and the least-frequent polyphonic characters and polyphones.
Learning persona-driven personalized sentimental representation for review-based recommendation

A large amount of information exists in many e-commerce and review websites as a valuable source for recommender systems. Recent solutions focus on exploring the correlation between sentiment and textual reviews in the review-based recommendation. However, these studies usually pay less attention to the differences of different users in sentimental expression styles or language usage habits when a user writes reviews. In this work, we argue that the individual reviewing behavior is closely related to personality, and sentimental expression is a manifestation of personality. Therefore, we propose a novel Persona-driven Sentimental Attentive Recommendation model (named PSAR) via personalized sentimental interactive representation learning for the review-based recommendation. The proposed model is devised to learn fragment-level and sequence-level personalized sentimental representation simultaneously from reviews. Besides, an attentive persona-driven interaction module is designed to capture word-level usage habits and sentence-level analogous tones. Comprehensive experimental results on four real-world datasets demonstrate that our model outperforms the state-of-the-art methods.
Learning Acoustic Frame Labeling for Phoneme Segmentation with Regularized Attention Mechanism

Phoneme segmentation plays an important role in various speech processing applications such as keyword spotting, automatic pronunciation assessment, and automatic speech recognition. In this paper, we propose a method for phoneme segmentation based on a regularized attention mechanism. Specifically, the representations of speech utterance for each frame are extracted from a pre-trained acoustic encoder and combined with presumed phoneme sequences based on the attention mechanism. By fusing acoustic representations with these aligned phoneme representations, we learn phoneme labeling for each frame to obtain final segmentation. For better alignment between the pronounced phoneme sequence and utterance, we regularize the attention matrix utilizing an extra attention loss. The whole network is optimized by a multi-task learning framework (MTL). Experimental results based on the TIMIT and Buckeye corpora show the proposed method is superior to the previous baselines and reaches the state-of-the-art (SOTA) performance in F1 score and R-value.
Vocbench: A Neural Vocoder Benchmark for Speech Synthesis

Neural vocoders, used for converting the spectral representations of an audio signal to the waveforms, are a commonly used component in speech synthesis pipelines. It focuses on synthesizing waveforms from low-dimensional representation, such as Mel-Spectrograms. In recent years, different approaches have been introduced to develop such vocoders. However, it becomes more challenging to assess these new vocoders and compare their performance to previous ones. To address this problem, we present VocBench, a framework that benchmark the performance of state-of-the-art neural vocoders. VocBench uses a systematic study to evaluate different neural vocoders in a shared environment that enables a fair comparison between them. In our experiments, we use the same setup for datasets, training pipeline, and evaluation metrics for all neural vocoders. We perform a subjective and objective evaluation to compare the performance of each vocoder along a different axis. Our results demonstrate that the framework can show competitive efficacy and quality of the synthesized samples for each vocoder. VocBench framework is available at https://github.com/facebookresearch/vocoder-benchmark.
A new approach to investigate the hydration process and the effect of algae powder on the strength properties of cement paste

Sargassum muticum, a Japanese brown seaweed, has recently invaded various shorelines throughout the world, including the Atlantic coast of Morocco, where it has developed significant populations. We report on the use of these algae as a partial replacement of Portland cement in cement paste as part of a long-term plan to combat this invasive seaweed. With a fixed water to cement ratio (W/C = 0.45), algae powder is added at 5%, 10%, 15% and 20% from the cement content in producing cement pastes. The hydration process was assessed using impedance spectroscopy throughout the setting period from 10 min to 28 days. Compressive strength tests were performed at 7, 14 and 28 days. An equivalent circuit model has been proposed. The results showed that the various strength properties of the cement paste increased or decreased with addition of algae powder. The compressive strength tends to increase with the addition of algae powder. This study shows that 10% addition of algae powder to cement paste showed an increase in compressive strength and when the addition increased to 20% the properties started to decrease.
On monomial generalized almost perfect nonlinear functions

Generalized almost perfect nonlinear (GAPN) functions are a generalization of APN functions to finite fields of odd characteristic p introduced in 2017 by Kuroda and Tsujie. In this paper we deal with GAPN functions of monomial type. To this aim, we connect the GAPN property for a monomial function over Fpn to the existence of suitable rational points of an algebraic curve defined over Fpn. We give necessary conditions for a monomial function to be GAPN, providing the converse of recent results by Özbudak and Sălăgean and by Zha, Hu and Zhang.
Drug Repositioning Using Temporal Trajectories of Accompanying Comorbidities in Diabetes Mellitus

Most studies of systematic drug repositioning have used drug-oriented data such as chemical structures, gene expression patterns, and adverse effect profiles. As it is often difficult to prove repositioning candidates' effectiveness in real-world clinical settings, we used patient-centered real-world data for screening repositioning candidate drugs for multiple diseases simultaneously, especially for diabetic complications.Using the National Health Insurance Service-National Sample Cohort (2002 to 2013), we analyzed claims data of 43,048 patients with type 2 diabetes mellitus (age ≥40 years). To find repositioning candidate disease-drug pairs, a nested case-control study was used for 29 pairs of diabetic complications and the drugs that met our criteria. To validate this study design, we conducted an external validation for a selected candidate pair using electronic health records.We found 24 repositioning candidate disease-drug pairs. In the external validation study for the candidate pair cerebral infarction and glycopyrrolate, we found that glycopyrrolate was associated with decreased risk of cerebral infarction (hazard ratio, 0.10; 95% confidence interval, 0.02 to 0.44).To reduce risks of diabetic complications, it would be possible to consider these candidate drugs instead of other drugs, given the same indications. Moreover, this methodology could be applied to diseases other than diabetes to discover their repositioning candidates, thereby offering a new approach to drug repositioning.
On Anti-Asian Hate as an Ego-Based and Social-Political Force in a Time of COVID-19

The COVID-19 pandemic, which started in late 2019 and continues through the present, has resulted in the scapegoating of Asians globally, according to crime reports and journalistic reportage. This chapter explores the phenomenon of anti-Asian hate both as a personal (ego-level person-related) and social-political (group or population-level) force during a pandemic. This focuses more specifically on the mixed roles of some U.S. leaders and politicians and the communications on the Social Web. In the 2020-2021 timeframe, U.S. political leaders have evoked the “China virus” to garner votes and political contributions and political loyalty, even as the legislature put into place laws against anti-Asian violence and law enforcement stepped up to prevent more race-on-race violence. Concurrently, there were incidences of racial strife expressed on the Social Web. This work is comprised of a review of the journalistic and academic literature and includes the extraction of related social media expressions (such as #stopasianhate and #stopaapihate).
Unified Speculation, Detection, and Verification Keyword Spotting

Accurate and timely recognition of the trigger keyword is vital for a good customer experience on smart devices. In the traditional keyword spotting task, there is typically a trade-off needed between accuracy and latency, where higher accuracy can be achieved by waiting for more context. In this paper, we propose a deep learning model that separates the keyword spotting task into three phases in order to further optimize both accuracy and latency of the overall system. These three tasks are: Speculation, Detection, and Verification. Speculation makes an early decision, which can be used to give a head-start to downstream processes on the device such as local speech recognition. Next, Detection mimics the traditional keyword trigger task and gives a more accurate decision by observing the full keyword context. Finally, Verification verifies previous decision by observing even more audio after the keyword span. We propose a latency-aware max-pooling loss function that can train a unified model for these three tasks by tuning for different latency targets within the same model. In addition, we empirically show that the resultant unified model can accommodate these tasks with desirable performance and without requiring additional compute or memory resources.
Access Control for Privacy-Preserving Gaussian Process Regression

In this paper, we propose access control for privacy-preserving Gaussian process regression (GPR), in which the encrypted data are generated through a random unitary transform (RUT). The proposed secure GPR enables computation in both encrypted input and output domains, and the access to inputs and prediction results can be controlled. We prove that our GPR for encrypted data has the same prediction accuracy as GPR for non-encrypted data. Furthermore, we demonstrate the effectiveness of our method by experimenting with diabetes data from the medical analysis field.
What is Beautiful is Secure

Visual appeal has been shown to influence perceptions of usability and credibility, and we hypothesize that something similar is happening with user judgments of website security: What is beautiful is secure . Web certificates provide reliable information about a website’s level of security, presented in browser interfaces. Users should use this to inform their trust decisions online, but evidence from laboratory studies and real-world usage suggests that they do not. We conducted two studies—one in lab, and one online—in which participants view and interact with websites with high and low visual appeal, and various security levels, and then make security-related judgments. In both studies, participants consistently rated visually appealing websites as more secure, and indicated they would be more likely to enter sensitive information into visually appealing websites—even when they were less secure. Our results provide evidence that users rely on visual appeal when making security and trust decisions on websites. We discuss how these results may be used to help users.
Towards performance portability of AI models using SYCL-DNN

The wide adoption of Deep Neural Networks (DNN) has served as an incentive to design and manufacture powerful and specialized hardware technologies, targeting systems from Edge devices to Cloud and supercomputers. This huge diversity soon becomes a burden due to the emerging dependencies between development stacks and deployment hardware.
Multi-Scale Reinforcement Learning Strategy for Object Detection

Feature Pyramid Network (FPN) has become a common detection paradigm by improving multi-scale features with strong semantics. However, most FPN-based methods typically treat each feature map equally and sum the loss without distinction, which might lead to suboptimal overall performance. In this paper, we propose a Multi-scale Reinforcement Learning Strategy (MRLS) for balanced multi-scale training. First, we design Dynamic Feature Fusion (DFF) to dynamically magnify the impact of more important feature maps in FPN. Second, we introduce Compensatory Scale Training (CST) to enhance the supervision of the under-training scale. We regard the whole detector as a reinforcement learning system while the state bases on multi-scale loss. And we develop the corresponding action, reward, and policy. Compared with adding more rich model architectures, MRLS would not add any extra modules and computational burdens on the baselines. Experiments on MS COCO and PASCAL VOC benchmark demonstrate that our method significantly improves the performance of commonly used object detectors.
HiFiDenoise: High-Fidelity Denoising Text to Speech with Adversarial Networks

Building a high-fidelity speech synthesis system with noisy speech data is a challenging but valuable task, which could significantly reduce the cost of data collection. Existing methods usually train speech synthesis systems based on the speech denoised with an enhancement model or feed noise information as a condition into the system. These methods certainly have some effect on inhibiting noise, but the quality and the prosody of their synthesized speech are still far away from natural speech. In this paper, we propose HiFiDenoise, a speech synthesis system with adversarial networks that can synthesize high-fidelity speech with low-quality and noisy speech data. Specifically, 1) to tackle the difficulty of noise modeling, we introduce multi-length adversarial training in the noise condition module. 2) To handle the problem of inaccurate pitch extraction caused by noise, we remove the pitch predictor in the acoustic model and also add discriminators on the mel-spectrogram generator. 3) In addition, we also apply HiFiDenoise to singing voice synthesis with a noisy singing dataset. Experiments show that our model outperforms the baseline by 0.36 and 0.44 in terms of MOS on speech and singing respectively.
Generic, efficient and isochronous Gaussian sampling over the integers

Abstract Gaussian sampling over the integers is one of the fundamental building blocks of lattice-based cryptography. Among the extensively used trapdoor sampling algorithms, it is ineluctable until now. Under the influence of numerous side-channel attacks, it is still challenging to construct a Gaussian sampler that is generic, efficient, and resistant to timing attacks. In this paper, our contribution is three-fold. First, we propose a secure, efficient exponential Bernoulli sampling algorithm. It can be applied to Gaussian samplers based on rejection samplings. We apply it to FALCON, a candidate of round 3 of the NIST post-quantum cryptography standardization project, and reduce its signature generation time by 13–14%. Second, we develop an isochronous Gaussian sampler based on rejection sampling. Our Algorithm can securely sample from Gaussian distributions with different standard deviations and arbitrary centers. We apply it to PALISADE (S&amp;P 2018), an open-source lattice-based cryptography library. During the online phase of trapdoor sampling, the running time of the G-lattice sampling algorithm is reduced by 44.12% while resisting timing attacks. Third, we improve the efficiency of the COSAC sampler (PQC 2020). The new COSAC sampler is 1.46x–1.63x faster than the original and has the lowest expected number of trials among all Gaussian samplers based on rejection samplings. But it needs a more efficient algorithm sampling from the normal distribution to improve its performance.
Applying Deep Learning to Known-Plaintext Attack on Chaotic Image Encryption Schemes

In this paper, we demonstrate that traditional chaotic encryption schemes are vulnerable to the known-plaintext attack (KPA) with deep learning. Considering the decryption process as image restoration based on deep learning, we apply Convolutional Neural Network to perform known-plaintext attack on chaotic cryptosystems. We design a network to learn the operation mechanism of chaotic cryptosystems, and utilize the trained network as the decryption system. To prove the effectiveness, we select three existing chaotic encryption schemes as the attacked targets. The experimental results demonstrate that deep learning can be applied to known-plaintext attack against chaotic cryptosystems successfully. Compared with traditional attack methods for chaotic cryptosystems, the proposed method shows obvious advantages: (1) One neural network may be applied to cryptanalysis of various chaotic cryptosystems, not limited to specific one; (2) the proposed method is significantly convenient and cost-efficient. This paper provides a new idea for the cryptanalysis of chaotic cryptosystems.