Reviewerly/exHarmony huggingface dataset:
An Improved Single Step Non-autoregressive Transformer for Automatic  Speech Recognition

Non-autoregressive mechanisms can significantly decrease inference time for speech transformers, especially when the single step variant is applied. Previous work on CTC alignment-based single step non-autoregressive transformer (CASS-NAT) has shown a large real time factor (RTF) improvement over autoregressive transformers (AT). In this work, we propose several methods to improve the accuracy of the end-to-end CASS-NAT, followed by performance analyses. First, convolution augmented self-attention blocks are applied to both the encoder and decoder modules. Second, we propose to expand the trigger mask (acoustic boundary) for each token to increase the robustness of CTC alignments. In addition, iterated loss functions are used to enhance the gradient update of low-layer parameters. Without using an external language model, the WERs of the improved CASS-NAT, when using the three methods, are 3.1%/7.2% on Librispeech test clean/other sets and the CER is 5.4% on the Aishell1 test set, achieving a 7%~21% relative WER/CER improvement. For the analyses, we plot attention weight distributions in the decoders to visualize the relationships between token-level acoustic embeddings. When the acoustic embeddings are visualized, we find that they have a similar behavior to word embeddings, which explains why the improved CASS-NAT performs similarly to AT.
Enriching Source Style Transfer in Recognition-Synthesis based  Non-Parallel Voice Conversion

Current voice conversion (VC) methods can successfully convert timbre of the audio. As modeling source audio's prosody effectively is a challenging task, there are still limitations of transferring source style to the converted speech. This study proposes a source style transfer method based on recognition-synthesis framework. Previously in speech generation task, prosody can be modeled explicitly with prosodic features or implicitly with a latent prosody extractor. In this paper, taking advantages of both, we model the prosody in a hybrid manner, which effectively combines explicit and implicit methods in a proposed prosody module. Specifically, prosodic features are used to explicit model prosody, while VAE and reference encoder are used to implicitly model prosody, which take Mel spectrum and bottleneck feature as input respectively. Furthermore, adversarial training is introduced to remove speaker-related information from the VAE outputs, avoiding leaking source speaker information while transferring style. Finally, we use a modified self-attention based encoder to extract sentential context from bottleneck features, which also implicitly aggregates the prosodic aspects of source speech from the layered representations. Experiments show that our approach is superior to the baseline and a competitive system in terms of style transfer; meanwhile, the speech quality and speaker similarity are well maintained.
Voting for the right answer: Adversarial defense for speaker  verification

Automatic speaker verification (ASV) is a well developed technology for biometric identification, and has been ubiquitous implemented in security-critic applications, such as banking and access control. However, previous works have shown that ASV is under the radar of adversarial attacks, which are very similar to their original counterparts from human's perception, yet will manipulate the ASV render wrong prediction. Due to the very late emergence of adversarial attacks for ASV, effective countermeasures against them are limited. Given that the security of ASV is of high priority, in this work, we propose the idea of "voting for the right answer" to prevent risky decisions of ASV in blind spot areas, by employing random sampling and voting. Experimental results show that our proposed method improves the robustness against both the limited-knowledge attackers by pulling the adversarial samples out of the blind spots, and the perfect-knowledge attackers by introducing randomness and increasing the attackers' budgets.
Learning from Multiple Noisy Partial Labelers

Programmatic weak supervision creates models without hand-labeled training data by combining the outputs of noisy, user-written rules and other heuristic labelers. Existing frameworks make the restrictive assumption that labelers output a single class label. Enabling users to create partial labelers that output subsets of possible class labels would greatly expand the expressivity of programmatic weak supervision. We introduce this capability by defining a probabilistic generative model that can estimate the underlying accuracies of multiple noisy partial labelers without ground truth labels. We prove that this class of models is generically identifiable up to label swapping under mild conditions. We also show how to scale up learning to 100k examples in one minute, a 300X speed up compared to a naive implementation. We evaluate our framework on three text classification and six object classification tasks. On text tasks, adding partial labels increases average accuracy by 9.6 percentage points. On image tasks, we show that partial labels allow us to approach some zero-shot object classification problems with programmatic weak supervision by using class attributes as partial labelers. Our framework is able to achieve accuracy comparable to recent embedding-based zero-shot learning methods using only pre-trained attribute detectors
Dual Attentive Sequential Learning for Cross-Domain Click-Through Rate  Prediction

Cross domain recommender system constitutes a powerful method to tackle the cold-start and sparsity problem by aggregating and transferring user preferences across multiple category domains. Therefore, it has great potential to improve click-through-rate prediction performance in online commerce platforms having many domains of products. While several cross domain sequential recommendation models have been proposed to leverage information from a source domain to improve CTR predictions in a target domain, they did not take into account bidirectional latent relations of user preferences across source-target domain pairs. As such, they cannot provide enhanced cross-domain CTR predictions for both domains simultaneously. In this paper, we propose a novel approach to cross-domain sequential recommendations based on the dual learning mechanism that simultaneously transfers information between two related domains in an iterative manner until the learning process stabilizes. In particular, the proposed Dual Attentive Sequential Learning (DASL) model consists of two novel components Dual Embedding and Dual Attention, which jointly establish the two-stage learning process: we first construct dual latent embeddings that extract user preferences in both domains simultaneously, and subsequently provide cross-domain recommendations by matching the extracted latent embeddings with candidate items through dual-attention learning mechanism. We conduct extensive offline experiments on three real-world datasets to demonstrate the superiority of our proposed model, which significantly and consistently outperforms several state-of-the-art baselines across all experimental settings. We also conduct an online A/B test at a major video streaming platform Alibaba-Youku, where our proposed model significantly improves business performance over the latest production system in the company.
PURS: Personalized Unexpected Recommender System for Improving User  Satisfaction

Classical recommender system methods typically face the filter bubble problem when users only receive recommendations of their familiar items, making them bored and dissatisfied. To address the filter bubble problem, unexpected recommendations have been proposed to recommend items significantly deviating from user's prior expectations and thus surprising them by presenting "fresh" and previously unexplored items to the users. In this paper, we describe a novel Personalized Unexpected Recommender System (PURS) model that incorporates unexpectedness into the recommendation process by providing multi-cluster modeling of user interests in the latent space and personalized unexpectedness via the self-attention mechanism and via selection of an appropriate unexpected activation function. Extensive offline experiments on three real-world datasets illustrate that the proposed PURS model significantly outperforms the state-of-the-art baseline approaches in terms of both accuracy and unexpectedness measures. In addition, we conduct an online A/B test at a major video platform Alibaba-Youku, where our model achieves over 3\% increase in the average video view per user metric. The proposed model is in the process of being deployed by the company.
Open Data in <i>Limnology and Oceanography Letters</i>

Limnology and Oceanography BulletinVolume 31, Issue 3 p. 89-90 ASLO News Open Data in Limnology and Oceanography Letters Carolina C. Barbosa, Corresponding Author Carolina C. Barbosa cbarbosa@uwyo.edu orcid.org/0000-0002-6393-5730 Zoology and Physiology, University of Wyoming, Laramie, Wyoming, USASearch for more papers by this author Carolina C. Barbosa, Corresponding Author Carolina C. Barbosa cbarbosa@uwyo.edu orcid.org/0000-0002-6393-5730 Zoology and Physiology, University of Wyoming, Laramie, Wyoming, USASearch for more papers by this author First published: 24 July 2022 https://doi.org/10.1002/lob.10499Read the full textAboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat No abstract is available for this article. Volume31, Issue3August 2022Pages 89-90 This article also appears in:ASLO’s Career Corner RelatedInformation
Implementasi Deteksi Judul Berita Clickbait Berbahasa Indonesia dengan pre-trained model Multilingual BERT Pada Aplikasi Berbasis Chrome Extension

Clickbait news title is often used by online news portal. The purpose of clickbait is to attract reader to open and read the news. Furthermore, news containing clickbait title can give negative impact by reducing the essence of important news. Therefore, clickbait detection tool is needed to avoid the clickbait news title. Chrome extension was chosen in this study because it supports all Chrome based browsers, such as Google Chrome, Chromium, Microsoft Edge, and Opera so that many users apply this program. In this study, Chrome extension-based application was designed and integrated by using artificial intelligence model. This application also utilized the availability of pre-trained multilingual BERT model as Natural Language Processing (NLP) which will be used to predict a clickbait news title. This study used Multilingual BERT model as NLP because this model has been trained into 104 languages, including Bahasa Indonesia and it has significant performance. The result of this study can detect clickbait news along with 92% of AUC-ROC value.
Analysis of the Implementation Practice of National English Curriculum Standards for Common Senior High School from the Perspective of Teacher Change

It has been five years since the Ministry of Education in People’s Republic of China issued the National English Curriculum Standards for Common Senior High School, an educational policy that covers all provinces and cities in Mainland China. This paper, from the perspective of teacher change, conducts analysis in the implementation of this curriculum policy and discuss how to facilitate this educational change from school level and government level respectively.
Empirical Study on the imPact of Green Culture on Corporate Financial Risk

Based on the perspective of corporate culture,this study takes 464 A-share listed companies in China’s heavy pollution industries from 2015 to 2019 as samples.The panel regression model is used to explore the impact of green culture on corporate financial risk and the mediating role of environmental information disclosure.The results show that green culture can significantly reduce corporate financial risk,and environmental information disclosure has a partial mediation effect on the relationship between green culture and corporate financial risk.
A Study of the Cognitive Mechanism of the Chinese Network Discourse Constructions

Language used in network features with active vertility and rapid change. By using the framework of discourse construction grammar analysis established by Yuan Ye, we analyze the discourse construction that have changed in form and function, tend to find out the motivation and the cognitive mechanism behind the variant constructions. Research invastigated that the factor of cognitive perception should be added to the discourse construal grammar analysis framework; (2) phonological coersion is the cognitive mechanism behind the variation of network language.
Teaching of Calculation methods Course Based on a Simple to Complex Perspective

Starting from simple knowledge in teaching,the transformation of complex methods in computational methods into &#x0D; combinations of simple methods is explained,which helps to improve teaching effectiveness.This paper analyzes the application &#x0D; of the principle of moving from simple to complex in the teaching of computational methods through specific cases based on the &#x0D; perspective of moving from simple to complex,and introduces the embodiment of the property of transforming complex problems &#x0D; into combinations of simple problems in computational methods based on specific knowledge points.The paper will play an active &#x0D; role in the teaching of computational methods course.
Information Shift in E-C Translation of Science and Technology

The essence of translation is to convert the information of one language into another. In this process, there exist great &#x0D; difficulties because of language differences. Therefore, translators should consider the readability of translation and various &#x0D; compensatory measures should be taken to shift the information of source text to better convey its information to the readers. This &#x0D; paper mainly discusses information shift in E-C translation of science and technology.
Practical Research on Music Teaching Reform of Higher Vocational Preschool Education Specialty Under New Situation

With the continuous advancement of curriculum reform, preschool education stage, music has a profound enlightenment &#x0D; effect on students, in addition to let students get constantly rich experience, but also edify sentiment, add fun to the dull learning &#x0D; life. This paper mainly expounds the necessity of music teaching reform of higher vocational preschool education specialty under &#x0D; the new situation, analyzes the problems of music teaching reform of current higher vocational preschool education specialty, and &#x0D; puts forward the measures of music teaching reform of higher vocational preschool education specialty under the new
THE EFFECT OF SCHOOL PROMOTION ON THE DECISIONS OF PROSPECTIVE NEW STUDENTS IN THE SELECTION OF SMAK ST. PETER KEWAPANTE

This study aims to find the effect of school promotion on the decision to choose. This research was conducted at SMAK St. Petrus Kewapante in the semester 2nd of the school year 2020-2021. The research population is 58 students. The technique used in this research is a saturated sampling technique whit total of 51 students. The method used in this research is descriptive quantitative. All statements are valid, reliable, normal, linear and non- heteroscedastic. Based on the results of simple linear regression calculations obtained that the decision to choose is 18.796 + 0.364 promotion. The results of hypothesis testing using t-test calculation obtained that the value of calculated-t is greater than the value of tabled-t (3,805 &gt; 2,00). This means H0 is rejected and H1 is accepted. This result indicates that the coefficient of the determination (Adjusted R Square) of the indpendent variable on the dependent variable is Adjusted R Square 0,212 or 21,2%. Therefore, it can be concluded that there is a relationship between promotion and the decision to choose new students in SMAK St. Petrus Kewapante
THE EFFICACY OF CHAIN PICTURES IN TRAINING EFL STUDENTS TO WRITE NARRATIVE PARAGRAPHS

Writing skill is very crucial in learning the language. However, it is the most challenging skill for learners because of the complexity of various nuances. Learners find it is hard to produce well-written essays, primarily narrative paragraphs. Therefore, this study aimed to discover the efficacy of using chain pictures in writing narrative paragraphs to Indonesian EFL students. This study population was the second year students of Senior High School 4 Bulukumba, South-Sulawesi, Indonesia. Thirty students were selected as the sample. The data were collected through the pre-experimental method by using pre and post-test design. The result showed that the students' writing skills were significantly improved after training them by using chain pictures. The analysis result shows that students' mean score in the pre-test was 56,87, categorized as poor writing skills. In contrast, the students' mean score in the post-test was 74,13, classified as good writing skill of narrative paragraph. The t-test value was also higher than the t-table value, which was 8,25 &gt; 2.045. Thus, it is recommended to use chain pictures in the English learning environment, especially in teaching EFL students to write narrative paragraphs.
Unsupervised Part Segmentation through Disentangling Appearance and  Shape

We study the problem of unsupervised discovery and segmentation of object parts, which, as an intermediate local representation, are capable of finding intrinsic object structure and providing more explainable recognition results. Recent unsupervised methods have greatly relaxed the dependency on annotated data which are costly to obtain, but still rely on additional information such as object segmentation mask or saliency map. To remove such a dependency and further improve the part segmentation performance, we develop a novel approach by disentangling the appearance and shape representations of object parts followed with reconstruction losses without using additional object mask information. To avoid degenerated solutions, a bottleneck block is designed to squeeze and expand the appearance representation, leading to a more effective disentanglement between geometry and appearance. Combined with a self-supervised part classification loss and an improved geometry concentration constraint, we can segment more consistent parts with semantic meanings. Comprehensive experiments on a wide variety of objects such as face, bird, and PASCAL VOC objects demonstrate the effectiveness of the proposed method.
Improved OOD Generalization via Adversarial Training and Pre-training

Recently, learning a model that generalizes well on out-of-distribution (OOD) data has attracted great attention in the machine learning community. In this paper, after defining OOD generalization via Wasserstein distance, we theoretically show that a model robust to input perturbation generalizes well on OOD data. Inspired by previous findings that adversarial training helps improve input-robustness, we theoretically show that adversarially trained models have converged excess risk on OOD data, and empirically verify it on both image classification and natural language understanding tasks. Besides, in the paradigm of first pre-training and then fine-tuning, we theoretically show that a pre-trained model that is more robust to input perturbation provides a better initialization for generalization on downstream OOD data. Empirically, after fine-tuning, this better-initialized model from adversarial pre-training also has better OOD generalization.
Pre-trained Language Model based Ranking in Baidu Search

As the heart of a search engine, the ranking system plays a crucial role in satisfying users' information demands. More recently, neural rankers fine-tuned from pre-trained language models (PLMs) establish state-of-the-art ranking effectiveness. However, it is nontrivial to directly apply these PLM-based rankers to the large-scale web search system due to the following challenging issues:(1) the prohibitively expensive computations of massive neural PLMs, especially for long texts in the web-document, prohibit their deployments in an online ranking system that demands extremely low latency;(2) the discrepancy between existing ranking-agnostic pre-training objectives and the ad-hoc retrieval scenarios that demand comprehensive relevance modeling is another main barrier for improving the online ranking system;(3) a real-world search engine typically involves a committee of ranking components, and thus the compatibility of the individually fine-tuned ranking model is critical for a cooperative ranking system. In this work, we contribute a series of successfully applied techniques in tackling these exposed issues when deploying the state-of-the-art Chinese pre-trained language model, i.e., ERNIE, in the online search engine system. We first articulate a novel practice to cost-efficiently summarize the web document and contextualize the resultant summary content with the query using a cheap yet powerful Pyramid-ERNIE architecture. Then we endow an innovative paradigm to finely exploit the large-scale noisy and biased post-click behavioral data for relevance-oriented pre-training. We also propose a human-anchored fine-tuning strategy tailored for the online ranking system, aiming to stabilize the ranking signals across various online components. Extensive offline and online experimental results show that the proposed techniques significantly boost the search engine's performance.
Unsupervised Multi-Target Domain Adaptation for Acoustic Scene  Classification

It is well known that the mismatch between training (source) and test (target) data distribution will significantly decrease the performance of acoustic scene classification (ASC) systems. To address this issue, domain adaptation (DA) is one solution and many unsupervised DA methods have been proposed. These methods focus on a scenario of single source domain to single target domain. However, we will face such problem that test data comes from multiple target domains. This problem can be addressed by producing one model per target domain, but this solution is too costly. In this paper, we propose a novel unsupervised multi-target domain adaption (MTDA) method for ASC, which can adapt to multiple target domains simultaneously and make use of the underlying relation among multiple domains. Specifically, our approach combines traditional adversarial adaptation with two novel discriminator tasks that learns a common subspace shared by all domains. Furthermore, we propose to divide the target domain into the easy-to-adapt and hard-to-adapt domain, which enables the system to pay more attention to hard-to-adapt domain in training. The experimental results on the DCASE 2020 Task 1-A dataset and the DCASE 2019 Task 1-B dataset show that our proposed method significantly outperforms the previous unsupervised DA methods.