pwc-archive/papers-with-abstracts huggingface dataset:
Employing low-pass filtered temporal speech features for the training of ideal ratio mask in speech enhancement
Authors: Yan-Tong Chen, Zi-Qiang Lin, Jeih-weih Hung
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Speech Enhancement, speech-recognition, Speech Recognition

The masking-based speech enhancement method pursues a multiplicative mask that applies to the spectrogram of input noise-corrupted utterance, and a deep neural network (DNN) is often used to learn the mask. In particular, the features commonly used for automatic speech recognition can serve as the input of the DNN to learn the well-behaved mask that significantly reduce the noise distortion of processed utterances. This study proposes to preprocess the input speech features for the ideal ratio mask (IRM)-based DNN by lowpass filtering in order to alleviate the noise components. In particular, we employ the discrete wavelet transform (DWT) to decompose the temporal speech feature sequence and scale down the detail coefficients, which correspond to the high-pass portion of the sequence. Preliminary experiments conducted on a subset of TIMIT corpus reveal that the proposed method can make the resulting IRM achieve higher speech quality and intelligibility for the babble noise-corrupted signals compared with the original IRM, indicating that the lowpass filtered temporal feature sequence can learn a superior IRM network for speech enhancement.
Hidden Advertorial Detection on Social Media in Chinese
Authors: Meng-Ching Ho, Ching-Yun Chuang, Yi-Chun Hsu, Yu-Yun Chang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10

Nowadays, there are a lot of advertisements hiding as normal posts or experience sharing in social media. There is little research of advertorial detection on Mandarin Chinese texts. This paper thus aimed to focus on hidden advertorial detection of online posts in Taiwan Mandarin Chinese. We inspected seven contextual features based on linguistic theories in discourse level. These features can be further grouped into three schemas under the general advertorial writing structure. We further implemented these features to train a multi-task BERT model to detect advertorials. The results suggested that specific linguistic features would help extract advertorials.
Automatic Extraction of English Grammar Pattern Correction Rules
Authors: Kuan-Yu Shen, Yi-Chien Lin, Jason S. Chang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10

We introduce a method for generating error-correction rules for grammar pattern errors in a given annotated learner corpus. In our approach, annotated edits in the learner corpus are converted into edit rules for correcting common writing errors. The method involves automatic extraction of grammar patterns, and automatic alignment of the erroneous patterns and correct patterns. At run-time, grammar patterns are extracted from the grammatically correct sentences, and correction rules are retrieved by aligning the extracted grammar patterns with the erroneous patterns. Using the proposed method, we generate 1,499 high-quality correction rules related to 232 headwords. The method can be used to assist ESL students in avoiding grammatical errors, and aid teachers in correcting students’ essays. Additionally, the method can be used in the compilation of collocation error dictionaries and the construction of grammar error correction systems.
Multi-Label Classification of Chinese Humor Texts Using Hypergraph Attention Networks
Authors: Hao-Chuan Kao, Man-Chen Hung, Lung-Hao Lee, Yuen-Hsien Tseng
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

We use Hypergraph Attention Networks (HyperGAT) to recognize multiple labels of Chinese humor texts. We firstly represent a joke as a hypergraph. The sequential hyperedge and semantic hyperedge structures are used to construct hyperedges. Then, attention mechanisms are adopted to aggregate context information embedded in nodes and hyperedges. Finally, we use trained HyperGAT to complete the multi-label classification task. Experimental results on the Chinese humor multi-label dataset showed that HyperGAT model outperforms previous sequence-based (CNN, BiLSTM, FastText) and graph-based (Graph-CNN, TextGCN, Text Level GNN) deep learning models.
Incorporating Domain Knowledge into Language Transformers for Multi-Label Classification of Chinese Medical Questions
Authors: Po-Han Chen, Yu-Xiang Zeng, Lung-Hao Lee
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

In this paper, we propose a knowledge infusion mechanism to incorporate domain knowledge into language transformers. Weakly supervised data is regarded as the main source for knowledge acquisition. We pre-train the language models to capture masked knowledge of focuses and aspects and then fine-tune them to obtain better performance on the downstream tasks. Due to the lack of publicly available datasets for multi-label classification of Chinese medical questions, we crawled questions from medical question/answer forums and manually annotated them using eight predefined classes: persons and organizations, symptom, cause, examination, disease, information, ingredient, and treatment. Finally, a total of 1,814 questions with 2,340 labels. Each question contains an average of 1.29 labels. We used Baidu Medical Encyclopedia as the knowledge resource. Two transformers BERT and RoBERTa were implemented to compare performance on our constructed datasets. Experimental results showed that our proposed model with knowledge infusion mechanism can achieve better performance, no matter which evaluation metric including Macro F1, Micro F1, Weighted F1 or Subset Accuracy were considered.
What confuses BERT? Linguistic Evaluation of Sentiment Analysis on Telecom Customer Opinion
Authors: Cing-Fang Shih, Yu-Hsiang Tseng, Ching-Wen Yang, Pin-Er Chen, Hsin-Yu Chou, Lian-Hui Tan, Tzu-Ju Lin, Chun-Wei Wang, Shu-Kai Hsieh
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Sentence, Sentiment Analysis

Ever-expanding evaluative texts on online forums have become an important source of sentiment analysis. This paper proposes an aspect-based annotated dataset consisting of telecom reviews on social media. We introduce a category, implicit evaluative texts, impevals for short, to investigate how the deep learning model works on these implicit reviews. We first compare two models, BertSimple and BertImpvl, and find that while both models are competent to learn simple evaluative texts, they are confused when classifying impevals. To investigate the factors underlying the correctness of the model’s predictions, we conduct a series of analyses, including qualitative error analysis and quantitative analysis of linguistic features with logistic regressions. The results show that local features that affect the overall sentential sentiment confuse the model: multiple target entities, transitional words, sarcasm, and rhetorical questions. Crucially, these linguistic features are independent of the model’s confidence measured by the classifier’s softmax probabilities. Interestingly, the sentence complexity indicated by syntax-tree depth is not correlated with the model’s correctness. In sum, this paper sheds light on the characteristics of the modern deep learning model and when it might need more supervision through linguistic evaluations.
Generative Adversarial Networks based on Mixed-Attentions for Citation Intent Classification in Scientific Publications
Authors: Yuh-Shyang Wang, Chao-Yi Chen, Lung-Hao Lee
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Citation Intent Classification, Classification, Generative Adversarial Network, intent-classification, Intent Classification, Language Modeling, Language Modelling, Multi-class Classification

We propose the mixed-attention-based Generative Adversarial Network (named maGAN), and apply it for citation intent classification in scientific publication. We select domain-specific training data, propose a mixed-attention mechanism, and employ generative adversarial network architecture for pre-training language model and fine-tuning to the downstream multi-class classification task. Experiments were conducted on the SciCite datasets to compare model performance. Our proposed maGAN model achieved the best Macro-F1 of 0.8532.
A Corpus for Dimensional Sentiment Classification on YouTube Streaming Service
Authors: Ching-Wen Hsu, Chun-Lin Chou, Hsuan Liu, Jheng-Long Wu
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Sentiment Analysis, Sentiment Classification

The streaming service platform such as YouTube provides a discussion function for audiences worldwide to share comments. YouTubers who upload videos to the YouTube platform want to track the performance of these uploaded videos. However, the present analysis functions of YouTube only provide a few performance indicators such as average view duration, browsing history, variance in audience’s demographics, etc., and lack of sentiment analysis on the audience’s comments. Therefore, the paper proposes multi-dimensional sentiment indicators such as YouTuber preference, Video preferences, and Excitement level to capture comprehensive sentiment on audience comments for videos and YouTubers. To evaluate the performance of different classifiers, we experiment with deep learning-based, machine learning-based, and BERT-based classifiers to automatically detect three sentiment indicators of an audience’s comments. Experimental results indicate that the BERT-based classifier is a better classification model than other classifiers according to F1-score, and the sentiment indicator of Excitement level is quite an improvement. Therefore, the multiple sentiment detection tasks on the video streaming service platform can be solved by the proposed multi-dimensional sentiment indicators accompanied with BERT classifier to gain the best result.
Improved Text Classification of Long-term Care Materials
Authors: Yi Fan Chiang, Chi-Ling Lee, Heng-Chia Liao, Yi-Ting Tsai, Yu-Yun Chang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Classification, regression, text-classification, Text Classification

Aging populations have posed a challenge to many countries including Taiwan, and with them come the issue of long-term care. Given the current context, the aim of this study was to explore the hotly-discussed subtopics in the field of long-term care, and identify its features through NLP. This study applied TF-IDF, the Logistic Regression model, and the Naive Bayes classifier to process data. In sum, the results showed that it reached a best F1-score of 0.920 in identification, and a best accuracy of 0.708 in classification. The results of this study could be used as a reference for future long-term care related applications.
Learning to Find Translation of Grammar Patterns in Parallel Corpus
Authors: Kai-Wen Tuan, Yi-Jyun Chen, Yi-Chien Lin, Chun-Ho Kwok, Hai-Lun Tu, Jason S. Chang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Translation

We introduce a method for assisting English as Second Language (ESL) learners by providing translations of Collins COBUILD grammar patterns(GP) for a given word. In our approach, bilingual parallel corpus is transformed into bilingual GP pairs aimed at providing native language support for learning word usage through GPs. The method involves automatically parsing sentences to extract GPs, automatically generating translation GP pairs from bilingual sentences, and automatically extracting common bilingual GPs. At run-time, the target word is used for lookup GPs and translations, and the retrieved common GPs and their example sentences are shown to the user. We present a prototype phrase search engine, Linggle GPTrans, that implements the methods to assist ESL learners. Preliminary evaluation on a set of more than 300 GP-translation pairs shows that the methods achieve 91% accuracy.
Keyword-centered Collocating Topic Analysis
Authors: Yu-Lin Chang, Yongfu Liao, Po-Ya Angela Wang, Mao-Chang Ku, Shu-Kai Hsieh
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Word Embeddings

The rapid flow of information and the abundance of text data on the Internet have brought about the urgent demand for the construction of monitoring resources and techniques used for various purposes. To extract facets of information useful for particular domains from such large and dynamically growing corpora requires an unsupervised yet transparent ways of analyzing the textual data. This paper proposed a hybrid collocation analysis as a potential method to retrieve and summarize Taiwan-related topics posted on Weibo and PTT. By grouping collocates of 臺灣 ‘Taiwan’ into clusters of topics via either word embeddings clustering or Latent Dirichlet allocation, lists of collocates can be converted to probability distributions such that distances and similarities can be defined and computed. With this method, we conduct a diachronic analysis of the similarity between Weibo and PTT, providing a way to pinpoint when and how the topic similarity between the two rises or falls. A fine-grained view on the grammatical behavior and political implications is attempted, too. This study thus sheds light on alternative explainable routes for future social media listening method on the understanding of cross-strait relationship.
Confiscation Detection of Criminal Judgment Using Text Classification Approach
Authors: Hsuan-Tzu Shih, Yu-Cheng Chiu, Hsiao-Shih Chen, Jheng-Long Wu
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Classification, text-classification, Text Classification

As the system of confiscation becomes more and more perfect, grasping the distribution of the types of confiscations actually announced by the courts will enable you to understand changing of the trend. In addition to assisting legislators in formulating laws, it can also provide other people with an understanding of the actual operation of the confiscation system. In order to enable artificial intelligence technology to automatically identify the distribution of confiscation, and consumes a lot of manpower and time costs of manual judgment. The purpose of this research is to establish an automated confiscation identification model that can quickly and accurately identify the multiple label categories of confiscation, and provide the needs of all social circles for confiscation information, so as to facilitate subsequent law amendments or discretion. This research uses the first instance criminal cases as the main experimental data. According to the current laws, the confiscation is divided into three categories: contrabands, criminal tools and criminal proceeds, and perform multiple label identification. This research will use Term Frequency–Inverse Document Frequency (TF-IDF) and Word2Vec algorithm as the feature extraction algorithm, with random forest classifier, and CKIPlabBERT pretrained model for training and identification. The experimental results show that under the CKIPlabBERT pretrained model, the best identification effect can be obtained when only use sentences with confiscated words mentioned in the judgment. When the task is case confiscation, the Micro F1 Score can be as high as 96.2716%, and when the task is defendant confiscation, the Micro F1 Score is as high as 95.5478%.
Incorporating speaker embedding and post-filter network for improving speaker similarity of personalized speech synthesis system
Authors: Sheng-Yao Wang, Yi-chin Huang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Speaker Verification, Speech Synthesis, text-to-speech, Text to Speech, Voice Conversion

In recent years, speech synthesis system can generate speech with high speech quality. However, multi-speaker text-to-speech (TTS) system still require large amount of speech data for each target speaker. In this study, we would like to construct a multi-speaker TTS system by incorporating two sub modules into artificial neural network-based speech synthesis system to alleviate this problem. First module is to add speaker embedding into encoding module for generating speech while a large amount of the speech data from target speaker is not necessary. For speaker embedding method, in our study, two main speaker embedding methods, namely speaker verification embedding and voice conversion embedding, are compared to deciding which one is suitable for our personalized TTS system. Second, we substituted the conventional post-net module, which is adopted to enhance the output spectrum sequence, to further improving the speech quality of the generated speech utterance. Here, a post-filter network is used. Finally, experiment results showed that the speaker embedding is useful by adding it into encoding module and the resultant speech utterance indeed perceived as the target speaker. Also, the post-filter network not only improving the speech quality and also enhancing the speaker similarity of the generated speech utterances. The constructed TTS system can generate a speech utterance of the target speaker in fewer than 2 seconds. In the future, we would like to further investigate the controllability of the speaking rate or perceived emotion state of the generated speech.
Identify Bilingual Patterns and Phrases from a Bilingual Sentence Pair
Authors: Yi-Jyun Chen, Hsin-Yun Chung, Jason S. Chang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Machine Translation, Sentence, Translation

This paper presents a method for automatically identifying bilingual grammar patterns and extracting bilingual phrase instances from a given English-Chinese sentence pair. In our approach, the English-Chinese sentence pair is parsed to identify English grammar patterns and Chinese counterparts. The method involves generating translations of each English grammar pattern and calculating translation probability of words from a word-aligned parallel corpora. The results allow us to extract the most probable English-Chinese phrase pairs in the sentence pair. We present a prototype system that applies the method to extract grammar patterns and phrases in parallel sentences. An evaluation on randomly selected examples from a dictionary shows that our approach has reasonably good performance. We use human judge to assess the bilingual phrases generated by our approach. The results have potential to assist language learning and machine translation research.
Extracting Academic Senses: Towards An Academic Writer’s Dictionary
Authors: Hsin-Yun Chung, Li-Kuang Chen, Jason S Chang
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Translation

We present a method for determining intended sense definitions of a given academic word in an academic keyword list. In our approach, the keyword list are converted into unigram of all possible Mandarin translations, intended or not.The method involve converting words in the keyword list into all translations using a bilingual dictionary, computing the unigram word counts of translations, and computing character counts from the word counts. At run-time, each definition (with associated translation) of the given word is scored with word and character counts, and the definition with the highest count is returned. We present a prototype system for the Academic Keyword List to generate definitions and translation for pedagogy purposes. We also experimented with clustering definition embeddings of all words and definitions, and identifying intended sense in favor of embedding in larger clusters.Preliminary evaluation shows promising performance. This endeavor is a step towards creating a full-fledged dictionary from an academic word list.
SCUDS at ROCLING-2021 Shared Task: Using Pretrained Model for Dimensional Sentiment Analysis Based on Sample Expansion Method
Authors: Hsiao-Shih Chen, Pin-Chiung Chen, Shao-Cheng Huang, Yu-Cheng Chiu, Jheng-Long Wu
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Prediction, Sentence, Sentiment Analysis

Sentiment analysis has become a popular research issue in recent years, especially on educational texts which is an important problem. According to literature, the similar sentence generation can help the prediction performance of machine learning. Therefore, the process of controlled expansional samples is a key component to prediction models. The paper proposed a sample expansion method which combined part-of-speech filter and similar word finder of Word2Vec. The generate samples have high quality with similar sentiment representation. The DistilBERT pretrained model is used to learn and predict Valence-Arousal scores from the expansion samples. Experimental result displays that the using the expansion samples as training data into prediction model has outperforms original training data without expansion, and obtains 80% mean square error reducing and 28% pearson correlation coefficient increasing.
ntust-nlp-1 at ROCLING-2021 Shared Task: Educational Texts Dimensional Sentiment Analysis using Pretrained Language Models
Authors: Yi-Wei Wang, Wei-Zhe Chang, Bo-Han Fang, Yi-Chia Chen, Wei-Kai Huang, Kuan-Yu Chen
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Sentiment Analysis

This technical report aims at the ROCLING 2021 Shared Task: Dimensional Sentiment Analysis for Educational Texts. In order to predict the affective states of Chinese educational texts, we present a practical framework by employing pre-trained language models, such as BERT and MacBERT. Several valuable observations and analyses can be drawn from a series of experiments. From the results, we find that MacBERT-based methods can deliver better results than BERT-based methods on the verification set. Therefore, we average the prediction results of several models obtained using different settings as the final output.
ntust-nlp-2 at ROCLING-2021 Shared Task: BERT-based semantic analyzer with word-level information
Authors: Ke-Han Lu, Kuan-Yu Chen
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Sentiment Analysis

In this paper, we proposed a BERT-based dimensional semantic analyzer, which is designed by incorporating with word-level information. Our model achieved three of the best results in four metrics on “ROCLING 2021 Shared Task: Dimensional Sentiment Analysis for Educational Texts”. We conducted a series of experiments to compare the effectiveness of different pre-trained methods. Besides, the results also proofed that our method can significantly improve the performances than classic methods. Based on the experiments, we also discussed the impact of model architectures and datasets.
CYUT at ROCLING-2021 Shared Task: Based on BERT and MacBERT
Authors: Xie-Sheng Hong, Shih-Hung Wu
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: regression, Sentiment Analysis

This paper present a description for the ROCLING 2021 shared task in dimensional sentiment analysis for educational texts. We submitted two runs in the final test. Both runs use the standard regression model. The Run1 uses Chinese version of BERT as the base, and in Run2 we use the early version of MacBERT that Chinese version of RoBERTa-like BERT model, RoBERTa-wwm-ext. Using powerful pre-training model of BERT for text embedding to help train the model.
SoochowDS at ROCLING-2021 Shared Task: Text Sentiment Analysis Using BERT and LSTM
Authors: Ruei-Cyuan Su, Sig-Seong Chong, Tzu-En Su, Ming-Hsiang Su
Date: 2021-10-01 00:00:00
Proceeding: ROCLING 2021 10
Tasks: Prediction, regression, Sentence, Sentiment Analysis

In this shared task, this paper proposes a method to combine the BERT-based word vector model and the LSTM prediction model to predict the Valence and Arousal values in the text. Among them, the BERT-based word vector is 768-dimensional, and each word vector in the sentence is sequentially fed to the LSTM model for prediction. The experimental results show that the performance of our proposed method is better than the results of the Lasso Regression model.