pwc-archive/papers-with-abstracts huggingface dataset:
QCRI @ DSL 2016: Spoken Arabic Dialect Identification Using Textual Features
Authors: Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, Kareem Darwish
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Dialect Identification, Machine Translation, Small Data Image Classification

The paper describes the QCRI submissions to the task of automatic Arabic dialect classification into 5 Arabic variants, namely Egyptian, Gulf, Levantine, North-African, and Modern Standard Arabic (MSA). The training data is relatively small and is automatically generated from an ASR system. To avoid over-fitting on such small data, we carefully selected and designed the features to capture the morphological essence of the different dialects. We submitted four runs to the Arabic sub-task. For all runs, we used a combined feature vector of character bi-grams, tri-grams, 4-grams, and 5-grams. We tried several machine-learning algorithms, namely Logistic Regression, Naive Bayes, Neural Networks, and Support Vector Machines (SVM) with linear and string kernels. However, our submitted runs used SVM with a linear kernel. In the closed submission, we got the best accuracy of 0.5136 and the third best weighted F1 score, with a difference less than 0.002 from the highest score.
Tuning Bayes Baseline for Dialect Detection
Authors: Hector-Hugo Franco-Penya, Liliana Mamani Sanchez
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Domain Adaptation

This paper describes an analysis of our submissions to the Dialect Detection Shared Task 2016. We proposed three different systems that involved simplistic features, to name: a Naive-bayes system, a Support Vector Machines-based system and a Tree Kernel-based system. These systems underperform when compared to other submissions in this shared task, since the best one achieved an accuracy of {\textasciitilde}0.834.
Vanilla Classifiers for Distinguishing between Similar Languages
Authors: Sergiu Nisioi, Alina Maria Ciobanu, Liviu P. Dinu
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Information Retrieval, Language Identification, Question Answering, regression, Task 2

In this paper we describe the submission of the UniBuc-NLP team for the Discriminating between Similar Languages Shared Task, DSL 2016. We present and analyze the results we obtained in the closed track of sub-task 1 (Similar languages and language varieties) and sub-task 2 (Arabic dialects). For sub-task 1 we used a logistic regression classifier with tf-idf feature weighting and for sub-task 2 a character-based string kernel with an SVM classifier. Our results show that good accuracy scores can be obtained with limited feature and model engineering. While certain limitations are to be acknowledged, our approach worked surprisingly well for out-of-domain, social media data, with 0.898 accuracy (3rd place) for dataset B1 and 0.838 accuracy (4th place) for dataset B2.
Simplification of Example Sentences for Learners of Japanese Functional Expressions
Authors: Jun Liu, Yuji Matsumoto
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12

Learning functional expressions is one of the difficulties for language learners, since functional expressions tend to have multiple meanings and complicated usages in various situations. In this paper, we report an experiment of simplifying example sentences of Japanese functional expressions especially for Chinese-speaking learners. For this purpose, we developed {``}Japanese Functional Expressions List{''} and {``}Simple Japanese Replacement List{''}. To evaluate the method, we conduct a small-scale experiment with Chinese-speaking learners on the effectiveness of the simplified example sentences. The experimental results indicate that simplified sentences are helpful in learning Japanese functional expressions.
Effectiveness of Linguistic and Learner Features to Listenability Measurement Using a Decision Tree Classifier
Authors: Katsunori Kotani, Takehiko Yoshimi
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Reading Comprehension

In learning Asian languages, learners encounter the problem of character types that are different from those in their first language, for instance, between Chinese characters and the Latin alphabet. This problem also affects listening because learners reconstruct letters from speech sounds. Hence, special attention should be paid to listening practice for learners of Asian languages. However, to our knowledge, few studies have evaluated the ease of listening comprehension (listenability) in Asian languages. Therefore, as a pilot study of listenability in Asian languages, we developed a measurement method for learners of English in order to examine the discriminability of linguistic and learner features. The results showed that the accuracy of our method outperformed a simple majority vote, which suggests that a combination of linguistic and learner features should be used to measure listenability in Asian languages as well as in English.
Distributed Vector Representations for Unsupervised Automatic Short Answer Grading
Authors: Oliver Adams, Shourya Roy, Raghuram Krishnapuram
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: automatic short answer grading, Reading Comprehension, text similarity, Word Embeddings

We address the problem of automatic short answer grading, evaluating a collection of approaches inspired by recent advances in distributional text representations. In addition, we propose an unsupervised approach for determining text similarity using one-to-many alignment of word vectors. We evaluate the proposed technique across two datasets from different domains, namely, computer science and English reading comprehension, that additionally vary between highschool level and undergraduate students. Experiments demonstrate that the proposed technique often outperforms other compositional distributional semantics approaches as well as vector space methods such as latent semantic analysis. When combined with a scoring scheme, the proposed technique provides a powerful tool for tackling the complex problem of short answer grading. We also discuss a number of other key points worthy of consideration in preparing viable, easy-to-deploy automatic short-answer grading systems for the real-world.
Overview of NLP-TEA 2016 Shared Task for Chinese Grammatical Error Diagnosis
Authors: Lung-Hao Lee, Gaoqi Rao, Liang-Chih Yu, Endong Xun, Baolin Zhang, Li-Ping Chang
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Grammatical Error Correction

This paper presents the NLP-TEA 2016 shared task for Chinese grammatical error diagnosis which seeks to identify grammatical error types and their range of occurrence within sentences written by learners of Chinese as foreign language. We describe the task definition, data preparation, performance metrics, and evaluation results. Of the 15 teams registered for this shared task, 9 teams developed the system and submitted a total of 36 runs. We expected this evaluation campaign could lead to the development of more advanced NLP techniques for educational applications, especially for Chinese error detection. All data sets with gold standards and scoring scripts are made publicly available to researchers.
Chinese Grammatical Error Diagnosis with Long Short-Term Memory Networks
Authors: Bo Zheng, Wanxiang Che, Jiang Guo, Ting Liu
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Information Retrieval, Language Modeling, Language Modelling, Named Entity Recognition (NER), Position

Grammatical error diagnosis is an important task in natural language processing. This paper introduces our Chinese Grammatical Error Diagnosis (CGED) system in the NLP-TEA-3 shared task for CGED. The CGED system can diagnose four types of grammatical errors which are redundant words (R), missing words (M), bad word selection (S) and disordered words (W). We treat the CGED task as a sequence labeling task and describe three models, including a CRF-based model, an LSTM-based model and an ensemble model using stacking. We also show in details how we build and train the models. Evaluation includes three levels, which are detection level, identification level and position level. On the CGED-HSK dataset of NLP-TEA-3 shared task, our system presents the best F1-scores in all the three levels and also the best recall in the last two levels.
Automatic Grammatical Error Detection for Chinese based on Conditional Random Field
Authors: Yajun Liu, Yingjie Han, Liyan Zhuo, Hong-ying Zan
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: General Classification, Grammatical Error Detection, Part-Of-Speech Tagging

In the process of learning and using Chinese, foreigners may have grammatical errors due to negative migration of their native languages. Currently, the computer-oriented automatic detection method of grammatical errors is not mature enough. Based on the evaluating task {---} CGED2016, we select and analyze the classification model and design feature extraction method to obtain grammatical errors including Mission(M), Disorder(W), Selection (S) and Redundant (R) automatically. The experiment results based on the dynamic corpus of HSK show that the Chinese grammatical error automatic detection method, which uses CRF as classification model and n-gram as feature extraction method. It is simple and efficient which play a positive effect on the research of Chinese grammatical error automatic detection and also a supporting and guiding role in the teaching of Chinese as a foreign language.
CYUT-III System at Chinese Grammatical Error Diagnosis Task
Authors: Po-Lin Chen, Shih-Hung Wu, Liang-Pu Chen, Ping-Che Yang
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12

This paper describe the CYUT-III system on grammar error detection in the 2016 NLP-TEA Chinese Grammar Error Detection shared task CGED. In this task a system has to detect four types of errors, in-cluding redundant word error, missing word error, word selection error and word ordering error. Based on the conditional random fields (CRF) model, our system is a linear tagger that can detect the errors in learners{'} essays. Since the system performance depends on the features heavily, in this paper, we are going to report how to integrate the collocation feature into the CRF model. Our system presents the best detection accuracy and Identification accuracy on the TOCFL dataset, which is in traditional Chi-nese. The same system also works well on the simplified Chinese HSK dataset.
Word Order Sensitive Embedding Features/Conditional Random Field-based Chinese Grammatical Error Detection
Authors: Wei-Chieh Chou, Chin-Kui Lin, Yuan-Fu Liao, Yih-Ru Wang
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Grammatical Error Detection, Language Modelling

This paper discusses how to adapt two new word embedding features to build a more efficient Chinese Grammatical Error Diagnosis (CGED) systems to assist Chinese foreign learners (CFLs) in improving their written essays. The major idea is to apply word order sensitive Word2Vec approaches including (1) structured skip-gram and (2) continuous window (CWindow) models, because they are more suitable for solving syntax-based problems. The proposed new features were evaluated on the Test of Chinese as a Foreign Language (TOCFL) learner database provided by NLP-TEA-3{\&}CGED shared task. Experimental results showed that the new features did work better than the traditional word order insensitive Word2Vec approaches. Moreover, according to the official evaluation results, our system achieved the lowest (0.1362) false positive (FA) and the highest precision rates in all three measurements.
A Fluctuation Smoothing Approach for Unsupervised Automatic Short Answer Grading
Authors: Shourya Roy, D, S apat, ipan, Y. Narahari
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: automatic short answer grading, Sequential Pattern Mining

We offer a fluctuation smoothing computational approach for unsupervised automatic short answer grading (ASAG) techniques in the educational ecosystem. A major drawback of the existing techniques is the significant effect that variations in model answers could have on their performances. The proposed fluctuation smoothing approach, based on classical sequential pattern mining, exploits lexical overlap in students{'} answers to any typical question. We empirically demonstrate using multiple datasets that the proposed approach improves the overall performance and significantly reduces (up to 63{\%}) variation in performance (standard deviation) of unsupervised ASAG techniques. We bring in additional benchmarks such as (a) paraphrasing of model answers and (b) using answers by k top performing students as model answers, to amplify the benefits of the proposed approach.
Japanese Lexical Simplification for Non-Native Speakers
Authors: Muhaimin Hading, Yuji Matsumoto, Maki Sakamoto
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Embeddings Evaluation, Language Modeling, Language Modelling, Lexical Simplification, Machine Translation, regression, Sentence, Word Alignment, Word Embeddings

This paper introduces Japanese lexical simplification. Japanese lexical simplification is the task of replacing difficult words in a given sentence to produce a new sentence with simple words without changing the original meaning of the sentence. We purpose a method of supervised regression learning to estimate difficulty ordering of words with statistical features obtained from two types of Japanese corpora. For the similarity of words, we use a Japanese thesaurus and dependency-based word embeddings. Evaluation of the proposed method is performed by comparing the difficulty ordering of the words.
A Corpus-based Approach for Spanish-Chinese Language Learning
Authors: Shuyuan Cao, Iria da Cunha, Mikel Iruskieta
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Discourse Segmentation, POS, Translation

Due to the huge population that speaks Spanish and Chinese, these languages occupy an important position in the language learning studies. Although there are some automatic translation systems that benefit the learning of both languages, there is enough space to create resources in order to help language learners. As a quick and effective resource that can give large amount language information, corpus-based learning is becoming more and more popular. In this paper we enrich a Spanish-Chinese parallel corpus automatically with part of-speech (POS) information and manually with discourse segmentation (following the Rhetorical Structure Theory (RST) (Mann and Thompson, 1988)). Two search tools allow the Spanish-Chinese language learners to carry out different queries based on tokens and lemmas. The parallel corpus and the research tools are available to the academic community. We propose some examples to illustrate how learners can use the corpus to learn Spanish and Chinese.
Syntactic Well-Formedness Diagnosis and Error-Based Coaching in Computer Assisted Language Learning using Machine Translation
Authors: Luis Morgado da Costa, Francis Bond, Xiaoling He
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Machine Translation, Translation

We present a novel approach to Computer Assisted Language Learning (CALL), using deep syntactic parsers and semantic based machine translation (MT) in diagnosing and providing explicit feedback on language learners{'} errors. We are currently developing a proof of concept system showing how semantic-based machine translation can, in conjunction with robust computational grammars, be used to interact with students, better understand their language errors, and help students correct their grammar through a series of useful feedback messages and guided language drills. Ultimately, we aim to prove the viability of a new integrated rule-based MT approach to disambiguate students{'} intended meaning in a CALL system. This is a necessary step to provide accurate coaching on how to correct ungrammatical input, and it will allow us to overcome a current bottleneck in the field {---} an exponential burst of ambiguity caused by ambiguous lexical items (Flickinger, 2010). From the users{'} interaction with the system, we will also produce a richly annotated Learner Corpus, annotated automatically with both syntactic and semantic information.
An Aligned French-Chinese corpus of 10K segments from university educational material
Authors: Ruslan Kalitvianski, Lingxiao Wang, Val{\'e}rie Bellynck, Christian Boitet
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Machine Translation, Translation

This paper describes a corpus of nearly 10K French-Chinese aligned segments, produced by post-editing machine translated computer science courseware. This corpus was built from 2013 to 2016 within the PROJECT{\_}NAME project, by native Chinese students. The quality, as judged by native speakers, is ad-equate for understanding (far better than by reading only the original French) and for getting better marks. This corpus is annotated at segment-level by a self-assessed quality score. It has been directly used as supplemental training data to build a statistical machine translation system dedicated to that sublanguage, and can be used to extract the specific bilingual terminology. To our knowledge, it is the first corpus of this kind to be released.
Analysis of Foreign Language Teaching Methods: An Automatic Readability Approach
Authors: Nasser Zalmout, Hind Saddiki, Nizar Habash
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12

Much research in education has been done on the study of different language teaching methods. However, there has been little investigation using computational analysis to compare such methods in terms of readability or complexity progression. In this paper, we make use of existing readability scoring techniques and our own classifiers to analyze the textbooks used in two very different teaching methods for English as a Second Language {--} the grammar-based and the communicative methods. Our analysis indicates that the grammar-based curriculum shows a more coherent readability progression compared to the communicative curriculum. This finding corroborates with the expectations about the differences between these two methods and validates our approach{'}s value in comparing different teaching methods quantitatively.
Generating and Scoring Correction Candidates in Chinese Grammatical Error Diagnosis
Authors: Shao-Heng Chen, Yu-Lin Tsai, Chuan-Jie Lin
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Decision Making, Sentence

Grammatical error diagnosis is an essential part in a language-learning tutoring system. Based on the data sets of Chinese grammar error detection tasks, we proposed a system which measures the likelihood of correction candidates generated by deleting or inserting characters or words, moving substrings to different positions, substituting prepositions with other prepositions, or substituting words with their synonyms or similar strings. Sentence likelihood is measured based on the frequencies of substrings from the space-removed version of Google n-grams. The evaluation on the training set shows that Missing-related and Selection-related candidate generation methods have promising performance. Our final system achieved a precision of 30.28{\%} and a recall of 62.85{\%} in the identification level evaluated on the test set.
Grammatical Error Detection Based on Machine Learning for Mandarin as Second Language Learning
Authors: Jui-Feng Yeh, Tsung-Wei Hsu, Chan-Kun Yeh
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: BIG-bench Machine Learning, Chinese Word Segmentation, Dependency Parsing, Grammatical Error Detection, Part-Of-Speech Tagging

Mandarin is not simple language for foreigner. Even using Mandarin as the mother tongue, they have to spend more time to learn when they were child. The following issues are the reason why causes learning problem. First, the word is envolved by Hieroglyphic. So a character can express meanings independently, but become a word has another semantic. Second, the Mandarin{'}s grammars have flexible rule and special usage. Therefore, the common grammatical errors can classify to missing, redundant, selection and disorder. In this paper, we proposed the structure of the Recurrent Neural Networks using Long Short-term memory (RNN-LSTM). It can detect the error type from the foreign learner writing. The features based on the word vector and part-of-speech vector. In the test data found that our method in the detection level of recall better than the others, even as high as 0.9755. That is because we give the possibility of greater choice in detecting errors.
Bi-LSTM Neural Networks for Chinese Grammatical Error Diagnosis
Authors: Shen Huang, Houfeng Wang
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Grammatical Error Detection, Sentence, Word Embeddings

Grammatical Error Diagnosis for Chinese has always been a challenge for both foreign learners and NLP researchers, for the variousity of grammar and the flexibility of expression. In this paper, we present a model based on Bidirectional Long Short-Term Memory(Bi-LSTM) neural networks, which treats the task as a sequence labeling problem, so as to detect Chinese grammatical errors, to identify the error types and to locate the error positions. In the corpora of this year{'}s shared task, there can be multiple errors in a single offset of a sentence, to address which, we simutaneously train three Bi-LSTM models sharing word embeddings which label Missing, Redundant and Selection errors respectively. We regard word ordering error as a special kind of word selection error which is longer during training phase, and then separate them by length during testing phase. In NLP-TEA 3 shared task for Chinese Grammatical Error Diagnosis(CGED), Our system achieved relatively high F1 for all the three levels in the traditional Chinese track and for the detection level in the Simpified Chinese track.
Chinese Grammatical Error Diagnosis Using Single Word Embedding
Authors: Jinnan Yang, Bo Peng, Jin Wang, Jixian Zhang, Xue-jie Zhang
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: Grammatical Error Detection, Language Modeling, Language Modelling, Multi-Label Classification

Abstract Automatic grammatical error detection for Chinese has been a big challenge for NLP researchers. Due to the formal and strict grammar rules in Chinese, it is hard for foreign students to master Chinese. A computer-assisted learning tool which can automatically detect and correct Chinese grammatical errors is necessary for those foreign students. Some of the previous works have sought to identify Chinese grammatical errors using template- and learning-based methods. In contrast, this study introduced convolutional neural network (CNN) and long-short term memory (LSTM) for the shared task of Chinese Grammatical Error Diagnosis (CGED). Different from traditional word-based embedding, single word embedding was used as input of CNN and LSTM. The proposed single word embedding can capture both semantic and syntactic information to detect those four type grammatical error. In experimental evaluation, the recall and f1-score of our submitted results Run1 of the TOCFL testing data ranked the fourth place in all submissions in detection-level.
Detecting Uncertainty Cues in Hungarian Social Media Texts
Authors: Veronika Vincze
Date: 2016-12-01 00:00:00
Proceeding: WS 2016 12
Tasks: BIG-bench Machine Learning, Domain Adaptation, Instance Search

In this paper, we aim at identifying uncertainty cues in Hungarian social media texts. We present our machine learning based uncertainty detector which is based on a rich features set including lexical, morphological, syntactic, semantic and discourse-based features, and we evaluate our system on a small set of manually annotated social media texts. We also carry out cross-domain and domain adaptation experiments using an annotated corpus of standard Hungarian texts and show that domain differences significantly affect machine learning. Furthermore, we argue that differences among uncertainty cue types may also affect the efficiency of uncertainty detection.