To What Extent Can E nglish-as-a-Second Language Learners Read Economic News Texts? (2021)

Abstract: In decision making in the economic field, an especially important requirement is to rapidly understand news to absorb ever-changing economic situations. Given that most economic news is written in English, the ability to read such information without waiting for a translation is particularly valuable in economics in contrast to other fields. In consideration of this issue, this research investigated the extent to which non-native English speakers are able to read economic news to make decisions accordingly – an issue that has been rarely addressed in previous studies. Using an existing standard dataset as training data, we created a classifier that automatically evaluates the readability of text with high accuracy for English learners. Our assessment of the readability of an economic news corpus revealed that most news texts can be read by intermediate English learners. We also found that in some cases, readability varies considerably depending on the knowledge of certain words specific to the economic field.

## introduction in the economic field, it is important to read news as primary information to make decisions quickly. while the majority of economic news is written in english, the economic activities of non-native english speakers, or english-as-a-second-language (esl) learners also have a strong influence. to what extent can esl learners read economic news? if, for example, certain economic news is difficult for esl learners to read, then the language gap may affect their economic activities. although this question is important, it was rarely addressed in previous studies. the importance of the aforementioned question is also related to the efficient market hypothesis. the hypothesis is one of the major propositions in financial economics that forms the basis of the black-scholes equation, which is a fundamental formula used for market modeling. it assumes that "asset prices reflect all available information" foot_0 . intuitively, the hypothesis maintains that no information gap, including the gaps caused by language deficiencies, exists in the market. however, this proposition does not hold in cases wherein most second language learners cannot read economic news to make economic decisions in the market. we tackled the challenge of illuminating this issue. to this end, we first sought to understand the difficulty that economic news poses for non-native speakers of english. we approached this problem in two ways. the first was based on the field of educational nlp @xcite . using a corpus that is standard in this field, we constructed a machine learning classifier that can determine the difficulty of a text with high accuracy using deep learning methods, such as bidirectional encoder representations from transformers (bert) @xcite . the second approach was to conduct readability assessments on the basis of information about the vocabulary of english learners. these methods have been well studied in the field of applied linguistics, where considerable research has shown that english learners need to know more than 95% of words in a text to read and understand them @xcite . the idea of assessing text readability via each learner's vocabulary knowledge is beneficial for interpreting readability assessment results. therefore, we also constructed a classifier that ascertains how many words in a text an english learner knows using a data set of english learners' vocabulary tests @xcite . in experiments carried out on a standard data set for evaluating readability @xcite in educational nlp, the two approaches to assessing readability derived results that were in close agreement. next, the experiments involving a real economic news corpus uncovered that most of the texts in the economic news corpus were readable by intermediate english learners. the analysis for which the second approach was used indicated that readability can be substantially improved for english learners through knowledge of a few words that are frequently used in economics. the contributions of this study are as follows: ## automatic readability assessment this section formalizes the problem of automatic readability assessment. let us suppose that we have n texts to assess: we write the set of texts as {t i |@xmath0, . . . , n }}. let y be the set of readability labels. labels are typically ordered in the order of difficulty. for example, in the on-estopenglish dataset @xcite , we can set @xmath1, 1, 2}, where 0 is elementary, 1 is intermediate, and 2 is advanced. the number of levels depends on the evaluation corpus. using y, we write the label for t i as y @xmath2. given each text t i , an assessor outputs its readability score s i . in a supervised setting, the assessor knows the number of levels in the evaluation corpus from training examples. hence, s i ranges within y: s @xmath3. however, in an unsupervised setting, it is noteworthy that the assessor does not know y, or how many levels the evaluation corpus has, because no label is given. hence, even if only integers are allowed for y i , s i can be a real value. throughout this paper, we write arrays using @xcite . given n texts [t i |@xmath4, . . . , n }], our goal is to make an assessor output arrays of readability scores [s i |@xmath5, . . . , n }] that correlate well with the array of labels [y i |@xmath6, . . . , n }]. here, there are multiple types of correlation coefficients between the array of scores and the array of labels, which we explain in the later sections. typically, we should use rank coefficients such as spearman's 15. deficit: the @xmath7. a: spent a lot more money than it earned b: went down a lot in value c: had a plan for its spending that used a lot of money d: had a lot of money stored in the bank figure 1 : examples of the vocabulary size test, one of the most widely accepted vocabulary tests to quickly assess language learners. they are asked to choose the option that paraphrases the part between "<" and ">" from a, b, c, and d. ## vocabulary testing-based readability fig. 1 shows example questions from the vocabulary size test, a widely used vocabulary test in applied linguistics @xcite . each question asks about a word in a multiple-choice question format. the test consists of 100 questions like those shown in fig. 1 . @xcite used this test to have 100 second-language learners take the test and to collect their responses. their data were published and made publicly available. we used their dataset to train our classifiers. we want to analyze vocabulary test results to obtain word difficulty values encoding learners' language knowledge. to this end, we employed the idea of item response theory @xcite , a statistical model that can estimate learners' abilities and test questions' difficulties from the learners' responses to the questions. let v be the set of vocabulary, and let l be the set of learners. let z v,@xmath8, 1} be the result of whether learner @xmath9: then, by using {z v,l } as the training data, we train the following model: in eq. 1, a l is the ability parameter of learner l, d v is the difficulty of word w, and sigmoid denotes the logistic sigmoid function, i.e., sigmoid(x) = 1 1+exp(-x) . the logistic sigmoid function is the binary version of the softmax function, which is frequently used in neural classifiers. it is a monotonously increasing function ranging within (0, 1). as sigmoid(0) = 1 1+1 = 1 2 , when a learner's ability a l is larger than the word difficulty d v , the probability that learner l knows word v can be written as follows: p(@xmath10, l) > 1 2 in eq. 1. likewise, by using eq. 1, we can compare a learner's ability and word difficulty in the same dimension. to estimate learner ability and word difficulty, z v,l is given as z in eq. 1 in the training phase. in this way, in item response theory, learner ability and word difficulty are comparable, and these parameters are estimated from the test result data. in eq. 1, d v denotes the word difficulty estimated from the vocabulary tests. here, in addition to the word difficulty for the words within the vocabulary test, we also want to obtain word difficulty values for all words that may appear in the target language. to this end, we calculate d v by using the word frequency in large balanced corpora as features as follows: eq. 2 assumes that we have k corpora to use as features to calculate word difficulty d v . in eq. 2, k is the number of corpora to use, freq k (v) denotes the frequency of word v in the k-th corpus, and w k is the weight parameter of the k-th corpus. in summary, given the vocabulary test results {z v,l } and corpus frequency features freq k (v), we can estimate the parameters: namely, the weight of the k-th corpus w k and learner l's ability a l . to implement the model, we used logistic regression, by following @xcite . note that this model does not use the valuable readability label {y i } in the training phase, so is unsupervised. after estimating the parameters using the abovementioned procedure, we use the following formula to obtain the readability of given t i . here, l avg denotes the test-taker whose estimated ability parameter is closest to the average of the estimated ability parameter values {a l }. intuitively, the following equation calculates the probability that the average learner knows all the words that appear in t i and uses it as the readability score. the use of the -log here has two reasons. the reason of using log is to prevent problems in numerical calculations, since the probability values can be very close to 0. while we want to design d v so that the more difficult the word v, the larger the value, whereas the higher the probability, the easier the word. hence, we use negative of log to make the two scales meet. ## experiments with economic news we used an economic news article dataset @xcite for our analysis because it is publicly available and easy to replicate. the dataset consisted of 109,110 reuters news articles published in 2006-2007. tab. 2 shows the readability assessment results for each month of 2007. each month included approximately 1,000 articles. few of the articles were elementary, whereas most were intermediate. approximately 2%-4% of the texts were advanced. the results show that intermediate english learners could read most but not 2%-4% of economic news. we also conducted a vocabulary-based analysis. using @xcite , we chose a learner who could successfully answer 75 vocabulary questions among the 100 questions in the vocabulary size test @xcite . this learner was estimated to know 15, 000 words. knowledge of this number of words should be sufficient for reading typical newspaper texts @xcite . however, in the vocabulary-based readability assessor, some words, such as "annuity" and "veritable", were predicted to be unfamiliar to this learner. in one article, these words greatly reduced the probability that this learner could read the text from 0.62 to 0.38: these probability values represented the probability that this learner knew at least 95% of the words in this text. the probability of knowing each word was represented by eq. 1. ehara (2019 ) proposed an algorithm to calculate the probability of knowing 95% or more of the words in a text from the probability of knowing each word. we simply used their algorithm to calculate these probabilities. ## discussion in eq. 2, word frequencies in the corpora are used as features. in this study, we used multiple balanced corpora. eq. 2 is applicable when preparing k corpora in advance. k denotes the index of the prepared corpora. in nlp, the proposed method is closely related to complex word identification (cwi) tasks @xcite . cwi is a task that aims to discover difficult words in a text. the relationship between cwi and personalized text readability was previously studied in @xcite . the task of obtaining the difficulty of an english word for each individual esl learner, as we performed in this study, can be regarded as personalized cwi @xcite 4 . personalized cwi has many downstream applications in nlp, such as lexical simplification @xcite ), text recommendation for language learners @xcite @xcite , and translator selection in crowdsourcing @xcite . some studies have focused on the relationship between word semantics and word difficulty @xcite @xcite . regarding the interpretability of cwi classifiers, @xcite studied the relationship between cwi classifiers' weights and vocabulary sizes. ## conclusions in this paper, we focused on the readability of economic news texts for esl learners. we conducted two approaches for measuring readability: the bert-based approach and the vocabulary-based approach. we found that although most texts were readable to intermediate learners, 2.4% of articles were not readable to them. furthermore, some economic words greatly reduced the readability for esl learners. future work will include investigating for differences in the readability of different types of economic news for esl learners.