Mama/Papa, Is this Text for Me? (2020)

Abstract: Children have less linguistic skills than adults, which makes it more difficult for them to understand some texts, for instance when browsing the Internet. In this context, we present a novel method which predicts the minimal age from which a text can be understood. This method analyses each sentence of a text using a recurrent neural network, and then aggregates this information to provide the text-level prediction. Different approaches are proposed and compared to baseline models, at sentence and text levels. Experiments are carried out on a corpus of 1, 500 texts and 160K sentences. Our best model, based on LSTMs, outperforms state-of-the-art results and achieves mean absolute errors of 1.86 and 2.28, at sentence and text levels, respectively.

## introduction in recent years, safe internet for children has gained interest in many research domains @xcite @xcite . however, most studies focus on abusive texts containing hate, violence, pornography, etc. @xcite . on the contrary, the adequacy of textual contents with the reading and understanding capabilities of children remains yet mainly unresolved in computational linguistics. hence, this paper propose a new method to predicting this adequacy. among the related works, @xcite explored the possibility of predicting from which us school grade newspaper articles could be read. this task was modelled as a classification problem among 4 classes using support vector machine fed with word-based n-gram probabilities, as well as lexical and syntactic features. @xcite has proposed a readability classification method for bangla news articles for children. this method predicts if a text is either very easy, easy, medium or difficult. more recently, @xcite proposed different feed-forward (ff) neural models for age recommendation on texts targeting either children (from 0 to 14) or adults. the authors consider this as a regression task and explore various linguistic features and word embedding features, from which word embedding are shown as the most contributory. overall, one can notice that these papers either rely on hand-crafted features or on simple models which consider texts as a global object rather than word sequences. this motivates us to further explore with word embeddings only and introduce recurrent neural networks (rnns). more broadly, in the field of text readability, inherited from historical approaches like @xcite , audiences other than children have been studied, e.g., second language learners @xcite , adults readers @xcite or patients interacting with doctors @xcite . in parallel, text understanding by children is a well known question in psycho-linguistics and cognitive sciences. in particular, key findings have shown the impact of memory @xcite , temporality @xcite , and emotions @xcite . related results also exist in the learning to read domain @xcite . following the recent trends in nlp, the contribution of this paper is to tackle age prediction as a regression problem using rnns based on long short term memories (lstms) and fed with pre-trained word embeddings. we propose several variants of this architecture and compare them extensively to naive and ff approaches. we also investigate the difference between predicting age at the sentence and text levels. let one note that the use of more advanced architectures like transformers @xcite is left for the future, since they are known to require very large amounts of data. the experiments are carried out on a french corpus of around 1, 500 texts of 160k sentences, from encyclopedia, newspapers and fictions for a wide range of different age levels, including adults. we think that this corpus is another interesting aspect of our work, since, compared to others, it is not limited to a specific genre or public. in the remainder, section 2 defines with more precision the age prediction task and presents the related data. then, section 3 presents the adopted approach and the underlying models. finally, section 4 details and discusses the results. ## definition of the problem and data in this paper, we consider texts annotated with recommended age ranges @xcite . these age ranges are interpreted as an approximation of the minimal age from which the text can be understood. that is, we define the target minimal age as the mean of the interval, a+b 2 . then, to predict this value for a given text, we decide to decompose the problem down to the sentence level, by associating each sentence of a text with the same age range and mean as the whole text. although this assumption is strong as the complexity of the sentences may vary, it has been shown to be an effective strategy @xcite . in practice, we have built a french dataset compiled from encyclopedia, newspapers, and fictions, either dedicated to children or adults. this dataset consists of 1500 texts and abount 160k sentences foot_1 . children texts ranges from 0 to 14 years, while adult texts are arbitrary associated with the range @xcite , and mean 16. overall, the average of the age range is around 10-14 years where the mean age is 12. the dataset is split into train, dev and test sets at the text level, i.e. all sentences of a given text are kept together in the same set. this partitioning follows the proportions 67/17/16% in terms of sentences, and 70/15/15% in terms of texts. detailed statistics are provided in ## approach and models since ages are continuous and sequential values, we consider age prediction as a regression task. it is studied at the sentence and text levels. our sentence-level models. at the sentence level, the core of the proposed approach is an lstmbased model @xcite . this kind of rnn is able to learn one-way long-term dependencies of a sequence and is widely used in text classification and time series prediction tasks. in our model, the input is the sequence of words from the input sentence. each word goes through a projection layer set with pre-trained word embeddings, before entering the lstm layer. in a first model, the output is directly the real-valued mean age. alternatively, we also study considering the age range [a, b] as the model's output, before manually deriving the mean. in the remainder, the first option is referred to as "lstm/direct", the second as "lstm/range". we also experiment with the use of a bidirectional lstm @xcite , i.e., the simultaneous use of a forward lstm and a backward one. the idea is it gets more contextual information on the input data, although the model is more complex (more parameters to be trained). the settings of input and output in this model remain similar to the lstm/direct model. ## experiments metrics. as a regression task, results are mainly given in terms mean absolute error (mae) between the target and predicted ages. global results. per-class results. figure 2 details how the feed-forward and lstm/direct models behave for each age at the text level for the test set. correct and wrong classifications are given, as well as per-class maes. overall, it appears that both models perform better around the median of the distribution, which seems logical for a machine learning approach. the main difference seems that the feed-forward model is worse than lstm/direct on very small ages, whereas it is better for adult texts. in complement to previous obversations on genres, these differences probably also contribute to the performance similarity of the ff model on texts inspite of differences at the sentence level. finally, these results also show that further efforts should be paid on improving predictions for low ages as this is where mistakes would have the strongest impact in a real-life application. ## conclusion and perspectives this paper proposed lstm models to predict age for sentences and texts. as opposed to the previous related work, these models consider sentences as a sequence of words and do not rely on hand-crafted features. our best model achieves significantly better scores than the baseline models for the sentence level predictions, confirming the interest of recurrent architectures. however, sentence-level experiments show that this improvement is not propagated at the text level. hence, in the future, we would like to improve this model for text predictions. to do this, a first perspective is to build more elaborate aggregation techniques of the sentence-level predictions. then, it would also be interesting to compare with approaches where predictions are directly made at the text-level, without decomposing into sentences. finally, we would like to explore more advanced recurrent models, like transformers. to do so, an option would be to train a first model using a coarsely annotated corpus, and then fine tuning it on the current corpus. such imprecise data can be rather easy to collect, for instance, using children-dedicated encyclopedia.