Cultural Topic Modelling over Novel W ikipedia Corpora for S outh- S lavic Languages (2021)

Abstract: There is a shortage of high-quality corpora for South-Slavic languages. Such corpora are useful to computer scientists and researchers in social sciences and humanities alike, focusing on numerous linguistic, content analysis, and natural language processing applications. This paper presents a workflow for mining Wikipedia content and processing it into linguistically-processed corpora, applied on the Bosnian, Bulgarian, Croatian, Macedonian, Serbian, Serbo-Croatian and Slovenian Wikipedia. We make the resulting seven corpora publicly available. We showcase these corpora by comparing the content of the underlying Wikipedias, our assumption being that the content of the Wikipedias reflects broadly the interests in various topics in these Balkan nations. We perform the content comparison by using topic modelling algorithms and various distribution comparisons. The results show that all Wikipedias are topically rather similar, with all of them covering art, culture, and literature, whereas they contain differences in geography, politics, history and science.

## introduction researchers studying the south-slavic languages often face difficulties finding corpora in the respective languages. while for slovenian there is a significant amount of corpora available both for search and download @xcite @xcite , most other languages in this language group do not enjoy this commodity. this is the reason why web corpora with all their limitations and uncertainties are so popular in this language group foot_0 . this paper describes an effort to add one additional, consistent source of good-quality text for south-slavic languages -wikipedia corpora. to perform that, we set up a robust pipeline for preparing and linguistically processing all currently available wikipedias of south-slavic (macro-)languages. we use the notion of (macro-)languages to signify that the selected wikipedias refer to six national languages with a respective iso-639-1 code, and one macrolanguage, serbo-croatian, with its iso-639-3 code hbs. processing wikipedia corpora could seem a relatively simple task for researchers in computer science, and therefore such processing is performed on a project basis, as was already done for some south-slavic languages @xcite . however, other scientific disciplines lack the technical expertise to perform these data preparations. furthermore, processing wikipedia data on a per-project basis entirely disregards the questions of replicability and reproducibility of research, making the measurements or experiments performed on different dumps with different preprocessing entirely incomparable. we are trying to break away from this poor practice by giving access to fully processed and linguistically annotated wikipedia corpora of south-slavic (macro-)languages whose updates we plan to publish on a yearly basis in the years to come. the current versions of the corpora are based on wikipedia dumps foot_2 of the bosnian (bs), bulgarian (bg), croatian (hr), macedonian (mk), serbian (sr), serbo-croatian (sh) and slovenian (sl) wikipedia, downloaded on october 17th 2020. they are made available for download and search via the clarin.si repository 3 , and ad-ditionally, they can be searched through the clarin.si concordances. 4 5 aside from documenting the methodology applied in preparing these corpora, we perform a topic modelling experiment on the corpora, shedding some light on the topical similarities and differences between the seven corpora. while wikipedia as a research object and a method for performing insights into a specific group's interests and views is considered by now mainstream methodology @xcite , there are just a handful of such inspections of wikipedias of south-slavic languages @xcite . we hope to spark additional interest in such research by making the wikipedia corpora of south-slavic languages standardised, versionable, and easily accessible. as the corpus data size increases, big data architectures might become handy for efficient and timely processing of it @xcite , which in turn might require efficient algorithms for cluster-size and cost optimization @xcite . the paper is structured as follows. in the following section, we overview the methods (1) applied in preparing and linguistically annotating the corpora and (2) performing topic modelling. in the third section, we perform the analysis of the topic modelling results. in the fourth section, we give a short discussion of the obtained results, wrapping up with a conclusion. ## results we start this section by presenting the ten topics obtained for each of the seven south-slavic corpora in alphabetical order. alongside each topic, in brackets, we present the percentage of documents from the 10 000 sample in which that particular topic was most prominent. additionally, to avoid confusion, we add an enumeration index to a topic that has already been designated in the ten topics obtained for a given language sample. an example of this occurs with the serbo-croatian topics, in which we obtained two topics that correspond to matters related to history. this is to be expected due to the lda model's probabilistic machinery to produce as many topics as configured before executing the model. as we previously set 10 topics as desired output, the lda model will strive to produce ten topics, even though naturally, there may be fewer. the detailed results per language are presented in we continue this section by presenting a grouped form of the previously reported results. these consist of the ten topics obtained from each one of the seven lda models and the percentage of documents from the sample of 10 000 in which that particular topic was most prominent. we emphasise the dominant topic because by design, the lda model conceptualises a document as a mixture of multiple topics. thus, as a simplifying measure, we designate a document based solely on its most dominant, i.e., most prominent topic. furthermore, this serves also as a normalisation measure, which enables easier comparison among the south-slavic samples. after noticing that some topics are semantically related to each other, we grouped similar topics to form topic groups. the topic groups which we designated are the following: art, country, culture, geo-politics, history, science. these topic groups are formed by merging more specific topics, such as physics, into broader topics, such as science. broader topics were designated because they contained contents belonging to multiple fields. for example, the culture topic contains some of the keywords met in the art topics, such as painter and film, or the literature topics, which consists of keywords such as writer, book and story. furthermore, the broader country topic captures keywords related to all matters of the country. keywords present in country topics are country, government, borders, language, work, territory, national holiday, culture etc. below we list the topic groups and the topics which they encapsulate. from the aggregated results demonstrated in figure 3 . we can see that approximately each of the seven south-slavic wikipedia corpora samples exhibit characteristics related to the science and art topic groups, which is to be expected from an encyclopedic data source such as wikipedia. additionally, according to figure 3 . based on our lda models, the serbian sample contains the most culture documents (47%), which is the most prominent outlier. other outliers are the bosnian and serbo-croatian samples, which contain 39% and 41% country documents, while the serbian sample contains none. finally, the bulgarian sample houses 35% geo-politics documents, and the bosnian samples contain 34% science documents, for which the bulgarian sample contains none. interestingly, the wikipedias, which have a more balanced distribution of topics, seem to be the macedonian and slovene, which might point towards a diverse editor structure and no agenda being pushed by the editors. on the opposite side, if we were to merge the country and the geo-politics topic groups to form a novel topic group whose probability mass distribution is the sum of the two topic groups' probability mass vectors, we can construct the following listing, namely, bulgarian (63%), slovene (51%), serbo-croatian (47.3%), macedonian (43%), bosnian (39%), croatian (28.9%), and serbian (11%). from this listing, we can measure that this new topic group's average is 47.2%, which places the bulgarian considerably above the mean and perhaps points towards a somewhat biased editor structure. these observations are just preliminary and should be followed up by a more in-depth analysis of topics and other types of analyses, which we hope to happen in the future, especially given the improved accessibility of wikipedia texts -the main contribution of this work. it should be noted that each of the entries in figure 3 . are below 50% as to emphasise the point that even in the constructed samples consisting of 10 000 unique documents, there is no apparent bias. by calculating the row-wise mean for the topic groups' entries in figure 3 . we can obtain the following ordering of topic groups and their average probability mass, namely, history (6.3%), art (14.7%), geo-politics (16.0%), culture (18.0%), science (20.9%), country (24.1%). this ordering is also a probability mass distribution and thus showcases that, on average, each sample's editors mostly focus on matters related to their country. however, the merger of the art, culture, and science topic groups from this novel probability distribution would result in a topic group with 53.6% probability mass, which is suggestive of the notion that despite the biases that the samples' editors might have, overall, wikipedia is still a source composed of encyclopedic knowledge. to further quantify our comparison of wikipedia contents, we applied pairwise jenson-shannon di-vergence (jsd) over pairs of topic distributions of wikipedias. jenson-shannon divergence is a symmetrized and smoothed version of the kullback-leibler divergence. the jsd measure enables us to compare probability distributions, such as the discrete probability distributions housed in each one of the columns presented in figure 3 . through this calculation, we obtain a distance or divergence estimate between each pair of wikipedias. in figure 4 . we present a pairwise matrix that quantifies the distance between every pair of the south-slavic wikipedia samples. the figure shows that the serbian sample is considerably distant from the rest of the samples, being closest to the croatian sample. similarly, the bulgarian sample is notably distant from the rest except for the serbo-croatian and the slovene samples. among all, the serbian sample and the bulgarian sample are the most distant pair with a distance of 53%, while the least distant are the macedonian-croatian and the macedonian-slovene pairs with a distance of 22%. furthermore, from figure 4 ., by calculating the vector component average for each row (or column, since the jsd is a symmetric matrix), we obtain the following ordering of least average distant to most average distant, namely, slovene (25.7%), macedonian (25.8%), serbo-croatian (28.2%), croatian (30.3%), bosnian (33.1%), bulgarian (35.8%), serbian (37.8%). as most average distant, the serbian sample is reflective of the most notable outlier of figure 3 ., that is, the 47% probability mass entry in the culture topic group. additionally, the bulgarian sample is second most average distant due to the emphasis of the country and the geo-politics topics. it should be noted that the aforementioned average jenson-shannon distance ordering consists for the most part of entries below 50% which is indicative of greater likeness than dissimilarity between the south-slavic wikipedia samples. ## discussion it is reasonable to expect that many of the wikipedia articles' topics are recognisable scientific fields or socio-economic disciplines because wikipedia contains various articles contributing to encyclopedic knowledge. such is the case for astronomy within the macedonian corpus, biology within the bosnian corpus, etc. in this work, we considered only 10 000 noundocuments out of a larger number of wikipedia articles, each varying in size and content. additionally, the serbian wikipedia corpus is considerably more comprehensive, and thus, more extensive in terms of the number of articles, while some of the other languages are half of this magnitude or less. this demonstrates that the number of documents considered, the sampling strategy, and original size of the corpora are some of the relevant factors that influence the generated topics from the lda models. additionally, the presence of zero element entries in the matrix depicted in figure 3 is most possibly related to the need for enlargement of the sample size to obtain a more comprehensive result set, in which all wikipedias samples would contain only non-zero entries for every topic group in the corresponding topic group matrix. the results obtained from our topic modelling endeavour are in line with our expectations. each language describes prominent figures and historical events, which entails considering geographical notions, political influences, artistic, cultural, and ideological interpretations. this showcases the difficulty in separating the contents into distinguishable topics. to further improve our results and obtain more disjoint topics, our work could benefit from an approach that has more insight and better language comprehension abilities. other modelling approaches can be employed, such as a hierarchical topic modelling approach. unlike the lda model, the hierarchical dirichlet process (hdp) model by design does not contain a configurable parameter for the number of topics. the hdp model thus outputs a varying number of topics based on the input. furthermore, other avenues that we may explore are different lda model evaluations, formulating and computing model perplexity, and measuring topic coherence. ## conclusion the main contribution of our work is a new collection of corpora of high-quality text for seven south-slavic (macro-)languages, namely, bosnian, bulgarian, croatian, macedonian, serbian, serbo-croatian, and slovenian. the corpora were generated by harvesting wikipedia dumps and postprocessing them to clean the text from all unnecessary phenomena. we linguistically processed these corpora on the levels of tokenization, morphosyntactic annotation and lemmatization. for all languages, except for macedonian, we also performed dependency parsing and named entity recognition. the final corpora are freely available for download foot_10 and concordancer search foot_11 . we plan to generate new versions of the corpora on an annual basis, improving the availability of linguistically processed high-quality corpora for the south-slavic language group significantly. using these linguistically processed corpora, we performed a content-analysis experiment via topic modelling, analysing the topics featured within the wikipedia articles across all mentioned south-slavic corpora. while our topic modelling results are a rather shallow and preliminary insight into the content of the mentioned wikipedias, a trend has already emerged. judging from the dominant topic percentage entries demonstrated in figure 3 ., of which all are below 50%, and from the aver-age pair-wise jenson-shannon distance ordering, whose entries are to the greatest part below 50%, we may gather that the results are suggestive of the notion that the interests of the peoples are more similar than they are opposed. the serbian wikipedia is surely an outlier in terms of similarity to other wikipedias, showing the most significant topical differences to other wikipedias, and will be the first next stop of our analysis. similarly, a large part of the samples contained documents designated with topics attributed to matters related to the country or politics, which warrant further investigation. while the presented corpora will be a very welcome addition to the list of resources for the south-slavic language group, we are aware of the wikipedia text's limitations for documenting a language. therefore, we consider it another direction for future work to be extending these wikipedia corpora with another source of relatively inexpensive but more diverse textual material, namely web corpora.