## 3 Data Quality Challenges in References Data
So, in this case, it is difficult to automatically extract source metadata about author, title, publisher and others for analysis of correctness or completeness. In Wikipedia, sources can be described using special citation templates different parameters depending on the source type and language version. Such parameters include authors, title, publisher, publication date, URL, access date, DOI, ISBN and others. Table 1 shows the number of references with  
Figure 1: Example of publication data from the Web of Science database. citations templates that contains special identifiers in the top 55 most developed languages versions of Wikipedia. The most developed languages were selected based on article count and depth as proposed in [26]. Based on extracted data it is possible to get the information that was not directly provided in the citation templates. For example, based on special identifiers we can obtain the data about the publishers of each reference using other open bibliographic databases such as Crossref [32]. Thus, after extraction of metadata for over 1 million unique scientific publications with DOI numbers from over 40 million Wikipedia articles in considered languages, we found the top 20 most common publishers of the Wikipedia scientific references:  
1. Elsevier BV (16.08%),
2. Wiley (11.12%),
3. Springer Science and Business Media LLC (6.04%)
4. Springer Nature (5.48%)
5. Oxford University Press (OUP) (5.03%)
6. Informa UK Limited (4.95%)
7. American Chemical Society (ACS) (4.27%)
8. JSTOR (2.70%)
9. SAGE Publications (2.48%)
10. Oxford University Press (2.37%)
11. Proceedings of the National Academy of Sciences (1.79%)
12. American Association for the Advancement of Science (AAAS) (1.73%)
13. IOP Publishing (1.48%)
14. Public Library of Science (PLoS) (1.34%)
15. BMJ (1.19%)
16. Ovid Technologies (Wolters Kluwer Health) (1.08%)
17. Cambridge University Press (CUP) (1.04%)
18. The Royal Society (0.87%)
19. University of Chicago Press (0.86%)
20. American Psychological Association (APA) (0.83%)  
\begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**lang** & **arxiv** & **doi** & **isbn** & **issn** & **jstor** & **pmc** & **pmid** & **oclc** \\ \hline ar & 7698 & 104,720 & 93,441 & 16,282 & 3505 & 15,581 & 71,194 & 5425 \\ de & 5132 & 66,030 & 216,212 & 38,38  
Citation templates can have different names depending on the source which we want to mention as a reference in Wikipedia article: journal, book, conference etc. For example, to give a reference to a book, in English Wikipedia we can use \"Cite book\" template [33]. However in German Wikipedia to cite the same source we need to use another template name--\"Literatur\" [34]. Figure 2 shows the different names of the Wikipedia citation templates related to scientific publications in various languages. Additionally, each of the citation templates in each language version of Wikipedia can have its own set of permitted parameter names that can be used to describe the reference. Even the first and last name of each author of the source can be provided separately. Figure 3 showed the most popular parameters in the Wikipedia citations templates related to scientific publications in English Wikipedia (related figures for other language versions can be found in the supplementary web page [35]). Citation templates are more convenient for machine processing compared to unstructured text with source metadata. However, there may be problems with the quality of data in these templates, such as:  
1. Incompleteness of the metadata. Not all parameters are filled by users in some cases. 2. Mismatches between parameters. For example, users can provide the DOI number related to other publications then it was provided in the URL parameter of the template. 3. Wrong value of the parameters such as title, authors and other parameters comparing to publisher database. 4. Invalid data format. Some parameters must have a special structure to be shown correctly in the Wikipedia article. For example, the \"year\" parameter cannot contain letters.