DOI: 10.82270/efg-wikicorpus
title: EFG_WikiCorpus -- discussions en coulisse de Wikipedia (anglais, français, allemand)
title: EFG WikiCorpus -- discussions in Wikipedia's backstage (English, French, German)

description: Le WikiCorpus EFG est un corpus composé de toutes les pages de discussion liées à la (co)rédaction d'un article dans les Wikipédias anglaise, française et allemande au 1er août 2019.


Les pages de discussion sont extraites des dumps de/en/fr du 1er août 2019 (enwiki-20190801-pages-meta-current.xml, frwiki-20190801-pages-meta-current.xml, dewiki-20190801-pages-meta-current.xml) et encodées selon le schéma TEI CMC-core (Beißwenger et Lüngen 2020, https://tei-c.org/release/doc/tei-p5-doc/en/html/CMC.html). Toutes les pages de discussion (versions actuelles et archivées) contenant au moins 1 message (ou fil de discussion) et 2 mots ont été extraites.

Le EFG_WikiCorpus est composé de plus de 3,3 millions de pages de discussion, 2 milliards de mots, près de 10 millions de fils de discussion et 30 millions de messages. Plus de détails sont donnés dans la description en version anglaise et dans Ho-Dac (2024). 
description: The EFG WikiCorpus is a corpus composed of all the talk pages dedicated 
to (co)writing an article in the English, French and German Wikipedias in August 1st 2019.


The talk pages are extracted from the August 1st
 2019 de/en/fr dumps (enwiki-20190801-pages-meta-current.xml, 
frwiki-20190801-pages-meta-current.xml, 
dewiki-20190801-pages-meta-current.xml) and encoded according to the TEI
 CMC-core schema (Beißwenger and Lüngen 2020, https://tei-c.org/release/doc/tei-p5-doc/en/html/CMC.html). All talk pages (current 
and archived versions) containing at least 1 message (or thread) and 2 
words have been extracted.


The EFG_WikiCorpus is composed of more 
than 3.3 million talk pages, 2 billion words, almost 10 million threads 
and 30 million messages. More details are given in Ho-Dac (2024).

The EFG_WikiCorpus: the whole dataset
The
 EFG_WikiCorpus is organized in three main repositories i.e. one for 
each language : E_talkPages, F_talkPages, G_talkPages. Thematic portals/categories information and multilingual links between articles are indicated in the TEI-P5 teiHeader.

	
	
	
	
	
	
		
			

			
		
		 E_talkPages
		
		
			F_talkPages
		
		
			G_talkPages
		
		
			EFG WikiCorpus
		
	
	
		
			Talk pages
		
		
			2,025,888
		
		
			266,699
		
		
			713,485
		
		
			3,006,072
		
	
	
		
			Archive
			(%)
		
		
			57,432 (3%)
		
		
			4,836 (2%)
		
		
			23,052 (3%)
		
		
			85,320 (3%)
		
	
	
		
			#threads
		
		
			6,636,783
		
		
			608,857
		
		
			2,121,852
		
		
			9,367,493
		
	
	
			#posts
		
		
			20,025,945
		
		
			1,832,416
		
		
			6,938,168
		
		
			28,796,529
		
	
	
		
			#words
		
		
			1,448,411,901
		
		
			146,230,896
		
		
			454,737,723
		
		
			2,049,380,520
		
	

(A word is defined as a sequence of alphanumeric characters with the character "_" i.e. using the \w+ perl regular expression.)


There are 4 files in each language repository: 
cmc-wikitalk.rng : the TEI CMC-core schema.idPages_xxxx.csv gives for each talk page its idno (in the dump file), 
its title, the number of threads, messages, words, the number of 
different users, of anonymous users and bots, its type (current or 
archived), the article's title and the quality grade of the article as 
assessed by Wikipedia users.idUsers_xxxx.csv lists each users and the number of messages they posted for each talk page (identified by its idno).X_talkPagesRep.zip: this archive contains 37 repositories gathering together all the talk pages (i.e.
 tei xml files) whose the title has the same initial character. For 
example, the repository "M" contains all talk pages whose title starts 
with "M" like the talk page about the article "Mars (Planet)". There are
 26 letter repositories (from "A" to "Z"), 10 number repositories (from 
"0" to "9") and 1 "Char" repository for all talk pages whose the title 
starts with a special character (diacritics, ç, €, ß).

The EFG_WikiDemoCorpus: a derived and easy-to-handled subcorpus

A 
subcorpus called the EFG_WikiDemoCorpus is derived from the EFG_WikiCorpus and processed in order to facilitate 
qualitative analyses. This subcorpus is composed of the talk 
pages related to diverse topics that are relevant for all 
languages and that show significant activity. Significant activity means that there is at 
least one archive talk page and a high number of messages in the current
 talk page. The table below lists some information about the selected talk pages. All EFG_WikiDemoCorpus files are organized in the EFG_WikiDemoCorpus repository:idPages_EFG_WikiDemoCorpus.csv gives for each talk page its idno (in the dump file), 
a multilingual title that is common to the three languages, its language, its title, the article's title, its type (current or 
archived), the number of threads, messages and words.xml repository contains all the TEI-P5 xml files of the EFG_WikiDemoCorpus. Each file has been renamed with the multilingual title, the letter of the language and the idno e.g. zelda_G_1787404.tei.xml is the name of one German talk page associated to the article about The legend of Zeldo with the idno 1787404.


body,div,table,thead,tbody,tfoot,tr,th,td,p { font-family:"Liberation Sans"; font-size:x-small }a.comment-indicator:hover + comment { background:#ffd; position:absolute; display:block; border:1px solid black; padding:0.5em;  }a.comment-indicator { background:red; display:inline-block; border:1px solid black; width:0.5em; height:0.5em;  }comment { display:none;  }


Talkpages in the EFG WikiDemoCorpus and their related articles are also 
searchable online on the Korap application (developed at the Leibniz Institute for the German Language), which allows different 
query languages: https://korap.ids-mannheim.de/instance/wikidemo the Contropedia platform (Borra et al. 2015), 
which provides an analysis of all the edits performed in the article 
pages: https://www.contropedia.net/demo.



	
	
	
	
	
	
	
	
	
		
			Portal
		
		
			Multilingual titleArticle title (WP.en, WP.fr, WP.de)
		
		
			#talk pages
		
		
			#words in talk pages
		
	
	
		
			

			
		
		
			

			
		
		
			E
		
		
			F
		
		
			G
		
		
			E
		
		
			F
		
		
			G
		
	
	
		
			EFG WikiDemoCorpus 
			
		
		
			162
		
		
			33
		
		
			41
		
		
			3949521
		
		
			719334
		
		
			762049
		
	
	
		
			Politics
		
		refugeeCrisis
			European_migrant_crisis,
			Crise_migratoire_en_Europe,
			Flüchtlingskrise_in_Europa_ab_2015
						
		
		
			2
		
		
			1
		
		
			6
		
		
			35430
		
		
			9708
		
		
			179527
		
	
	
		
			Biology
		
		chiro
			Chiropractic,
			Chiropratique, Chiropraktik
		
		
			41
		
		
			3
		
		
			2
		
		
			1709125
		
		
			33533
		
		
			13825
		
	
	
		
			People
			

			
		
		VPVladimir_Putin, Vladimir_Poutine, Wladimir_Wladimirowitsch_Putin
						
		
		

			16
		
		
			3
		
		
			5
		
		
			345205
		
		
			57531
		
		
			95709
		
	
	
		
			History
			

			
		
		0911attacks
			September_11_attacks, Attentats_du_11_septembre_2001, Terroranschläge_am_11._September_2001
						
		
		
			64
		
		
			4
		
		
			6
		
		
			1330399
		
		
			133667
		
		
			135662
		
	
	
		
			Society
		
		psy
			Psychoanalysis, Psychanalyse, Psychoanalyse
						
		
		
			6
		
		
			9
		
		
			8
		
		
			55655
		
		
			93328
		
		
			137887
		
	
	
		
			Technology
		
		GMO
			Genetically_modified_organism
, Organisme_génétiquement_modifié, Gentechnisch_veränderter_Organismus
		
		
			4
		
		
			10
		
		
			2
		
		
			54954
		
		
			343124
		
		
			3693
		
	
	
		
			Society
		
		fem
			Feminism, Féminisme, Feminismus
						
		
		
			22
		
		
			2
		
		
			10
		
		
			357281
		
		
			41780
		
		
			164352
		
	
	
		
			Life
		
		zelda
			The_Legend_of_Zelda, The_Legend_of_Zelda, The_Legend_of_Zelda
		
		
			7
		
		
			1
		
		
			2
		
		
			61472
		
		
			6663
		
		
			31394
		
	


References

Michael 
Beißwenger et Harald Lüngen. (2020). CMC-core: a schema for the 
representation of CMC corpora in TEI, Corpus [En ligne], 20. URL : 
http://journals.openedition.org/corpus/4553 ; DOI : 
https://doi.org/10.4000/corpus.4553

Erik Borra, David Laniado,  Esther 
Weltevrede, Michele Mauri, Giovanni Magni, Tommaso Venturini, Paolo Ciuccarelli, Richard Rogers and Andreas Kaltenbrunner. (2015) A Platform for Visually Exploring the Development of Wikipedia Articles, ICWSM '15 - Proceedings of the 9th International AAAI Conference on Web and Social Media. 

Lydia-Mai Ho-Dac (2024) Building a comparable corpus of online discussions on Wikipedia: the EFG WikiCorpus. In Poudat, C., Lüngen, H. and Herzberg, L. (eds.), Investigating Wikipedia: Linguistic corpus building, exploration and analysis. Coll. Studies in Corpus Linguistics, John Benjamins.