Package: wikirevparser
WikiRevParserWikiRevParseris a Python library that parses Wikipedia revision histories. It allows you to analyse the development of pages on Wikipedia over time and across language versions.The library takes a language code and Wikipedia page title as input, extracts the revision history, and parses the noisy, unstructured content into clean, accessible data for each timestamp in the revision history.
You can use this library to access the development of references of a page, analyse the content or images over time, compare the tables of content across languages, create editor networks, and much more.Get StartedBeside the WikiRevParser, you'll need our version of theWikipedia API wrapper, which extracts and returns the entire revision history of a Wikipedia page. Note that Python3+ is required.$ pip3 install wikirevparser
$ git clone git@github.com:ajoer/Wikipedia.gitExampleTo get the revision history for the page onMarie Curieon the English Wikipedia, run:>>> from wikirevparser import wikirevparser
>>> parser_instance = wikirevparser.ProcessRevisions("en", "Marie Curie") 
>>> parser_instance.wikipedia_page()
>>> data = parser_instance.parse_revisions()Now you have the revisions of theMarie Curiepage in a structured dictionary format, and you can start exploring the data.Let's look at the use oflinks.
I want to know whether the links on the page are the same now as when the page was first made?>>> edits = list(data.items())
>>> first_links = edits[-1][1]["links"]
>>> latest_links = edits[0][1]["links"]
>>> present_now = first_links[0] in latest_links 
>>> print("The only link in the first version was '%s'.\nThat link is still present in the current version: %s." % (first_links[0], present_now))
The only link in the first version was 'pierre and marie curie'.
That link is still present in the current version: False.Okay, but what are then the most frequent links on the page now?>>> from collections import Counter
>>> links = Counter()
>>> for l in latest_links:
...	links[l] += 1
>> print(links)
Counter({'polonium': 5, 'radium': 5, 'university of paris': 5, 'russian empire': 4, 'gabriel lippmann': 4, 'nobel prize in physics': 4, 'nobel prize in chemistry': 4, ... })With the parsed revision history, you could also get answers for questions like these:When was the 'pierre and marie curie' link deleted?Who made that edit?Did that editor also edit theAfrikaans pageon Marie Curie?What are the most referenced sources on the page?Which references are used on both the English page and theArabic one?How many Wikipedians have edited the English page? And theDutch page?Do all language versions use the same image of Marie Curie as the top image?Where are the Wikipedians located?How frequently is the page edited?Has the English page developed consistently or did editing intensify at one point?How does the editing pattern of the English page match that of theKorean page?... and many other questionsRead the documentation for more inspiration and functionalities, and go toFAQ or file a bugif you run into issues!DocumentationRead the docs atwikirevparser.readthedocs.iofor more details and use case examples.LicenseThis work is MIT licensed. See theLICENSE filefor full details.Credits@goldsmith for the Python Wikipedia API wrapperWikipedia.TheWikimedia Foundationand all Wikipedians for creating and maintaining the data.This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 812997.