Package: wptools
Python and command-line MediaWiki access for Humansget page extracts, image, Infobox data, Wikidata, and moreget a random page, category, or siteget page statisticsget category membersget site info and statsget data in any languageThis package is intended to make it as easy as possible to get data
from MediaWiki instances, expose more Wikidata, and extend Wikimedia
APIs just for kicks. We say “(for Humans)” because that is agoal.
Questions, feedback, and especiallycontributionsare welcome!Install$pipinstallwptools✨🦄✨Example>>>importwptoolsGet a page object:>>>page=wptools.page('Gandhi')GetAPI:Querydata:>>>page.get_query()en.wikipedia.org(query)Gandhien.wikipedia.org(imageinfo)File:PortraitGandhi.jpgMahatmaGandhi(en)data{aliases:<list(10)>MKGandhi,MohandasGandhi,Bapu,Gandhi,M...assessments:<dict(10)>Pakistan,AlternativeViews,SouthAfric...description:<str(67)>pre-eminentleaderofIndiannationalism...extext:<str(3077)>Mahātmā**MohandasKaramchandGandhi**(;H...extract:<str(3372)><p>Mahātmā<b>MohandasKaramchandGandhi</b...image:<list(2)>{u'size':2951123,'kind':'query-pageimage',u...label:MahatmaGandhilength:262,790links:<list(500)>10Janpath,14thDalaiLama,1915SingaporeM...modified:<dict(1)>pagepageid:19379random:Saltredirected:<list(1)>{u'to':u'Mahatma Gandhi',u'from':u'Gandhi'}redirects:<list(53)>{u'ns':0,u'pageid':55342,u'title':u'M...requests:<list(2)>query,imageinfotitle:MahatmaGandhiurl:https://en.wikipedia.org/wiki/Mahatma_Gandhiurl_raw:https://en.wikipedia.org/wiki/Mahatma_Gandhi?action=rawwatchers:1,811wikibase:Q1001wikidata_url:https://www.wikidata.org/wiki/Q1001}GetAPI:Parsedata:>>>page.get_parse()en.wikipedia.org(parse)Gandhien.wikipedia.org(imageinfo)File:MKGandhi.jpgMahatmaGandhi(en)data{image:<list(1)>{u'size':2951123,'kind':'parse-image',u'des...infobox:<dict(25)>known_for,other_names,image,signature,bi...iwlinks:<list(10)>https://biblio.wiki/wiki/Mohandas_K._Gandhi,...pageid:19379parsetree:<str(331808)><root><template><title>Redirect</title>...requests:<list(2)>parse,imageinfotitle:MahatmaGandhiwikibase:Q1001wikidata_url:https://www.wikidata.org/wiki/Q1001wikitext:<str(261349)>{{Redirect|Gandhi}}{{pp-move-indef}}{{pp...}GetWikidata:>>>page=wptools.page(wikibase='Q1001')>>>page.get_wikidata()www.wikidata.org(wikidata)Q1001www.wikidata.org(labels)Q1280678|P535|P434|Q1860|P3762|Q668|P12...www.wikidata.org(labels)P119|Q1930187|P691|P18|P19|P1066|P509|P...www.wikidata.org(labels)Q6512732|Q1568|P972|Q84|P1430|P31|Q2140...www.wikidata.org(labels)P1576|Q4964182|P1368|P140|Q22336956|P12...en.wikipedia.org(imageinfo)File:PortraitGandhi.jpgMahatmaGandhi(en)data{aliases:<list(10)>MKGandhi,MohandasGandhi,Bapu,Gandhi,M...claims:<dict(113)>P646,P535,P906,P434,P648,P3762,P1711,...description:<str(67)>pre-eminentleaderofIndiannationalism...image:<list(1)>{u'size':2951123,'kind':'wikidata-image',u'...label:MahatmaGandhilabels:<dict(171)>Q1280678,P535,Q131149,P434,Q1860,P3762,...modified:<dict(1)>wikidatarequests:<list(6)>wikidata,labels,labels,labels,labels,im...title:Mahatma_Gandhiwhat:humanwikibase:Q1001wikidata:<dict(112)>Geni.comprofileID(P2600),NationalLibr...wikidata_pageid:1330wikidata_url:https://www.wikidata.org/wiki/Q1001}GetRESTBasedata:>>>page.get_restbase('/page/summary/')en.wikipedia.org(restbase)/page/summary/GandhiMahatmaGandhi(en)data{description:<str(67)>pre-eminentleaderofIndiannationalism...exhtml:<str(1168)><p>Mahātmā<b>MohandasKaramchandGandhi</b>...exrest:<str(931)>MahātmāMohandasKaramchandGandhi(;Hindust...image:<list(2)>{'kind':'restbase-original',u'width':2024,'...pageid:19379requests:<list(1)>restbasetitle:Mahatma_Gandhiurl:https://en.wikipedia.org/wiki/Gandhiurl_raw:https://en.wikipedia.org/wiki/Gandhi?action=raw}Get all the things (at once):>>>page.get()en.wikipedia.org(query)Gandhien.wikipedia.org(parse)19379www.wikidata.org(wikidata)Q1001www.wikidata.org(labels)Q1280678|P535|P434|Q1860|P3762|Q668|P12...www.wikidata.org(labels)P119|Q1930187|P691|P18|P19|P1066|P509|P...www.wikidata.org(labels)Q6512732|Q1568|P972|Q84|P1430|P31|Q2140...www.wikidata.org(labels)P1576|Q4964182|P1368|P140|Q22336956|P12...en.wikipedia.org(restbase)/page/summary/Mahatma_Gandhien.wikipedia.org(imageinfo)File:MKGandhi.jpg|File:PortraitGandhi.jpgMahatmaGandhi(en)data{aliases:<list(10)>MKGandhi,MohandasGandhi,Bapu,Gandhi,M...assessments:<dict(10)>Pakistan,AlternativeViews,SouthAfric...claims:<dict(113)>P646,P535,P906,P434,P648,P3762,P1711,...description:<str(67)>pre-eminentleaderofIndiannationalism...exhtml:<str(1168)><p>Mahātmā<b>MohandasKaramchandGandhi</b>...exrest:<str(931)>MahātmāMohandasKaramchandGandhi(;Hindust...extext:<str(3077)>Mahātmā**MohandasKaramchandGandhi**(;H...extract:<str(3372)><p>Mahātmā<b>MohandasKaramchandGandhi</b...image:<list(6)>{u'size':2951123,'kind':'query-pageimage',u...infobox:<dict(25)>known_for,other_names,image,signature,bi...iwlinks:<list(10)>https://biblio.wiki/wiki/Mohandas_K._Gandhi,...label:MahatmaGandhilabels:<dict(171)>Q1280678,P535,Q131149,P434,Q1860,P3762,...length:262,790links:<list(500)>10Janpath,14thDalaiLama,1915SingaporeM...modified:<dict(2)>wikidata,pagepageid:19379parsetree:<str(331808)><root><template><title>Redirect</title>...random:Saltredirected:<list(1)>{u'to':u'Mahatma Gandhi',u'from':u'Gandhi'}redirects:<list(53)>{u'ns':0,u'pageid':55342,u'title':u'M...requests:<list(9)>query,parse,wikidata,labels,labels,labe...title:Mahatma_Gandhiurl:https://en.wikipedia.org/wiki/Mahatma_Gandhiurl_raw:https://en.wikipedia.org/wiki/Mahatma_Gandhi?action=rawwatchers:1,811what:humanwikibase:Q1001wikidata:<dict(112)>Geni.comprofileID(P2600),NationalLibr...wikidata_pageid:1330wikidata_url:https://www.wikidata.org/wiki/Q1001wikitext:<str(261349)>{{Redirect|Gandhi}}{{pp-move-indef}}{{pp...}Get more (expensive) data:>>>page.get_more()en.wikipedia.org(querymore)GandhiMahatmaGandhi(en)data{categories:<list(68)>Category:1869births,Category:1948death...contributors:2,606files:<list(53)>File:AumOmred.svg,File:Commons-logo.svg,Fi...languages:<list(168)>{u'lang':u'af',u'title':u'Mahatma Gand...pageid:19379redirected:<list(1)>{u'to':u'Mahatma Gandhi',u'from':u'Gandhi'}requests:<list(1)>querymoretitle:MahatmaGandhiviews:19,242}Get data inanother language:>>>page=wptools.page(lang='zh')zh.wikipedia.org(random)🍰哈莉特·塔布曼(zh)data{pageid:211070title:哈莉特·塔布曼}Get data fromanother wiki:>>>page=wptools.page(wiki='en.wikiquote.org')en.wikiquote.org(random)🍪Malala_Yousafzai(en){pageid:146817title:Malala_Yousafzai}DocumentationSee ourwiki.Please enjoy!@siznax 👹Release History0.4.17 (2018-08-16)Support proxy authentication (#130)Support page links continuations (#129)Support alternate API endpoints (#127)Merged @mcepl update to package tests (#125)Put API warnings in data attribute (#120)Support getting backlinks w/continuations (#119, #122)Always respect silent flag (#118)Improved infobox parsing (#91, #109)Added image (license, assessment) metadata (#104, #105)Fixed Wikidata pageid, image bugs (#101, #102)Added support for category continuations, subcategories (#99)Merged @RobbieClarken fix for non-value Wikidata snaktypes (#98)Merged @seansay fix for deleted Wikidata item (#97)Fixed TypeError on Windows 10 (#95)Fixed RESTBase LookupError on slash titles (#94)Added boxterm param for more Infobox reach (#91)Improved stress tests, wptool CLI script outputFixed get_imageinfo() issue (#93)Getting ALL Wikidata (#56)Surfaced redirects, disambiguation links (#25)Increased test coverage to 95%, pylint score 10.0Added more page data, split off expensive queriesImplemented basic category and site functionalityChanged page._WIKIPROPS to wikidata.LABELSRefactored attributes to only cache, data, flags, paramsUnified Wikimedia APIs access pattern via wptools.core0.3 (2017-08-13)Implemented general RESTBase /page/ access (#67)Merged @rikima’s contrib plus py3 patch (#66)Made page.info(), query(), response() methods useableImproved wiki documentation (see Actions, Attributes)0.2.3 (2017-04-17)Merge @aplz improvement: fetch full lists in infobox values (#62)Put HTML extract with wikilinks into ‘exhtml’Fix AttributeError bug (#47)Refactoring fixes0.2 (2016-11-06)Many fixes/improvements with help of @seansay (thank you!)Raise LookupError on API missing page/titleConsolidated images and cachesSupport get_query() by pageidSupport proxy, timeout on all API entrypointsSupport for language variants (e.g. variant=’zh-cn’)Support for non-https wikis via wiki=’http://example.com’Automatically get_imageinfo() via MediaWiki API:Imageinfomoved NOTES.md to wiki0.1.7 (2016-09-20)Minor fixesProvide local SSL certs with python-certifiEnabled extending Wikidata propertiesImproved cache checking0.1.4 (2016-09-06)Python 3 support!Implemented get_claims()Patch get_rest() path, tests/test_advancedImplemented get_wikidata() by title, langGeo coordinates Property:P625 from WikidataAdded RESTBase support, lead attributeMade wptools.page() the atomic object0.1.0 (2016-08-25)Many fixes, enhancements.ONE CLI SCRIPT TO RULE THEM ALL: wptool.Much testing of random (title, lang, and wiki).Allow chaining get_s or just get().Enabled full-hydration from wikibase only.Block further requests on fatal error.Get 240x thumbnails instead of default 50x.Harmonized silent and verbose arguments.Mock-up tests/test_advanced.py.Made NOTES.md a knowledge base.Use _humans_ in README examples. ;-)0.0.5 (2016-08-23)Major re-write.Exposed core.WPTools as entrypoint.Added get_parse(), get_query(), and get_wikidata().Added get(self) to query all APIs.Added show(self) method to display fetched attrs.Show instance attributes after each request.Ignore requests if attrs will not be updated.Enabled language support across APIs.Gets random article if no arguments.CLI scripts and tests disabled pending update.0.0.4 (2016-08-16)Added wptools.lead.Added safe_exit() to CLI scripts.Removed a fair amount of unused code.0.0.3 (2016-08-12)Implemented wptools.image choices.Added wptools.api to simplify python i/f and CLI scripts.Merged @0x9900’s CLI dist fixes.A little more test coverage.Starting to look like a legit module.0.0.1 (2015)Still better than alternatives for working with articles.0.0.0 (2012)It seems to work!