Saravanan666/pypi-org huggingface dataset:
Package: wikibase-reconcile
Wikibase ReconcileAn multi-threaded client to reconcile against wikidata.This project is inspired byreconciler. If you look for stable and mature software, I recommend using the excellent packagereconciler.FeaturesConcurrent requests to wikidata reconiliation APIMakes atomic requests and is more robust when the API returns errorsTakes a pandas dataframe as input and also as outputUser has rather high degree of control (but also needs more code)Installationpip install wikibase-reconcileUsagefromwikibase_reconcileimportClientclient=Client()# Initialize some simple test datacharacters=[{"type":"Q15632617","query":"Marge Simpson"},{"type":"Q15632617","query":"Homer Simpson"},{"type":"Q15632617","query":"Bart Simpson"},{"type":"Q15632617","query":"Lisa Simpson"},{"type":"Q15632617","query":"Maggie Simpson"},{"type":"Q15632617","query":"Abraham Simpson"},{"type":"Q15632617","query":"Apu Nahasapeemapetilon"},{"type":"Q15632617","query":"Barney Gumble"}]df=pd.DataFrame(characters)# reconcile each search query against the type "Fictional Human"results=client.reconcile(df)# Turn results into a dataframe, keep only the top resultdf_reconciled=client.results_to_pandas(results)Fallback LogicYou can specify an additional fallback search string in case the first attempt did not yield any result:query={"type":"Q15632617","query":"Marge Simpson","fallback_query":"Marjorie Bouvier"}One use case for fallback queries are lemmatized versions of the original query.If this still yields no results, then the reconciler will attempt to match the first part of the query, by splitting the query at every hyphen character (-) and taking the first part.This logic is likely to change in future versions.The final fallback is to ignore the type and try to reconcile against any type. The default API server (https://wikidata.reconci.link/) will then match against the generic entity type (Q35120).
Package: wikibrain
Stores knowledge and data necessary to properly use links from OpenStreetMap to Wikipedia, Wikidata and Wikimedia Commons.InstallationIt has a very unfortunate dependecies - sorry for that, pull requests improving this situation are welcome.first you need to clone, modify and install as a python packagethis thing- pull request providing cache location in a better way (config file? environment variable?) is welcomeFinally, install packages in a standard way (providing config and this should be the only steps, sorry):pip3 install wikibrainOrgit clone https://github.com/matkoniecz/wikibrain.git
cd wikibrain
pip3 install -r requirements.txtBut in either casehttps://github.com/matkoniecz/wikibrain/issues/1remains unsolved.Running testspython3 -m unittestFixing WikidataSearch fortoo_abstract_or_wikidata_bugsand disable that to get more errors caused by Wikidata
Package: wikicite
wikiciteInstallationpip install wikiciteUsageCommand line tool to create ready-to-use Wikipedia citations from a series of inputs.$ wikicite news bbc -t 'An Important Article' -a Fred Bloggs -d 01062020 -url http://www.bbc.co.uk/an-important-article

<ref name=Bloggs200601>
{{cite news |last=Bloggs |first=Fred |title=An Important Article |work=[[BBC]] |url=http://www.bbc.co.uk/an-important-article |date=1 June 2020 |access-date=28 January 2023}}
</ref>Where any options are not specified, the user will be prompted to add them, but all except date are optional.
A note on formatting for certain options:Titles-tor--titleshould be provided in quotation marksAuthors-aor--authorshould be provided as<firstname> <lastname>. Where multi-part names are needed, quotation marks should be used to identify the parts, e.g.Lynn 'Faulds Wood'or'John Paul' JonesDates-dor--dateshould be provided inddmmyyyyformatFor a full list of options, see$ cite --helpFor more details on Wikipedia citations please seeWikipedia
Package: wikiclass
# Wikipedia article quality classificationThis library provides a set of utilities for performing automatic detection of assessment classes of Wikipedia articles.  For more information, see the full documentation athttp://pythonhosted.org/wikiclass.Compatible with Python 3.x only.Sorry.Install:pip install wikiclassModels:https://github.com/halfak/wikiclass/tree/master/modelsDocumentation:http://pythonhosted.org/wikiclass## Basic usage>>> import wikiclass
>>> from revscoring.scorer_models import MLScorerModel
>>>
>>> scorer_model = MLScorerModel.load(open("models/enwiki.wp10.rf.model", "rb"))
>>>
>>> text = "I am the text of a page.  I have a <ref>word</ref>"
>>> wikiclass.score(scorer_model, text)
{'prediction': 'stub',
 'probability': {'stub': 0.27156163795807853,
                 'b': 0.14707452309674252,
                 'fa': 0.16844898943510833,
                 'c': 0.057668704007171959,
                 'ga': 0.21617801281707663,
                 'start': 0.13906813268582238}}## Authors
* Aaron Halfaker –https://github.com/halfak* Morten Warncke-Wang –http://www-users.cs.umn.edu/~morten
Package: wikicleaner
WikicleanerWikipedia provides a series ofdumpsin XML format. While developing this project, I personally used thesimple wikidumps.PurposeThe intent of this project is to provide some low-level routines that allow us to manipulate the content of XML dumps. One such routine removes all of the annotations like[[File:*]]and{{some metadata}}in order to make our input more digestible for an AI application that uses embeddings.Example Usage>>>importwikicleaneraswc>>>article_raw_text="[[File:Colorful spring garden.jpg|thumb|180px|right|[[Spring]] flowers in April in the [[Northern Hemisphere]].]] April comes between [[March]] and [[May]]">>>wc.clean_article_text(article_raw_text)' April comes between March and May'
Package: wikicodename
wikicodenameGenerate code names using lists and tables from Wikipedia articles.InstallationUsepipto install the application.pip install wikicodenameWarning!On Windows you may need to set thePYTHONUTF8environmental
variable to1for the application to work properly.UsageUsewikicodenamecommand to generate a list of code names for the default
profile.$ wikicodename
sugar-plum-winnipeg
bud-green-betim
livid-malolos
onyx-ituiutaba
alizarin-belem
pink-flamingo-tokai
tuscan-brown-glasgow
warm-black-getafe
hot-magenta-managua
field-drab-qingdaoUse--list-profilesflag to list all available profiles.$ wikicodename --list-profiles
adjective
adjective-dinosaur
adjective-dinosaur-short
city
coctail
coffee
color
color-city
color-city-short
dinosaur
gwiazdozbior
main
women-scientist
women-scientist-transformedUse--profileor-pflag to set a profile.$ wikicodename --profile adjective-dinosaur
windy-yandangornis
wet-jiangjunosaurus
soft-muttaburrasaurus
starry-pleurocoelus
cool-gresslyosaurus
flat-lucianovenator
late-libycosaurus
long-arstanosaurus
busy-gigantosaurus
salty-dollodonUse--countor-cflag to set a length of the generated list.$ wikicodename --count 5
cultured-el-progreso
coral-baidyabati
yellow-green-olmaliq
indian-red-new-haven
infrared-madinaUse--sortor-sflag to sort the generated list.$ wikicodename --sort
antique-ruby-lynn
black-shadows-toyota
cg-blue-tiaong
desert-sand-gorgan
fire-opal-bac-lieu
magic-mint-kyoto
middle-green-madurai
navy-blue-tsuchiura
purple-laayoune
sand-santa-claraUse--helpor-hflag for more information.Defining a profileYou can define your own profile by creating a YAML file in the configuration
directory.The default configuration directory path is:~/.config/wikicodenamefor Linux,~/Library/Application Support/wikicodenamefor Mac OS X,%localappdata%\wikicodenamefor Windows.While you are there, please refer tomain.yaml,city.yamlandgwiazdozbior.yamlfiles. They are well documented and will help you create
your own profile.You may also visit the"List of lists of lists" articlefor inspiration.LicensePlease refer to theLICENSEfile.
Package: wiki-con
wiki-con - A console app that displays random facts using the Wikipedia APIwiki-conis a simple CLI-based application that displays random facts using the Wikipedia API. The main focus of this repository is the implementation of various testing components and practices for the release and packaging of a Python project.Installationgit clone https://github.com/kevinbowen777/wiki-con.gitcd wiki-conpoetry run wiki_conPackages:TestPyPi:https://test.pypi.org/project/wiki-con/PyPi:https://pypi.org/project/wiki-con/FeaturesTestingpytestcoveragenoxpytest-mockLintingBlackFlake8flake8-annotationsflake8-banditflake8-blackflake8-bugbearflake8-docstringsflake8-import-orderdarglintSafetypre-commitTypingReporting BugsVisit theIssues pageto view currently open bug reports or open a new issue.
Package: wikicreator
wikicreatorPython Boilerplate contains all the boilerplate you need to create a Python package.Free software: MIT licenseDocumentation:https://wikicreator.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2016-07-15)First release on PyPI.0.3.0 (2016-07-15)Click integration0.4.1 (2016-07-16)Removed dependencies, put in cookiecutter template
Package: wikidata2df
wikidata2dfUtility package for easily turning a SPARQL query into a dataframeEver wished you could easily and programatically get data from Wikidata into a nice and analysable Pandas DataFrame?
Well, this package solves that problem: With a single function you can turn your SPARQL query into a pandas DataFrame,
without having to deal with the messy JSON intermediate.Free software: BSD licenseDocumentation:https://wikidata2df.readthedocs.io.Basic UsageTo install:$ pip install wikidata2dffrom wikidata2df import wikidata2df

# A SPARQL query to return all cats in Wikidata!

cat_query = """
#Cats
SELECT ?item ?itemLabel
WHERE
{
?item wdt:P31 wd:Q146.
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
"""

cats_dataframe = wikidata2df(cat_query) # Returns a Pandas DataFrameAlternativesMaybe you want more sofisticated functions? Or a way to edit Wikidata programatically? Awesome!Check outWikidataIntegratorWould you rather use R? That’s cool too!Check outWikidataQueryServiceRCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2020-08-08)First release on PyPI.
Package: wikidata-dl
wikidata-dlDescriptionwikidata-dlis a command line program for downloading data fromWikidatabased onSPARQLqueries.Installationpip install wikidata-dlUsageDownload result returned from the query and save it in the directory./wikidata:wikidata-dl /path/to/my-query.sparqlFor a complete reference of the command line options run:wikidata-dl --helpSample QueryThe following query returns IDs for all Wikidata entities that are an instance of (P31) a continent (Q5107).SELECT ?item WHERE {
    ?item wdt:P31 wd:Q5107.
}Try the query
Package: wiki-data-dump
wiki_data_dumpA library that assists in traversing and downloading fromWikimedia Data Dumpsand its mirrors.PurposeTo make the maintenance of large wiki datasets easier and more stable.In addition, the purpose is to lighten the load on Wikimedia and its mirrors by
accessing only the index of the site, and doing the inevitable searching and
navigation of its contents offline.A web crawler might make multiple requests to find its file (in addition to
navigating with the notorious fragility of a web crawler), whilewiki_data_dumpcaches the site's contents - which not only provides a speed boost for multiple
uses of the library but protects against accidentally flooding
Wikimedia with requests by not relying on requests for site navigation.Installationpip install wiki_data_dumpUsageOne could easily get all available job names for any given wiki with this
short script:fromwiki_data_dumpimportWikiDump,Wikiwiki=WikiDump()en_wiki:Wiki=wiki.get_wiki('enwiki')print(en_wiki.jobs.keys())Or, you could see the available files from thecategorytablessql job.fromwiki_data_dumpimportWikiDump,Jobwiki=WikiDump()categories:Job=wiki.get_job("enwiki","categorytables")print(categories.files.keys())A slightly more nontrivial example - querying for specific file types when a job
may contain more files than we need.For example, it's not uncommon to find a job that has partial data dumps - making
it necessary to know the file paths of all parts. If you're hard-coding all the
file names, it becomes increasingly difficult to find the relevant files.This is a solution thatwiki_data_dumpprovides:fromwiki_data_dumpimportWikiDump,FileimportrefromtypingimportListwiki=WikiDump()xml_stubs_dump_job=wiki["enwiki","xmlstubsdump"]stub_history_files:List[File]=xml_stubs_dump_job.get_files(re.compile(r"stub-meta-history[0-9]+\.xml\.gz$"))forfileinstub_history_files:wiki.download(file).join()Download processes are threaded by default, and the call toWikiDump.downloadreturns a reference to the thread it's running in.The process is simple and readable:Get the job that contains the files desired.Filter the files to only contain those that you need.Download the files concurrently (or in parallel).For more direction on how to use this library, seetests.pyor
examples inexamples.Next stepsAutomatic detection of which mirror has the fastest download speed at any
given time.Caching that updates only when a resource is out of date, instead of just when
the current date has passed the cache's creation date.The ability to access Wikimedia downloads available in/other/.DisclaimerThe author of this software is not affiliated, associated, authorized, endorsed by,
or in any way officially connected with Wikimedia or any of its affiliates and is
independently owned and created.
Package: wikidata-plain-sparql
WikiData Plain SPARQL ClientThis package allows to query the WikiData database. For example this package is intended for people who need to be able to query WikiData in plain SPARQL while staying whithin a Jupyter Notebook.Currently data can be returned as a dataframe or as a map. SeeexamplesInstallpipinstallwikidata_plain_sparqlUsageThe module only has one function which is namedquerywhich has two parameters:query: a sparql queryview: should data be displayed as a map, function will return a data frame ifviewisNone(seeexamplesfor map use)# import the moduleimportwikidata_plain_sparqlaswikidatawikidata.query(query,view=None)ExamplesThe following examples are also available on Google Colab and can be executed without any additional setup:https://colab.research.google.com/github/jelleschutter/wikidata-plain-sparql/blob/assets/wikidata_plain_sparql_examples.ipynbData Frame# query WikiDatawikidata.query('''SELECT ?actor ?actorLabelWHERE{# tv series "Person of Interest" has actorwd:Q564345 wdt:P161 ?actor.# actor has won a Golden Globe Award?actor wdt:P166 wd:Q1011547.SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }}''')Which will result in the following pandas data frame:actoractorLabelhttp://www.wikidata.org/entity/Q3295000Sterling K. BrownMapThe following query was copied from the officalWikiData Examplesand shows all places within 1km of the Empire State Building:# query WikiDatawikidata.query('''#Places within 1km of the Empire State BuildingSELECT ?place ?placeLabel ?location ?instanceLabelWHERE{wd:Q9188 wdt:P625 ?loc .SERVICE wikibase:around {?place wdt:P625 ?location .bd:serviceParam wikibase:center ?loc .bd:serviceParam wikibase:radius "1" .}OPTIONAL {    ?place wdt:P31 ?instance  }SERVICE wikibase:label { bd:serviceParam wikibase:language "en" }BIND(geof:distance(?loc, ?location) as ?dist)} ORDER BY ?dist''',view='map')Which will result in the following map:Creditsrequests- used for sending query to WikiData serverbokeh- used for displaying map datapandas- used for returning data as data frame@jelleschutter- development of this package
Package: wikidatasets
Breaking WikiData dumps into smaller knowledge graphs (e.g. graph of human entities).Free software: BSD licenseDocumentation:https://wikidatasets.readthedocs.io.Paper:https://arxiv.org/abs/1906.04536Dataset downloads:hereCitationsIf you find this code useful in your research, please consider citing ourpaper:@article{boschin_wikidatasets_2019,
    title = {{WikiDataSets}: {Standardized} sub-graphs from {Wikidata}},
    shorttitle = {{WikiDataSets}},
    url = {http://arxiv.org/abs/1906.04536},
    journal = {arXiv:1906.04536 [cs, stat]},
    author = {Boschin, Armand and Bonald, Thomas},
    month = oct,
    year = {2019},
    note = {arXiv: 1906.04536},
    keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Computer Science - Social and Information Networks, Statistics - Machine Learning}
}
Package: wikidict
WikiDictWikipedia at your fingertipsInstall (automatic)$ pip install wikidictInstall (manual)$ git clone git@github.com:walidsa3d/wikidict.git
$ cd wikidict
$ python setup.py installLicenseThe MIT License (MIT)
Copyright (c) 2015 Walid Saad

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
OR OTHER DEALINGS IN THE SOFTWARE.
Package: wikidictparser
All rights reserved.Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.Description: README.md
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Description-Content-Type: text/markdown
Package: wikidpy
WIKIDpyThis will be the MIDAS pipeline re-written in pythonextensive use of GBO's new python pipeline for things such as the timestreams to maps is plannedIt is intended to be backwards and forwards compatible with WIKID, and MUSTANG2 (and maybe even MUSTANG!)
To Do:basic routines to read in data and save it to SD fits format.  Includes quadrant detector etc.Extend this to save it in format for OOFcmsub, pca etcCalibration pipelinegui
Package: wikidump
wikidumpIntroductionThis module contains code for manipulating wikipedia dumps available fromhttp://download.wikimedia.org/backup-index.htmlInstallationThis module is published onPyPIand can be installed with easy_installFor example:easy_install wikidumpAlternatively, you can use pip:pip install wikidumpI highly recommend usingvirtualenvto isolate the install environment.For those on ubuntu systems, a built package is available in aPPA.
Please go to the PPA for details on how to install from it.ConfigurationUpon first importing the module, a file ‘wikidump.cfg’ will be created.
Modify the paths in this file to point to your data.scratch : where indices are stores (must be writeable)xml_dumps : where the xml dumps are located (can be read-only)UsageIn addition to python modules, wikidump also comes with a command-line
tool to quickly access wikidump functionality. Runwikidump helpfor a list of options.CreditsDistributeBuildoutmodern-package-templateNews0.1Release date: 04-Aug-2010Initial release of wikidump module0.1.3Release date: 10-Apr-2013Rewrote CLI
Package: wiki-dump-parser
A simple but fast python script that reads the XML dump of a wiki and
output the processed data in a CSV file.All revisions history of a mediawiki wiki can be backed up as an XML
file, known as a XML
dump.This file is a record of all the edits made in a wiki with all the
corresponding data regarding date, page, author and the full content
within the edit.Very often we just want the metadata for the edit regarding date, author
and page; and therefore, we do not need the content of the edit, which
by far the longest piece of data.This script converts this very long XML dump in csv files much smaller
and easiest to read and work with. It takes care ofUsageInstall the package using pip:pip install wiki_dump_parserThen, use it directly from command line:python-mwiki_dump_parser <dump.xml>Or from python code:importwiki_dump_parserasparserparser.xml_to_csv('dump.xml')The output csv files should be loaded using ‘|’ as an escape character
for quoting string. An example to load the output file “dump.csv”
generated by this script using pandas would be:df=pd.read_csv('dump.csv',quotechar='|',index_col=False)df['timestamp']=pd.to_datetime(df['timestamp'],format='%Y-%m-%dT%H:%M:%SZ')Dependenciespython 3Yes, nothing more.How to get a wiki history dumpThere are several ways to get the wiki dump:If you have access to the server, follow theinstructions in the
mediawiki
docs.ForWikia wikisandmany other
domains,
you can use our in-house developed script made to accomplish this
task. It is straightforward to use and very fast on it.Wikimedia project wikis: For wikis belonging to the Wikimedia
project, you already have a regular updated repo with all the dumps
here:http://dumps.wikimedia.org.Select your target wiki from the
listand
download the complete edit history dump and uncompress it.Forother wikis, like self-hosted wikis, you should use the
wikiteam’s dumpgenerator.py script. You have a simple tutorialin
their
wiki.
Its usage is very straightforward and the script is well maintained.
Remember to use the –xml option to download the full history dump.
Package: wiki-dump-reader
Wiki-Dump ReaderExtract corpora from wiki-dump.Installpipinstallwiki-dump-readerUsageThe dump file*wiki-*-pages-articles.xmlshould be downloaded first. Then you can iterate and get cleaned text from the text:fromwiki_dump_readerimportCleaner,iteratecleaner=Cleaner()fortitle,textiniterate('*wiki-*-pages-articles.xml'):text=cleaner.clean_text(text)cleaned_text,links=cleaner.build_links(text)Just ignorelinksif you don't need them:cleaned_text, _ = cleaner.build_links(text)Seeexamplesfor an intuitive feeling.