Saravanan666/pypi-org huggingface dataset:
Package: pars-hitmotop
ОглавлениеОглавлениеЧто именно парсит?Как использовать модуль entered_tracksКак использовать модуль rating_tracks_countКак использовать модуль rating_tracks_pageЧто можно достать при запросе?Этот проект парситмузыкальный сайтЧто именно парсит?Рейтинговые трекиот 1 до 48;Тожерейтиновые трекино можно выбрать количество страниц, с которых будет произведен парсинг;Треки введеные пользователем. Парсит от 1 трека до конечной страницы (на одной старнице 48 треков)Как использовать модульentered_tracksfrom pars_hitmotop.entered_tracks import EnteredTrack
result=EnteredTrack('linkin park',10)1 аргументом (musci_name) передается название пенси или автора. 2 Аргументом (count) передается количество трековКак использовать модульrating_tracks_countfrom pars_hitmotop.rating_tracks_count import RatingCount
result=RatingCount(10)1 аргументом (count) передается количество песенКак использовать модульrating_tracks_pagefrom pars_hitmotop.rating_tracks_page import RatingPage
result=RatingPage(10)1 аргументом (count) передается количество страниц (max 11)Что можно достать при запросе?Все вовзращается в виде listМетодОписаниеresult.get_authorПолучить автора трекаresult.get_titleПолучить название трекаresult.get_url_downПолучить ссылку на скачивание трекаresult.direct_download_linkПолучть пряму ссылку на скачивание трекаresult.get_duractionПолучить продолжительность трекаresult.get_picture_urlПолучить ссылку на обложку трекаresult.get_url_trackПолучить ссылку трекresult.get_allПолучить все данные в виде словаря
Package: parshu
Filter URLs to save your time using regexParshuuses regex to filter out the custom results. Remembering every regex or writing regexes for a task which you do daily is not easy, soparshuwill help you to automate the task.Inspired fromtomnomnom's gftoolFeaturesYou can also add your custom regex to get resultsFilter out all the query parameters which you use to check vulnerabilitiesInstallation$pip3installparshu$wget-Oparshuhttps://raw.githubusercontent.com/R0X4R/Parshu/main/parshu.py-q&&chmod+xparshu&&mvparshu/usr/bin/UsageForlinux,unixanddebianbased systems$waybackurlssub.domain.tld|parshuhttp://sub.domain.tld/wvstests/http://sub.domain.tld/wvstests/pmwiki_2_1_19http://sub.domain.tld/wvstests/pmwiki_2_1_19/scriptshttp://sub.domain.tld/wvstests/pmwiki_2_1_19/scripts/Forwindowsbased systemscmd> type urls.txt | python3 parshu.pyIfno argumentspassed$waybackurlssub.domain.tld|parshuhttp://sub.domain.tld/wvstests/http://sub.domain.tld/wvstests/pmwiki_2_1_19http://sub.domain.tld/wvstests/pmwiki_2_1_19/scriptshttp://sub.domain.tld/wvstests/pmwiki_2_1_19/scripts/To filter out the results for xssNote: To get parameters for other vulnerabilites use other arguments. To get all the arguments useparshu -h$waybackurlstestphp.vulnweb.com|parshu-xhttp://testphp.vulnweb.com/artists.php?artist=1http://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpghttp://testphp.vulnweb.com/showimage.php?file=./pictures/1.jpg&size=160DonateIf this tool helped you or you like my workDonate Now
Package: parsinghook
Python Parsing HookGitHub:https://github.com/acaos/python-parsinghookPyPI:https://pypi.org/project/parsinghook/This package adds support for easily adding parsing hooks to Python, using a
special comment at or near the top of a file.UsageAfter this package has been installed, it is activated by placing the
following at the top of any module you wish to use an alternate parser
for:# -*- parsing: <parsing-module> -*-Note that this will not work for code executed directly from the command
line (e.g.python3 foo.py), but will work if you execute the code as
a module (e.g.python3 -m foo).Technical InformationParsing HooksOperationWhen a parsing hook module is found, that module is imported, and theparse_module()function in that module is invoked. The expected return
of theparse_module()function is an AST tree.Theparse_module()functionMUSTaccept at least a file object and
filename string as arguments, andMUSTalso accept arbitrary keyword
arguments after those two arguments.Existing Parsing HooksKnown existing parsing hooks include:parsinghook.dumpast (from this package)pep505Modifying the Wheel / Installing from SourceNote that after usingpython3 -m buildto build the wheel, it is necessary
to manually add theparsinghook.pthfile to the wheel, with (for example):cdsrc
zip-g../dist/parsinghook-*-py3-none-any.whlparsinghook.pthcd..If installing from source, thesrc/parsinghook.pthfile must be placed in yoursite-packagesdirectory.Future DevelopmentParser OptionsIn the future, theparsingcomment might be updated to support optional
arguments to parsers, e.g.:# -*- parsing: parsinghook.dumpast colorize=True -*-MutatorsIn the future, additional magic comments might be supported such as:# -*- mutator: <mutation-module> -*-Which would allow other modules to modify the AST after the initial parsing.Thanks and CreditsThis package stands on the shoulders of giants.Theparsinghook.hook.activate()function is based on André Roberge'sideaspackage.
Package: parsingmo
moparsing: A Python package for ...Installation - developmentCreate a virtual environment, for example usingvenv:$python-mvenvvenv-moparsing
$sourcevenv-moparsing/bin/activate
$mkdirmoparsing
$cdmoparsing
$python-mpipinstall-e.[dev]Note that the last command installs a development environment, as it also installs packages needed for development, likeblack(for code formatting),pytest(for unit testing) andwheel(for creating wheel files from the package).TestingRunning the tests requires to run the following command in the root folder (of course in the virtual environment):(venv-moparsing)>pytestIf you use doctests in your docstrings (asmakepackageassumes), you can run them using the following command (in the root folder):(venv-moparsing)>python-mdoctestmoparsing/moparsing.pyIn a similar way, you can run doctests from any other file that contains doctests.VersioningRemember to update package version once a change is made to the package and the new version is pushed to the repository. Don't forget about releases, too.How to build a Python package?To build the package, you need to go to the root folder of the package and run the following command:(venv-moparsing)>pythonsetup.pysdistbdist_wheelNote that this assumes you havewheelinstalled in your virtual environment, andmakepackagedoes this for you.The built package is now located in the dist/ folder.Publishing your package in PyPiIf you want to publish it toPyPi, you need to installtwine, create an account there, and run the following command (also in the package's root folder):(venv-moparsing)>twineuploaddist/*Nonetheless, if you first want to check what it will look like in PyPi, you can first upload the package toa test version of PyPi, that is,twineupload-rtestpypidist/*Check if everything is fine, and if so, you're ready to publish the package to PyPi.Installation from PyPiIf the package is in PyPi, you can install it from there like any other Python package, that is,pipinstall
Package: parsing-steps
Parsing stepsA simple helper for data parsing.SimpleParsingStep usage exampleimporttimefromparsing_stepsimportSimpleParsingStepclassParseMainPage(SimpleParsingStep):"""input_data is main page url.input_data = "https://domain.com/""""defdownload(self,url):time.sleep(0.1)return'<html>...<a href="https://domain.com/category/15/"></a>...</html>'defget_category_urls(self,html_data):"""Plug"""return[f"https://domain/category/{num}/"fornuminrange(15,16)]defparse(self,input_data):url=input_datahtml_data=self.download(url)forcategory_urlinself.get_category_urls(html_data):yieldParseListStep({"url":category_url})classParseListStep(SimpleParsingStep):"""input_data from previous step.input_data = {"url": "https://domain.com/category/<id>/"}"""defdownload(self,url):time.sleep(0.1)if"?page=2"inurl:return"<html>...</html>"return'<html>...<a href="https://domain.com/category/15/?page=2"></a>...</html>'defget_item_urls(self,html_data):"""Plug"""return[f"https://domain/category/15/item/{num}/"fornuminrange(10)]defexists_next_page(self,html_data):return"?page="inhtml_datadefparse(self,input_data):print(f"category_url:{input_data.get('url')}")#  here you get data from html, through bs4 or something else.html_data=self.download(input_data.get("url"))foritem_urlinself.get_item_urls(html_data):yieldParseDetailsStep(input_data={"url":item_url},inherited_data={# optional part"category_id":15,"page":1ifself.exists_next_page(html_data)else2})ifself.exists_next_page(html_data):# get this data from htmlnext_page_url="https://domain.com/category/15/?page=2"yieldParseListStep(input_data={"url":next_page_url})classParseDetailsStep(SimpleParsingStep):"""input_data from previous step.input_data = {"url": "https://domain/category/15/item/{num}/"}"""defdownload_and_format(self,url):"""plug"""time.sleep(0.2)num=int(list(filter(None,str(url).split("/")))[-1])# returns numpage=int(self.inherited_data.get("page"))return{"id":page*10+int(num),"name":f"product_{page*10+num}","price":num*10,}defparse(self,input_data):product_data=self.download_and_format(input_data.get("url"))yieldSaveStep(product_data)classSaveStep(SimpleParsingStep):"""input_data from previous step.input_data = {"id": 12345,"name": "product","price": 16.50,}"""defsave_to_db(self,data_as_dict):self.inherited_data=self.inherited_dataordict()data_as_dict.update(self.inherited_data)print(data_as_dict)defparse(self,input_data):self.save_to_db(input_data)if__name__=="__main__":first_step=ParseMainPage(input_data="https://domain.com/")first_step.perform()# start scraping
Package: parsing-tools
The MIT License (MIT)This repository contains multiple small projects each of which can be used
separately. There are, at this point, no dependencies between them.This would be a great package to use when creating your own custom Django
REST Framework renderers and parsers based on custom MIME types.Current ProjectsmimeparserThis tool would be used to parse HTTP headers to derive the ‘best fit’ mime
type. It handles suffix and quality parsing and can be used to find the best
match from anAcceptheader from a list of available mime types.xml2dictThis tool should be able to parse any XML document into Python objects. The
output will become very verbose since it will include all attributes,
elements and namespaces found in the XML document.
Package: parsini
Esta librería permite leer y modificar fichero de configuración en tiempo de ejecución.Los parámetros en los ficheros de configuración se dividirán en sectores al estilo de la libreríaconfigparser.Uso de la clase y ejemplos:https://github.com/gmolino/parsiniVersiones0.0.1Parsea fichero de configuración, crear nuevos parámetros y/o actualiza los existentes.Sobrescribe el fichero de configuración o crear copia con los nuevos parámetros o actualizados.0.0.2from parsini import Parsini (fixed)0.0.3No crea nuevo parámetro si existente.Comprueba el nombre del valor antes de actulizar para modificar repeticiones.0.0.4Recarga parámetros por defecto en la llamada al métodoread(True)para evitar incongruencia en caso de múltiples instancias.0.0.5fixed0.4 y Readme
Package: parsita
ParsitaThe executable grammar of parsers combinators made available in the executable pseudocode of Python.Parsita is a parser combinator library written in Python. Parser combinators provide an easy way to define a grammar using code so that the grammar itself effectively parses the source. They are not the fastest at parsing, but they are the easiest to write.Like all good parser combinator libraries, Parsita abuses operators to provide a clean grammar-like syntax. The__or__method is defined so that|tests between two alternatives. The__and__method is defined so that&tests two parsers in sequence. Other operators are used as well.In a technique that I think is new to Python, Parsita uses metaclass magic to allow for forward declarations of values. This is important for parser combinators because grammars are often recursive or mutually recursive, meaning that some components must be used in the definition of others before they themselves are defined.See theDocumentationfor the full user guide.InstallationThe recommended means of installation is withpipfrom PyPI.pipinstallparsitaHello worldThe following is a very basic parser for extracting the name from aHello, {name}!string.fromparsitaimport*classHelloWorldParsers(ParserContext,whitespace=r'[ ]*'):hello_world=lit('Hello')>>','>>reg(r'[A-Z][a-z]*')<<'!'# A successful parse produces the parsed valuename=HelloWorldParsers.hello_world.parse('Hello, David!').unwrap()assertname=='David'# A parsing failure produces a useful error messagename=HelloWorldParsers.hello_world.parse('Hello David!').unwrap()# parsita.state.ParseError: Expected ',' but found 'David'# Line 1, character 7## Hello David!#       ^
Package: parslepy
Parslepy lets you extract content from HTML and XML documents
where extraction rules are defined using a JSON object
or equivalent Python dict,
where keys are names you want to assign to extracted content,
and values are CSS selectors or XPath expressions.Parslepy is an implementation of the Parsley extraction
language definedhere,
using lxml and cssselect.You can nest objects, generate list of objects, and (to
a certain extent) mix CSS and XPath.Parslepy uderstands what lxml and cssselect understand,
which is roughly CSS3 selectors and XPath 1.0 expressions.Documentation & examplesSeehttps://github.com/redapple/parslepy/wiki#usage
Package: parspy
parspyA tool to parse text into a list where it can be interpreted and executed.InstallInstall usingpip install parspy.UsageFirst, import and initiate the class:importreimportparspyparsertool=parspy.Parser()Then, add elements to the alphabet:parsertool.addelems("ad","bc"," ")If you have things more complicated than static elements, use thesetregelemsmethod by feeding it compiled regex objects:parsertool.setregelems(number=re.compile("Number: \d+"),string=re.compile("String: '.*'"))But what use is this without using it to parse something? None. Here is how you parse a string with this:parsertool.parse("adbcbcbcad  Number: 123      String: 'banana!'")The output should be:['ad','bc','bc','bc','ad',' ',' ',{'number':'Number: 123'},' ',' ',' ',' ',' ',' ',{'string':"String: 'banana!'"}]Now you can pass this into a program where it can be processed more easily.If something goes wrong during parsing, aParseErrorwill be thrown.NotesIt seems to have an issue when executing in a REPL environment (name 're' is not defined).
Package: parsr-client
Parsr ClientProvides a python interface to the Parsr tool via its API.
Parsr transforms PDF, documents and images into enriched, structured data.Find out all about Parsr (including download) athttps://github.com/axa-group/Parsr.1 Installationpipinstallparsr-client2 UsageMake sure that the Parsr Server is already running. Let us suppose that the address islocalhost:30012.1 Connect to the Parsr serverfromparsr_clientimportParsrClientparsr=ParsrClient('localhost:3001')2.2 Send the documentparsr.send_document(file_path='README.pdf',config_path='defaultConfig.json'document_name='The Readme',save_request_id=True)2.4 Retrieve resultsGet everything as a JSON:parsr.get_json()As Markdown:parsr.get_markdown()As text:parsr.get_text()Get the first table on the first page:parsr.get_table(page=1,table=1,)Get all the versions of the document:parsr.get_revisions('The Readme')Get pretty diffs between each successive pair of a document's revisions:parsr.compare_revisions('The Readme',pretty_html=True)3 Interpreting the whole JSON output locallyThe suppliedParsrOutputInterpreterclass can be used to interpret the downloaded JSON output and generate higher level structures like the text body.Here's an example to generate text body on the first page from the above example.fromparsr_clientimportParsrOutputInterpreterparsr_interpreter=ParsrOutputInterpreter(parsr.get_json())t=parsr_interpreter.get_text(page_number=1)print(t)
Package: parsrt
Module parsrtA minimalistic (100LOC) library to read SRT files into a workable format.Simply provides a way to convert SRT strings into Sub objects (via the parse
functions) and vice versa (via the Sub class'strfunction).
By avoiding regex it gets speeds around 50% faster than the fastest python
SRT/Subtitle library I could find (cdown/srt).
Install via: pip install parsrtFunctionsparse_file(path, encoding='ascii'):   Reads a file and converts the contents into an Sub object array.Args:
    path: The path the SRT file is located at.
    encoding: The encodding of the file (same as with open()).

Returns:
    An Sub object array.parse_str(srt):   Converts a string into an Sub object array.Args:
    srt: A valid SRT string.

Returns:
    An Sub object array.

Raises:
    ValueError: Invalid SRT string.ClassesSub(index, time, text):   Subtitle info object.Attributes:
    index: An SRT index value.
    time: A tuple containing the start and end timestamps as timedelta objects.
    text: A string value containing the subtitle's text.
Package: parstools
parstoolsParsing algorithms, currently:LR(1)LR(1) with state-mergingplotting LR state-machinesprinting of LR tablesgrammar definition viap_...methodslexer definition viat_...methodsstateful lexingregular language matching using nondeterministic automataconversions between regular grammars, regexes, NFArepresenting grammars, productionsprinting trees
Package: parsuricata
parsuricataParse Suricata rulesInstallationpipinstallparsuricataUsagefromparsuricataimportparse_rulessource='''alert http $HOME_NET any -> !$HOME_NET any (msg: "hi mum!"; content: "heymum"; http_uri; sid: 1;)'''rules=parse_rules(source)print(rules)## alert http $HOME_NET any -> !$HOME_NET any ( \#   msg: hi mum!; \#   content: heymum; \#   http_uri; \#   sid: 1; \# )rule=rules[0]print(rule.action)# alertprint(rule.protocol)# httpprint(rule.src)# $HOME_NETprint(rule.src_port)# anyprint(rule.direction)# ->print(rule.dst)# !$HOME_NETprint(rule.dst_port)# anyforoptioninrule.options:print(f'{option.keyword}={option.settings}')## msg = hi mum!# content = heymum# http_uri = None# sid = 1
Package: parsy
Parsy is an easy and elegant way to parse text in Python by combining small
parsers into complex, larger parsers. If it means anything to you, it’s a
monadic parser combinator library for LL(infinity) grammars in the spirit ofParsec,Parsnip, andParsimmon. But don’t worry, it has really good
documentation and it doesn’t say things like that!Parsy requires Python 3.7 or greater.For a good example of the kind of clear, declarative code you can create using
parsy, see theSQL SELECT statement exampleorJSON parser.Links:DocumentationHistory and changelogPyPITo contribute, please create a fork and submit a pull request on GitHub, after
checking thecontributingsection of the
docs. Thanks!If you like parsy and think it should be better known, you could:Star this project on GitHub.Votefor it being included on awesome-python.Parsy was originally written byJeanine Adkisson,
with contributions by other people as can be found in the git commit history.
Package: parsy-extn
Some small extensions to parsyThis offers a small extension to the behaviour of Parsy.Note-keepingInput streams gain the ability to keep notes. These are additional information
associated with a particular index in the stream. As the parsing process proceeds,
new notes may be written. An attempt to fetch the most recent set of notes can be performed
by:recent_notes = yield parsy_extn.get_notesand notes at the current point in parsing may be written byyield put_notes(new_notes)As the parser backtracks, older sets of notes may be thrown away.Notes on the mutability of notesNotes are kept in a dictionary.get_notesperforms a shallow copy of this dictionary;
however, references to embedded objects are identical.This is intentional; the utility of notes is in allowing a parser to back-fill
already-generated placeholder values with actual content.However, care should be taken when writing new sets of notes that references to other
mutable objects (such as lists) are not duplicated accidentally; this will result in
the backtracking process losing the immutability of previous notes.Handling augmentationManually augmenting the streamAn inputstrorlistmay be augmented before being passed to a parser. This should
be done at the initial point of ingestion into the grammar:result = grammar_entry_point.parse(parsy_extn.Noted.augment(original_input))Wrapping a single entry-point'sparsemethodAlternatively, a grammar's entry-point may be decorated to automatically wrap values as
they are passed to theparsemethod:grammar_entry_point = parsy_ext.keeps_notes(grammar_entry_point)
result = grammar_entry_point.parse(original_input)Monkey-patchingparsy.ParserFinally, theparsy.Parserobject itself may be monkey-patched to wrap any input on
entry toparseorparse_partial:parsy_extn.monkeypatch_parsy()
result = grammar_entry_point.parse(original_input)MotivationThe motivation for this is to handle constructs like shell 'heredocs' with a single
pass of a parser. They are otherwise notoriously difficult to handle; even with
this facility, the resulting grammars are fiddly.A trivial example is given in thetest_parsy_extnmodule.
Package: parsyfiles
Project page :https://smarie.github.io/python-parsyfiles/What’s newTravis and codecov integrationDoc now generated from markdown usingmkdocsWant to contribute ?Contributions are welcome ! Simply fork this project on github, commit
your contributions, and create_not_able_to_convert pull requests.Here is a non-exhaustive list of interesting open topics:https://github.com/smarie/python-parsyfiles/issuesRunning the testsThis project usespytest.pytest-vparsyfiles/tests/You may need to install requirements for setup beforehand, usingpipinstall-rci_tools/requirements-test.txtGenerating the documentation pageThis project usesmkdocsto generate its documentation page.
Therefore building a local copy of the doc page may be done using:mkdocsbuild-fdocs/mkdocs.ymlYou may need to install requirements for doc beforehand, usingpipinstall-rci_tools/requirements-doc.txtGenerating the test reportsThe following commands generate the html test report and the associated
badge.pytest--junitxml=junit.xml-vparsyfiles/tests/ant-fci_tools/generate-junit-html.xmlpythonci_tools/generate-junit-badge.pyPyPI Releasing memoThis project is now automatically deployed to PyPI when a tag is
created. Anyway, for manual deployment we can use:twineuploaddist/*-rpypitesttwineuploaddist/*
Package: parsync
parsyncParsync lets you recursively copy missing files to a directory.This is equivalent tocp -r --ignore-existing SRC DSTbut using concurrent copies to accelerate the process.ExampleCopy the content of the current directory:parsync * /destCopy with 2 threads:parsync -p 2 src destCopy only jpeg files and skip a specific directory, note the usage ofpython regex syntax:parsync --filter='\.jpg$' --exclude='/somedir/' src destUsageusage: parsync [-h] [--procs PROCS] [--filter [FILTER ...]] [--exclude [EXCLUDE ...]] [--version] SRC [SRC ...] DST

Recursively copy missing files to a directory.

This is equivalent to: cp -r --ignore-existing SRC DST

Note: matching patterns use python regex syntax.

positional arguments:
  SRC                   source files and directories
  DST                   destination directory

options:
  -h, --help            show this help message and exit
  --procs PROCS, -p PROCS
                        maximum number of concurrent copies
  --filter [FILTER ...], -f [FILTER ...]
                        pattern to filter source files
  --exclude [EXCLUDE ...], -e [EXCLUDE ...]
                        pattern to exclude source files
  --version, -v         show program's version number and exit