Saravanan666/pypi-org huggingface dataset:
Package: pii-extract
Note: this repository is obsolete. It has been superseded by pii-extract-base(https://github.com/piisa/pii-extract-base) and pii-extract-plg-regex(https://github.com/piisa/pii-extract-plg-regex)Pii ExtractorThis repository builds a Python package that performs PII detection for text
data i.e. extraction of PII (Personally Identifiable Information aka Personal
Data) items existing in the text.The PII Tasks in the package are structured by language & country, since many
of the PII elements are language- and/or -country dependent.RequirementsThe packageneeds at least Python 3.8needs the pii-data base packageuses the python-stdnum package to validate identifiers, and needs theUsageThe package can be used:As an API, in two flavors: function-based API and object-based APIAs a command-line toolFor details, see the usage document.BuildingThe provided Makefile can be used to process the package:make pkgwill build the Python package, creating a file that can be
installed withpipmake unitwill launch all unit tests (using pytest, so pytest must be
available)make installwill install the package in a Python virtualenv. The
virtualenv will be chosen as, in this order:the one defined in theVENVenvironment variable, if it is definedif there is a virtualenv activated in the shell, it will be usedotherwise, a default is chosen as/opt/venv/bigscience(it will be
created if it does not exist)ContributingTo add a new PII processing task, please see the contributing instructions.
Package: pii-extract-base
Pii Extract BaseThis repository builds a Python package providing a base library for PII
detection for Source Documents i.e. extraction of PII (Personally Identifiable
Information aka Personal Data) items existing in the document.The package itself doesnotimplement any PII Detection tasks, it only
provides the base infrastructure for the process. Detection tasks must be
supplied externally.RequirementsThe package needsat least Python 3.8the pii-data base packageone or more pii-extract plugins (to actually do real detection work)UsageThe package can be used:As an API, in two flavors: function-based API and object-based APIAs a command-line toolFor details, see the usage document.BuildingThe provided Makefile can be used to process the package:make pkgwill build the Python package, creating a file that can be
installed withpipmake unitwill launch all unit tests (using pytest, so pytest must be
available)make installwill install the package in a Python virtualenv. The
virtualenv will be chosen as, in this order:the one defined in theVENVenvironment variable, if it is definedif there is a virtualenv activated in the shell, it will be usedotherwise, a default is chosen as/opt/venv/pii(it will be
created if it does not exist)
Package: pii-extract-plg-regex
Pii Extractor plugin: regexThis repository builds a Python package that installs a pii-extract-base
plugin to performs PII detection for text data based on regular expressions
(with optional context). The name of the plugin entry point ispiisa-detectors-regex.The PII Tasks in the package are structured by language & country, since many
of the PII elements are language- and/or -country dependent.RequirementsThe packageneeds at least Python 3.8needs the pii-data and the pii-extract-base base packagesuses the regex package (instead of the standardrepackage in the core
Python library)uses the python-stdnum package to validate many identifiers (and the
python-phonenumbers to validate phone numbers)UsageThe package does not have any user-facing entry points, and it is used
automatically by the PIISA framework.BuildingThe provided Makefile can be used to process the package:make pkgwill build the Python package, creating a file that can be
installed withpipmake unitwill launch all unit tests (using pytest, so pytest must be
available)make installwill install the package in a Python virtualenv. The
virtualenv will be chosen as, in this order:the one defined in theVENVenvironment variable, if it is definedif there is a virtualenv activated in the shell, it will be usedotherwise, a default is chosen as/opt/venv/bigscience(it will be
created if it does not exist)ContributingTo add a new PII processing task, please see the contributing instructions.
Package: pii-manager
Pii ManagerThis repository builds a Python package that performs PII processing for text
data i.e. replacement/tagging/extraction of PII (Personally Identifiable
Information aka Personal Data) items existing in the text.The PII Tasks in the package are structured by language & country, since many
of the PII elements are language- and/or -country dependent.RequirementsThe package needs at least Python 3.8, and uses the python-stdnum package to
validate identifiers.UsageThe package can be used:As an API, in two flavors: function-based API and object-based APIAs a command-line toolFor details, see the usage document.BuildingThe provided Makefile can be used to process the package:make pkgwill build the Python package, creating a file that can be
installed withpipmake unitwill launch all unit tests (using pytest, so pytest must be
available)make installwill install the package in a Python virtualenv. The
virtualenv will be chosen as, in this order:the one defined in theVENVenvironment variable, if it is definedif there is a virtualenv activated in the shell, it will be usedotherwise, a default is chosen as/opt/venv/bigscience(it will be
created if it does not exist)ContributingTo add a new PII processing task, please see the contributing instructions.
Package: pii-optimized-logger
Package pii_optimized_loggerConfiguration:log_disable_handler
value 0 is the default value and the handler is enable
value 1 will disable the handlerlog_max_lengthdefault value is 500
it is used to define the max length for each message to print in logSample to use the libraryimport logging
from pii_optimized_logger import log_handlerlogger = logging.getLogger()
logger.setLevel(logging.DEBUG)logger.handlers = log_handler.create()
Package: pii-preprocess
pii-preprocessThis package is intended for the data/document preprocessing stage in the PII
Management flow designed by PIISA.It will contain:a Python API and command-line entry points to read a number of file formats
and convert them to PII Source Documents, as defined by pii-dataUtilities for document transformation (to ease PII processing)ContentsThe current contents of the package are:Classes and an API for reading some file types:CSV files (into Table source documents)Microsoft Word files (into Sequence or Tree source documents)Raw text files (read plain text files into Sequence source documents
or, using indentation, into Tree source documents).A configurable loader class thar can load formats by dispatching to
appropriate subclassesSome command-line scripts:a generic script that uses the loader class to convert any implemented
format to a YAML or plain text filescripts for specific formats:a script to convert between CSV files and the YAML canonical
representation for Source Documentsa script to convert between plain text files and the YAML
canonical representation for Source Documents
Package: pii-process
pii-processFull end-to-end processing for PII (preprocess, extract, decide, transform)DescriptionThis package wraps around the relevant API blocks in the full PIISA workflow:pii-preprocess, to read document formatspii-extract(plus any installed pii-extract plugins), to detect and
extract PII instances from documentspii-decide, to consolidate the list of PII instancespii-transform, to substitute detected PII instances in documentsIt provides both a Python API and a command-line interfaceInstallationDependencies have been included in the package so that all necessary PIISA
packages are installed along. So what is needed is just:creation of a Python virtualenv (using Python >= 3.8)and installation of the package in the virtualenvChoices are:Simple installation: this will install the package, the packages for the
four above mentioned PIISA processing steps, and the extraction plugin for PII
instances using regular expressions:pip install pii-processthe dependencies installed automatically are thuspii-preprocess,pii-extract-base,pii-extract-plg-regex,pii-decideandpii-transformComplete installation: this installs all the above, plus the extraction
plugin for PII instances using trained Transformer models (usually to extract
PERSON and LOCATION types for some languages):pip install pii-processtransformersOver the previous installation, this adds also thepii-extract-plg-transformerspackage. Note thatPytorch needs to be
installed too(either GPU or CPU versionss) , so that the models used by
thepii-extract-plg-transformerspackage can run. See the transformers
plugin documentation for more information,Alternate installation: this option performs the first install, and it adds
the extraction plugin for PII instances using the Presidio library (usually
to extract PERSON and LOCATION types for some languages):pip install pii-processpresidiothe additional package installed is in this casepii-extract-plg-presidio. And in order to work the relevant models need
to be downloaded, see the presidio plugin documentation for detailsIt is also possible to install all plugins, i.e.pip install pii-processtransformers,presidio, though the Transformers and Presidio
plugins overlap in functionality (note that detection overlaps would be resolved
by thepii-decideblock).
Package: piiscan
piiscanA simple Python library that detects PII in text. Optimized for network traffic and structured data (e.g. JSON Objects, HTTP Requests).Powered bybeki/en_spacy_pii_fastandMicrosoft Presidio.InstallationThis package is published on PyPi, and can be installed usingpip.pip3 install piiscanUsageThe module exposes two methods:scan()returns alistof PresidioRecognizerResultobjects.annotate()returns a human-readablelistof all fields detected and the corresponding entity type.Scanning structured data for PIIimportpiiscanoriginal_value='''{"name": "John Doe","email": "john@doe.com","address": "123 Front Street, San Francisco, CA","phone": "+1 (415) 123-4567"}'''detected_pii=piiscan.scan(original_value)print(detected_pii)Expected result printed in console:[
    type: EMAIL_ADDRESS, start: 41, end: 53, score: 1.0, 
    type: PERSON, start: 16, end: 24, score: 0.85, 
    type: LOCATION, start: 72, end: 107, score: 0.85, 
    type: URL, start: 46, end: 53, score: 0.5, 
    type: PHONE_NUMBER, start: 127, end: 141, score: 0.4
]Annotating data to display discovered PIIimportpiiscanoriginal_value='''{"name": "John Doe","email": "john@doe.com","address": "123 Front Street, San Francisco, CA","phone": "+1 (415) 123-4567"}'''detected_pii=piiscan.scan(original_value)annotated=piiscan.annotate(original_value,detected_pii)print(annotated)Expected result printed in console (cleaned for readability):{  
    "name": "('John Doe', 'PERSON')",  
    "email": "('john@doe.com', 'EMAIL_ADDRESS'), ('doe.com', 'URL')",   
    "address": "('123 Front Street, San Francisco, CA', 'LOCATION')", 
    "phone": "('+1 (415) 123-4567', 'PHONE_NUMBER')"
}CreditsThe basis for this project is derived fromPixie Labs' blog poston using NLP to anonymize sensitive PII in structured data, and its correspondingHugging Face Space.This project usesBenjamin Kilimnik'sen_spacy_pii_fastspaCy model for NER. It is trained using adatasetderived from Privy's synthetic payload generator. The entire model is less than 7 MiB!This library was designed for use in resource-constrained environments, hence the small 7 MiBen_spacy_pii_fastmodel. However, if you would like to increased accuracy there are other alternatives described in theoriginal blog post.This library also makes use ofMicrosoft PresidioandspaCy.
Package: pii-transform
pii-transformThis package takes a source document, a collection of detected PII instances,
and transforms the document by replacing the PII instances in the document
with a different representation.The type of substitution done is defined by transformation policies.Note:pii-transformdoesnotimplement or use Transformer models for PII
purposes (for the extraction of PII instances using Transformer models, see
pii-extract-plg-transformers or pii-extract-plg-presidio).Command-line scriptThe package provides a console script:pii-transformloads a source document
& a collection of already-detected PII, and produces a transformed document
following the specified policies.APIThe same functionality provided by the command-line script can also be
accessed via a Python APIEnd-to-end workflowThe end-to-end scripts and APIs have been migrated; they are now in thepii-processpackage. That one is also the module to use to process generic
documents, since this package supports only the PIISA Source Document
format, which contains the document written as a YAML file.
Package: pijavski
PijavskiThis is an example of how to use CFFI to call a Pijavski function written in C++ that optimises a test function and returns the minimum.InstallationTo install simply type:$ pip install pijavskiUsageTo test, open a python console and import the package. The functionpijavski.get_minimum, with argumentslip,xl,xu,precisionandmaxiter, printsres,x0,f,prec,maxitas result.>>> import pijavski
>>> pijavski.get_minimum()
0 -5323.428786928975 1.2546522006214123e-09 3.5218863499790176 65533Defining custom functions to optimiseThe function to optimise needs to be declared as a callback function for CFFI so that the Pijavski program can process it. The user can implement up to 20 functions following the conditions below:The function definition needs to be preceeded by@ffi.def_extern().The function name must befun[1-20](suffix between 1 and 20) as this is how the callback function is defined in the CFFI builder.When writing the function,  argumentsfandxneed to be declared as if they were pointers using the bracket notation f[] and x[].Use numpy math functions.Example:>>> import numpy as np
>>> from pijavski import get_minimum, ffi, lib
>>> # Simple declaration of f = -cos^2(x) as callback function.
>>> @ffi.def_extern()
... def fun1(f, x):
...     f[0] = (-1)*np.cos(x[0])**2
>>> # Simple declaration of f = sin^2(x) as callback function.
>>> @ffi.def_extern()
... def fun2(f, x):
...     f[0] = np.sin(x[0])**2
>>> # Call get_minimum for fun1
>>> get_minimum(func=lib.fun1, xl=-100, xu=100)
0 -1.0 4.6838846e-317 4.6838846e-317 1 
>>> # Call get_minimum for fun2
>>> get_minimum(func=lib.fun2, xl=-100, xu=100)
0 -1.0 4.6838846e-317 4.6838846e-317 1
Package: pijaz-sdk
Pijaz SDK for [Python](https://www.python.org)### Initial platform setupSee [Initial platform setup](https://github.com/pijaz/pijaz-sdk#initial-platform-setup) in the main README for the SDK.### APIThe main API documentation resides in the [README](https://github.com/pijaz/pijaz-sdk#api) for the SDK.
Package: pikachuwechat
Documents:https://pikachuwechat.readthedocs.io/zh/latest/PikachuWeChatPikachuWeChat:PlayWeChatwithitchat-uos.
Youcanstarthisrepositorytokeeptrackoftheprojectifit'shelpfulforyou,thankyouforyoursupport.Support ListFunctionIntroductionCodein ChineseAntiWithdrawalclickclick微信消息防撤回AnalysisFriendsclickclick微信好友分析AutoReplyclickclick微信自动回复InstallPip installrun"pip install pikachuwechat"Source code install(1)Offline
Step1:gitclonehttps://github.com/CharlesPikachu/pikachuwechat.git
Step2:cdpikachuwechat->run"python setup.py install"(2)Online
run"pip install git+https://github.com/CharlesPikachu/pikachuwechat.git@master"Quick Startimportrandomfrompikachuwechatimportpikachuwechatwechat_helper=pikachuwechat.pikachuwechat()all_supports=wechat_helper.getallsupported()wechat_helper.execute(random.choice(list(all_supports.values())))ScreenshotProjects in Charles_pikachuGames: Create interesting games by pure python.DecryptLogin: APIs for loginning some websites by using requests.Musicdl: A lightweight music downloader written by pure python.Videodl: A lightweight video downloader written by pure python.Pytools: Some useful tools written by pure python.PikachuWeChat: Play WeChat with itchat-uos.Pydrawing: Beautify your image or video.ImageCompressor: Image compressors written by pure python.FreeProxy: Collecting free proxies from internet.Paperdl: Search and download paper from specific websites.Sciogovterminal: Browse "The State Council Information Office of the People's Republic of China" in the terminal.CodeFree: Make no code a reality.DeepLearningToys: Some deep learning toys implemented in pytorch.DataAnalysis: Some data analysis projects in charles_pikachu.Imagedl: Search and download images from specific websites.Pytoydl: A toy deep learning framework built upon numpy.MoreWeChat Official AccountsCharles_pikachu
Package: pik-address-formatter
pik-address-formatterThis project aim is to provide address formatter from address components of housing buildingExample:fromaddress_formatterimportall_formatsaddress_components={"region":"Курганская","region_type_full":"область","area":"Катайский","area_type_full":"район","city":"Серов","city_type_full":"город","city_district":"Кировский","city_district_type_full":"округ","settlement":"Дрянное","settlement_type_full":"село","street":"Майская","street_type_full":"улица","house":"5","house_type_full":"дом","section":"6","building":"7",}print(all_formats("plain address ",address_components,"5",7)['all'])Курганская обл., Катайский р⁠-⁠н, г. Серов, Кировский окр., с. Дрянное, ул. Майская, д. 5, корп. 6, стр. 7, м. 45all_formats return dict of address formats
        all - full address with region, district, city, township, etc
        street_only - street or village
        finishing_with_village - region, district, city, township and village
        starting_with_street - street, building, section, construction, premise
        finishing_with_street - region, district, city, township, village, streetHowToUseAdd pik-address-formatter to requirements.txtpik-address-formatter>=1.0,<2.0Add address_formats method to target modelfromdjango.dbimportmodelsfromaddress_formatterimportall_formatsclassAccount(models.Model):...@propertydefaddress_formats(self):returnall_formats(self.premise.address,self.building.user_address_components,self.premise.user_number,self.building.type,)If you dosn't have a premise model, just use without premise datafromdjango.dbimportmodelsfromaddress_formatterimportall_formatsclassAccount(models.Model):...@propertydefaddress_formats(self):returnall_formats(self.building.address,self.building.user_address_components,building_type=self.building.type,)For details see docstring of all_formats
Package: pika-dev
Pika is a RabbitMQ (AMQP-0-9-1) client library for Python.IntroductionPika is a pure-Python implementation of the AMQP 0-9-1 protocol including RabbitMQ’s
extensions.Python 2.7 and 3.4+ are supported.Since threads aren’t appropriate to every situation, it doesn’t
require threads. It takes care not to forbid them, either. The same
goes for greenlets, callbacks, continuations and generators. It is
not necessarily thread-safe however, and your mileage will vary.People may be using direct sockets, plain oldselect(),
or any of the wide variety of ways of getting network events to and from a
python application. Pika tries to stay compatible with all of these, and to
make adapting it to a new environment as simple as possible.DocumentationPika’s documentation can be found athttps://pika.readthedocs.ioExampleHere is the most simple example of use, sending a message with the BlockingConnection adapter:importpikaconnection=pika.BlockingConnection()channel=connection.channel()channel.basic_publish(exchange='example',routing_key='test',body='Test Message')connection.close()And an example of writing a blocking consumer:importpikaconnection=pika.BlockingConnection()channel=connection.channel()formethod_frame,properties,bodyinchannel.consume('test'):# Display the message parts and ack the messageprint(method_frame,properties,body)channel.basic_ack(method_frame.delivery_tag)# Escape out of the loop after 10 messagesifmethod_frame.delivery_tag==10:break# Cancel the consumer and return any pending messagesrequeued_messages=channel.cancel()print('Requeued%imessages'%requeued_messages)connection.close()Pika provides the following adaptersAsyncioConnection  - adapter for the Python3 AsyncIO event loopBlockingConnection - enables blocking, synchronous operation on top of library for simple usesSelectConnection   - fast asynchronous adapterTornadoConnection  - adapter for use with the Tornado IO Loophttp://tornadoweb.orgTwistedConnection  - adapter for use with the Twisted asynchronous packagehttp://twistedmatrix.com/ContributingTo contribute to pika, please make sure that any new features or changes
to existing functionalityinclude test coverage.Pull requests that add or change code without coverage will most likely be rejected.Additionally, please format your code usingyapfwithgooglestyle prior to issuing your pull request.
Package: pikahandler
PikaHandlerКласс-хендлер для удобной работы с обработкой сообщений из RabbitMQУстановкаpython-mpipinstallpikahandlerИспользованиеfrompikahandlerimportPikaHandlerserver='192.168.0.73'port=8472host="/"user="eugene"password="spell-thickness-macarena-belie-gawk-fiske-puffy"# Без авторизацииph=PikaHandler(server,port,host)# С авторизациейph=PikaHandler(server,port,host,user,password)# Через connectionimportpikaconnection=pika.BlockingConnection(pika.ConnectionParameters(server,port,host))ph=PikaHandler(connection=connection)@ph.handler("main",lambdar:r["body"]=="HelloWorld")defhelloworld(ch,method,poperties,body):print(body)@ph.handler("main",lambdar:r["body"]=="HelloPython")defhellopython(ch,method,poperties,body):print(body)ph.start()
Package: pika-multithreaded
# Pika (Multithreaded)## IntroductionThis repository uses Pika as a base to create connections to AMQP brokers. Pika
doesn’t use threads at all so it fails when using a consumer for long lived
tasks. This project builds on top of Pika to enable long lived task execution
for consumers by implementing a threading layer on top of the base Pika
connections.