Saravanan666/pypi-org huggingface dataset:
Package: pgsql-network-diagram
Copyright (c) 2018 The Python Packaging AuthorityPermission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the “Software”), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.Description: Parses SQL and builds an interactive table relational diagram using vis.js. Built on sqlparse and focused on PostgreSQL.
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 2.7
Package: pgsqlx-generator
Installpipinstallpgsqlx-generatorUsage Samplefrompgsqlx.generatorimportGeneratorif__name__=='__main__':coder=Generator(host='127.0.0.1',port='3306',user='xxx',password='xxx',database='test')# you can generate a model class for one tablecoder.generate_with_tables(tables='user',path='models.py')# you can generate model classes for tablescoder.generate_with_tables(tables=['user','person'],path='models.py')# you can generate model classes for all tables from a given schema. default current schema if not givencoder.generate_with_schema(schema='test',path='models.py')If you run last code, then generate a file ‘models.py’ in current
directory like follow:fromdecimalimportDecimalfromdatetimeimportdate,datetimefrompgsqlx.ormimportModel,PkStrategyclassBaseModel(Model):__pk__='id'__del_flag__='del_flag'__update_by__='update_by'__update_time__='update_time'__pk_strategy__=PkStrategy.DB_AUTO_INCREMENTdef__init__(self,id:int=None,create_by:int=None,create_time:datetime=None,update_by:int=None,update_time:datetime=None,del_flag:int=None):self.id=idself.create_by=create_byself.create_time=create_timeself.update_by=update_byself.update_time=update_timeself.del_flag=del_flagclassPerson(BaseModel):__table__='person'__key_seq__='person_id_seq'def__init__(self,id:int=None,name:str=None,age:int=None,birth_date:date=None,sex:int=None,grade:float=None,point:float=None,money:Decimal=None,create_by:int=None,create_time:datetime=None,update_by:int=None,update_time:datetime=None,del_flag:int=None):super().__init__(id=id,create_by=create_by,create_time=create_time,update_by=update_by,update_time=update_time,del_flag=del_flag)self.name=nameself.age=ageself.birth_date=birth_dateself.sex=sexself.grade=gradeself.point=pointself.money=moneyPgSqlx:https://pypi.org/project/pgsqlx
Package: pgsrip
PGSRipRip your PGS subtitles.Project pagehttps://github.com/ratoaq2/pgsripPGSRipis a command line tool that allows you to extract and convert
PGS subtitles into SRT format. This tool requires MKVToolNix and
tesseract-ocr and tessdata (https://github.com/tesseract-ocr/tessdataorhttps://github.com/tesseract-ocr/tessdata_best)Installationpgsrip:$ pip install pgsripMKVToolNix:[Linux/WSL - Ubuntu/Debian]
$ sudo apt-get install mkvtoolnix

[Windows/Chocolatey]
$ choco install mkvtoolnixtesseract:PPA is used to install latest tesseract 5.x. Skip PPA repository if you decide to stick with latest official Debian/Ubuntu package[Linux/WSL - Ubuntu/Debian]
$ sudo add-apt-repository ppa:alex-p/tesseract-ocr5
$ sudo apt update
$ sudo apt-get install tesseract-ocr

[Windows/Chocolatey]
$ choco install tesseract-ocrtessdata:$ git clone https://github.com/tesseract-ocr/tessdata_best.git
export TESSDATA_PREFIX=~/tessdata_bestIf you prefer to build the docker image Build Docker:$ git clone https://github.com/ratoaq2/pgsrip.git
cd pgsrip
docker build . -t pgsripUsageCLIRip from a .mkv:$ pgsrip mymedia.mkv
3 PGS subtitles collected from 1 file
Ripping subtitles  [####################################]  100%  mymedia.mkv [5:de]
3 PGS subtitles ripped from 1 fileRip from a .mks:$ pgsrip mymedia.mks
3 PGS subtitles collected from 1 file
Ripping subtitles  [####################################]  100%  mymedia.mks [3:pt-BR]
3 PGS subtitles ripped from 1 fileRip from a .sup:$ pgsrip mymedia.en.sup
1 PGS subtitle collected from 1 file
Ripping subtitles  [####################################]  100%  mymedia.en.sup
1 PGS subtitle ripped from 1 fileRip from a folder path:$ pgsrip -l en -l pt-BR ~/medias/
11 PGS subtitles collected from 9 files / 2 files filtered out
Ripping subtitles  [####################################]  100%  ~/medias/mymedia.mkv [4:en]
11 PGS subtitles ripped from 9 filesUsing docker:$ docker run -it --rm -v /medias:/medias -u $(id -u username):$(id -g username) ratoaq2/pgsrip -l en -l de -l pt-BR -l pt /medias
11 PGS subtitles collected from 9 files / 2 files filtered out
Ripping subtitles  [####################################]  100%  /medias/mymedia.mkv [4:en]
11 PGS subtitles ripped from 9 filesAPIfrompgsripimportpgsrip,Mkv,OptionsfrombabelfishimportLanguagemedia=Mkv('/subtitle/path/mymedia.mkv')options=Options(languages={Language('eng')},overwrite=True,one_per_lang=False)pgsrip.rip(media,options)
Package: pg-stage
pg_stageA utility for generating a database dump, the data in which will be obfuscated. This dump can be used in development and
stage servers without fear of their theft.ContentHow does it work?Usage exampleSupported types of obfuscationWhy did I write my utility?Thanks for the inspirationHow does it work?The utility processes the output of the pg_dump command line by line and decides whether to obfuscate data at the level
of comments to a table or column.Usage exampleYou need to create a file with approximately the following contents:# main.pyfrompg_stage.obfuscatorimportObfuscatorobfuscator=Obfuscator(locale='ru_RU')obfuscator.run()Add comments to a column or table:COMMENTONCOLUMNtable_1.first_nameIS'anon: [{"mutation_name": "first_name"}]';Run pg_dump and redirect the stream to the running script process:pg_dump-ddatabase|python3main.py>dump.sqlAfter that you will get the obfuscated data in the tableSupported types of obfuscationYou can see the current listhere.Why did I write my utility?I also adhere to the rule that you do not need to place third-party plugins in the working database for its security
(most utilities are in the form of database extensions).Also, in similar utilities, I could not find the functionality for uniform obfuscation of data in related tables.
This prompted me to write my own utility that will be able to obfuscate data in related tables with the same result
by a foreign key.Example:COMMENTONCOLUMNtable_1.first_nameIS'anon: [{"mutation_name": "first_name", "relations": [{"table_name": "table_1", "column_name": "last_name", "from_column_name": "id", "to_column_name": "id"}]}]';whererelations- links on tables where it is necessary to obfuscate fields according to the current field.Thanks for the inspirationtrikifake_pipe
Package: pg_stats
Tool: pg_statsDescription:************This program iterates over all the databases and their datasets, get and shows these info:- Average of rows per database- Total rows in the entire serverHow to:******Note: The next commands are meant to be run in the same that directory in which this Readme.txt file is stored.- Disable any previously activated virtualenv:$ deactivate- Setup environment:from this directory, run the next command:$ ../../setup_env.sh- Enable virtualenv:$ source venvs/3_stats/bin/activate- Get the program help:$ pg_stats -v- Run the tool:$ pg_stats
Package: pgstatus
HTTP daemon to check postgresql server status for haproxyThis repository implements a permanent HTTP daemon to monitor postgresql database state
and to decide which server in configuration is active master.Response from tool is used by haproxy to redirect traffic to writable postgres master server, to readonly slave
servers and to detect unavailable postgres servers.Configuration filesConfiguration path is ~/.config/pgstatus.conf. Configuration file is required if you want to monitor
a postgres server that is not localhost:5432 as user postgres and not using postgres as database to check.See example configuration file in examples directory.See example configuration for haproxy in examples/haproxy.conf.Running the serviceThere is example systemd service unit available in examples/pgstatus-daemon.service, configured by default
to run as postgres user (so normally no user and password needs to be configured for default postgres).To install:cp examples/pgstatus-daemon.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable pgstatus-daemon
systemctl start pgstatus-daemonCreditsThe basic idea is based on websitehttps://severalnines.com/blog/postgresql-load-balancing-using-haproxy-keepalivedInstead of xinetd and shell script, this implements the check as a systemd controlled
permanent daemon.
Package: pg-stream-copy
PG Stream CopyFast COPY TO postgresql table directly from python by converting input data to bytes and stream to psycopg2 cursor usingCOPY <table> FROM STDIN BINARYBenchmark:The test with 1 mln rows of different column types on docker environment gave results:~21.5s forpg_stream_copy~54s forpsycopg2.extras.execute_valuesUsage:fromdatetimeimportdatefrompsycopg2importconnectfrompg_stream_copyimportSchema,WriterEncoderconn=connect('postgresql://postgres@localhost')cursor=conn.cursor()table_name='public.example_table'cursor.execute(f'''CREATE TABLE{table_name}(_smallint SMALLINT NULL,_integer INTEGER NULL,_bigint BIGINT NULL,_float DOUBLE PRECISION NULL,_numeric NUMERIC NULL,_character_varying CHARACTER VARYING NULL,_date DATE NULL);''')schema=Schema.load_from_table(cursor,table_name)withWriterEncoder(cursor,table_name,schema)aswriter_encoder:writer_encoder.append_tuple((2,3,4,2.34,'foo bar',date(2019,2,1)))writer_encoder.append_dict({'_smallint':200,'_integer':300,'_bigint':400,'_float':234,'_numeric':Decimal("-12.34")'_character_varying':'bar baz','_date':date(2019,2,3),})conn.commit()conn.close()Supported PostgreSQL types:booleansmallintintegerbigintdouble precisionnumericcharacter varyingtextdatetimestamptimestamp with time zonejsonjsonbDevelopment:# prepare envdocker-composerunpybash
python-mvenvvenv
pipinstall-e.[dev,e2e]# run testspytesttests/If you need to test different PostgreSQL and Python version, you can use env vars:PYTHON_VERSION=3.7PG_VERSION=10docker-composebuildPYTHON_VERSION=3.7PG_VERSION=10docker-composerunpy...or use CI script:PG_VERSION=11PYTHON_VERSION=3.7./bin/tests.shcurrently pg_stream_copy is supporting Python 3.7 and 3.8 and PostgreSQL v10, v11, v12Changelog1.2.0Added new data type - numeric1.1.0Added new data types - text, json, jsonb1.0.3Added README to pypi.orgAdded LICENCE, CHANGELOG1.0.2Updated pypi token1.0.1Added Travis CI1.0.0Initial version
Package: pg-streamline
pg-streamline-pyStreamlining PostgreSQL Logical Replication and Event Processingpg-streamline is a Python library designed to simplify and streamline the process of logical replication and event processing with PostgreSQL. It provides modular components for producers, consumers, parsers, and plugins, making it highly extensible and customizable.Table of ContentsInstallationFeaturesUsageProducerConsumerParserPluginsContributingLicenseInstallationpipinstallpg-streamlineFeaturesProducerHandles PostgreSQL logical replication.Supports multiple output plugins like 'pgoutput' and 'wal2json'.Pooling support for better performance.For more details, see theProducer README.ConsumerConsumes and processes the events replicated by the producer.Extensible: Can be extended to perform custom actions when specific database changes occur.For more details, see theConsumer README.ParserParses different types of logical events from PostgreSQL.Supports parsing of INSERT, UPDATE, and DELETE events.Makes it easier to understand and act upon the changes in the database.For more details, see theParser README.PluginsExtend the functionality of pgStreamline with various plugins.Currently supports RabbitMQ for message queuing.More plugins are in development.For more details, see thePlugins README.UsagePlease refer to the README files in each module's directory for specific usage instructions:Producer UsageConsumer UsageParser UsagePlugins UsageContributingWe welcome contributions! Please see theContributing Guidelinesfor more details.LicenseThis project is licensed under the MIT License. See theLICENSEfile for details.
Package: pgsu
pgsuConnect to an existing PostgreSQL cluster as a PostgreSQLSUPERUSERand execute SQL commands.psycopg2has a great API for interacting with PostgreSQL, once you provide it with the connection parameters for a given database.
However, what if your desired database and database user do not yet exist?
In order to create them, you will need to connect to PostgreSQL as a SUPERUSER.Featuresautodetects postgres setup, tested onUbuntu 20.04& PostgreSQL installed viaaptUbuntu 16.04& PostgreSQL installed viaaptUbuntu 18.04& PostgreSQL docker containerMacOS 12and PostgreSQL installed viacondaWindows Server 2019and PostgreSQL installed viacondausespsycopg2to connect if possiblecan usesudoto become thepostgresUNIX user if necessary/possible (default Ubuntu PostgreSQL setups)UsagePython APIfrompgsuimportPGSUpgsu=PGSU()# On Ubuntu, this may prompt for sudo passwordpgsu.execute("CREATE USER newuser WITH PASSWORD 'newpassword'")users=pgsu.execute("SELECT usename FROM pg_user WHERE usename='newuser'")print(users)While the main point of the package is toguesshow to connect as a postgres superuser, you can also provide partial or all information abut the setup using thedsnparameter.
These are the default settings:frompgsuimportPGSUpgsu=PGSU(dsn={'host':None,'port':5432,'user':'postgres','password':None,'database':'template1',# Note: you cannot drop databases you are connected to})Command line toolThe package also comes with a very basicpgsucommand line tool that allows users to execute PostgreSQL commands as the superuser:$ pgsu "SELECT usename FROM pg_user"
Trying to connect to PostgreSQL...
Executing query: SELECT usename FROM pg_user
[('aiida_qs_leopold',),
 ('postgres',)]TestsRun the tests as follows:pipinstall-e.[testing]pytest
Package: pgsync
PostgreSQL to Elasticsearch/OpenSearch syncPGSyncis a middleware for syncing data fromPostgrestoElasticsearch/OpenSearchorOpenSearch.It allows you to keepPostgresas your source of truth data source and
expose structured denormalized documents inElasticsearch/OpenSearch.RequirementsPython3.8+Postgres9.6+Redis3.1.0Elasticsearch6.3.1+ orOpenSearch1.3.7+SQlAlchemy1.3.4+Postgres setupEnablelogical decodingin your
Postgres setting.You also need to set up two parameters in your Postgres config postgresql.confwal_level = logicalmax_replication_slots = 1InstallationYou can install PGSync fromPyPI:$ pip install pgsyncConfigCreate a schema for the application named e.gschema.jsonExample schemaExample spec.. code-block::[
    {
        "database": "[database name]",
        "index": "[Elasticsearch or OpenSearch index]",
        "nodes": {
            "table": "[table A]",
            "schema": "[table A schema]",
            "columns": [
                "column 1 from table A",
                "column 2 from table A",
                ... additional columns
            ],
            "children": [
                {
                    "table": "[table B with relationship to table A]",
                    "schema": "[table B schema]",
                    "columns": [
                      "column 1 from table B",
                      "column 2 from table B",
                      ... additional columns
                    ],
                    "relationship": {
                        "variant": "object",
                        "type": "one_to_many"
                    },
                    ...
                },
                {
                    ... additional children
                }
            ]
        }
    }
]Environment variablesSetup environment variables required for the applicationSCHEMA='/path/to/schema.json'

ELASTICSEARCH_HOST=localhost
ELASTICSEARCH_PORT=9200

PG_HOST=localhost
PG_USER=i-am-root # this must be a postgres superuser or replication user
PG_PORT=5432
PG_PASSWORD=*****

REDIS_HOST=redis
REDIS_PORT=6379
REDIS_DB=0
REDIS_AUTH=*****RunningBootstrap the database (one time only)$ bootstrap --config schema.jsonRun pgsync as a daemon$ pgsync --config schema.json --daemon
Package: pgsync-fork
PostgreSQL to Elasticsearch/OpenSearch syncPGSyncis a middleware for syncing data fromPostgrestoElasticsearch/OpenSearchorOpenSearch.It allows you to keepPostgresas your source of truth data source and
expose structured denormalized documents inElasticsearch/OpenSearch.RequirementsPython3.7+Postgres9.6+Redis3.1.0Elasticsearch6.3.1+ orOpenSearch1.3.7+SQlAlchemy1.3.4+Postgres setupEnablelogical decodingin your
Postgres setting.You also need to set up two parameters in your Postgres config postgresql.confwal_level = logicalmax_replication_slots = 1InstallationYou can install PGSync fromPyPI:$ pip install pgsyncConfigCreate a schema for the application named e.gschema.jsonExample schemaExample spec.. code-block::[
    {
        "database": "[database name]",
        "index": "[Elasticsearch or OpenSearch index]",
        "nodes": {
            "table": "[table A]",
            "schema": "[table A schema]",
            "columns": [
                "column 1 from table A",
                "column 2 from table A",
                ... additional columns
            ],
            "children": [
                {
                    "table": "[table B with relationship to table A]",
                    "schema": "[table B schema]",
                    "columns": [
                      "column 1 from table B",
                      "column 2 from table B",
                      ... additional columns
                    ],
                    "relationship": {
                        "variant": "object",
                        "type": "one_to_many"
                    },
                    ...
                },
                {
                    ... additional children
                }
            ]
        }
    }
]Environment variablesSetup environment variables required for the applicationSCHEMA='/path/to/schema.json'

ELASTICSEARCH_HOST=localhost
ELASTICSEARCH_PORT=9200

PG_HOST=localhost
PG_USER=i-am-root # this must be a postgres superuser or replication user
PG_PORT=5432
PG_PASSWORD=*****

REDIS_HOST=redis
REDIS_PORT=6379
REDIS_DB=0
REDIS_AUTH=*****RunningBootstrap the database (one time only)$ bootstrap --config schema.jsonRun pgsync as a daemon$ pgsync --config schema.json --daemon
Package: pgtail
### pgtailTail your postgres table inserts like a file.### Installation$ pip install pgtail### UsageTo tail a table, run$ pgtail postgres://user:pass@localhost:5432/database tableBy default, it will tail from the last row of the table.To tail last 10 rows, every 5 seconds, run$ pgtail postgres://user:pass@localhost:5432/database table 10 5To see help, run$ pgtail –help
Package: pg-temp
pg_tempProvides a simple class for creating a temporary userland Postgres db server.TheTempDB()class constructor will set up the database server in a temporary
directory.  If any databases are specified they are created inside the newly
created server.  The class provides acleanup()method for stopping the
server and disposing of the temporary files.The module also provides aninit_temp_db()function for managing theTempDBclass as a singleton.  This is useful for scenarios where you want to import in
more than one module, but ensure that they all use the same database (and that
the server and database are only created once).  For example, this is handy in
unit test code where you want to be able to test a single file or use a test
runner to run tests in multiple files with only a single database.init_temp_db()also registers anatexithandler to cleanup the temp
database when the process exits.Examples:Create a temporary server with a database called 'testdb':frompg_tempimportTempDBtemp_db=TempDB(databases=['testdb'])# you can connect to this database using temp_db's pg_socket_dirconnection=psycopg2.connect(host=temp_db.pg_socket_dir,database='testdb')# ... do stuff...temp_db.cleanup()Alternatively, useful in a test runner:importpg_temptemp_db=pg_temp.init_temp_db(databases=['testdb'])# repeat above in multiple modules# you can connect to this database using temp_db's pg_socket_dirconnection=psycopg2.connect(host=temp_db.pg_socket_dir,database='testdb')# ... do stuff...# the db is automatically cleaned up when the process exitsLast an interactive example:>>>importpg_temp>>>importpsycopg2>>>temp_db=pg_temp.TempDB(databases=['testdb'])CreatingtempPGserver...done(Connecton:`psql-h/var/folders/d7/n3_h9vnn3w3bbmsnbdb73fmw0000gn/T/pg_tmp_OQMGwC/socket`)>>>connection=psycopg2.connect(host=temp_db.pg_socket_dir,database='testdb')>>>cur=connection.cursor()# The rest is stolen from psycopg's documentation.  You get the idea...>>>cur.execute("CREATE TABLE test (id serial PRIMARY KEY, num integer, data varchar);")>>>cur.execute("INSERT INTO test (num, data) VALUES (%s,%s)",...(100,"abc'def"))>>>cur.execute("SELECT * FROM test;")>>>cur.fetchone()(1,100,"abc'def")>>>connection.close()>>>temp_db.cleanup()DevelopmentInstall dependencies for testing:# Create a virtualenv
virtualenv venv

# Activate the virtualenv
. venv/bin/activate

# Install dependencies for testing
pip install -r requirements-dev.txtRun the unit tests directly:make testTo test against multiple Python versions, without needing to use
Virtualenv directly, run the unit tests using tox:make toxCheck code style using flake8 and black:make checkCode is auto-formatted usingblack. Since this was done relatively
recently, you should configure git to ignore the reformatting commit
using the --ignore-rev or the --ignore-revs-file option to git blame,
or configure this by:git config blame.ignoreRevsFile .git-blame-ignore-revsThe ignore revisions file is called.git-blame-ignore-revs