Saravanan666/pypi-org huggingface dataset:
Package: pgo
![Version 0.2.2](https://img.shields.io/badge/version-0.2.2-blue?style=flat-square)
![Python 3.6+](https://img.shields.io/badge/python-3.6%2B-blue?style=flat-square)
![License: MIT](https://img.shields.io/badge/license-MIT-blue?style=flat-square)[![tests status](https://img.shields.io/github/workflow/status/esoma/pgo/test?label=tests&style=flat-square)](https://github.com/esoma/pgo/actions/workflows/test.yml)
[![coverage](https://img.shields.io/codecov/c/github/esoma/pgo?style=flat-square)](https://app.codecov.io/gh/esoma/pgo)
[![quality](https://img.shields.io/lgtm/grade/python/github/esoma/pgo?label=quality&style=flat-square)](https://lgtm.com/projects/g/esoma/pgo/alerts/?mode=tree)pgobrings easy and unobtrusive profile guided optimization to Python
extensions.Work in progress.Copyright 2021 Erik SomaPermission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the “Software”), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Package: pgobserver-gatherer
==============pgobserver_gatherer==============A postgres metrics collection daemon, successor of the original [PGObserver](https://github.com/zalando/PGObserver) Java gathering daemonNB! BETAInstallation------------------ Bootstrap the database (datastore) that will keep the configuration and metrics data for Postgres databases you want to monitor- pip3 install --upgrade pgobserver_gatherer- python3 -m pgobserver_gatherer --init  # to create a config file with connect details to your datastore- python3 -m pgobserver_gatherer -v  # to run the daemon in verbose modeImprovements compared to old gatherer------------------ automatic detection of gatherer configuration changes (hosts.host_settings column)- connection pooling to datastore- generalized data storing, less boilerplate for gatherers- single transaction bulk inserts (copy protocol)- connectivity check to all DBs under monitoring on startup- single host/single gatherer test mode- "delta engine" diffing engine for getting real changes- custom output plugins. separate processes that get incoming datapoints via queuesOperating principles-----------------1. gather data2. store to postgres as currently - optional3. feed the analytics framework (delta engine) that calculates the diffs for cumulative columns4. feed the data coming from delta engine to plugins (simplest one being the "console" outputting plugin)- config files describing handlers and their formats and wanted datasets. they can have extra config files for output- loading of modules and feeding them when data comes in
Package: pgOperations
Welcome to the pgOperations Python moduleIt is a simple lightweight Python module to perform the most
common operations inPostgreSQLandPostgGIS, avoiding most of the SQL syntax.This module class methods allow insert, delete, update, select, create
and delete tables and databases. Also, you will be able to know is a table exists, get the field names
of a table, and manage counters for your applications.The  class methods use Python dictionaries to insert and update data.This library supposes all python dictionary keys match with the table field names.This module requires Python 3 andPsycopg2.Read thedocumentationto know how to usepgOperations. In the tutorial you will find
how to install, and use the class methods. The reference contains the classes and methods
description.
Package: pgos-cli-python-sdk-v1
PGOS CLI Python SDK V1IntroPGOS CLI Python SDK.Requires Python3.8 or higher versions.InstallingInstall with pip:python -m pip install --upgrade pgos-cli-python-sdk-v1UsageExamples for JSON paramsRefer to./tests/json_params_demo/.json*Examples for SDK usageRefer to./tests/test.py
Package: pgpagecache
pgpagecache is a Python script, showing PostgreSQL objects resident in memory.
PostgreSQL relies on OS file system cache to keep performance high and it is
pretty common not to see gains in performance for buffer cache size over about
a specific limit.
Since the OS buffer cache keeps file buffers in memory as well, there are
lots of database objects in PostgreSQL cache, OS cache, or both. This is called
double buffering.If someone wants to examine the PostgreSQL cache, he/she can use the contrib
module pg_buffercache to map each object with it’s memory footprint in cache.
But, if one wanted to examine the entire cache picture, including OS cache,
he/she has to determine whether PostgreSQL object pages are resident in memory,
for this purpose one can use pgpagecache.Required ExtensionFor the PostgreSQL buffercache usage the pg_buffercache extension is needed.
You can install it from the contrib modules by simply executing the command
below:postgres=# CREATE EXTENSION pg_buffercache;
CREATE EXTENSIONExample$ pgpagecache -u postgres -p apassword -d postgres -H localhost -P 5432 \
  -b /var/lib/postgresql/9.3/main/base
  PageCache Usage
  ---------------

  DB Name    Table                    Cached Pages    Total Pages    Ratio (%)    Bytes
  ---------  ---------------------  --------------  -------------  -----------  -------
  postgres   pgbench_accounts_pkey             552            552          100  2260992
  postgres   pgbench_tellers                    18             18          100    73728
  postgres   pgbench_tellers_pkey                4              4          100    16384
  postgres   pgbench_branches                   12             12          100    49152
  postgres   pgbench_branches_pkey               4              4          100    16384

  PostgreSQL BufferCache Usage
  -----------------------------

  DB Name    Table                                         Bytes
  ---------  ---------------------------------  ----------------
  postgres   pgbench_accounts                        1.38854e+07
  postgres   pgbench_accounts_pkey                   2.2528e+06
  postgres   pgbench_history                    188416
  postgres   pg_statistic                       122880
  postgres   pg_operator                        106496
  postgres   pgbench_tellers                    106496
  postgres   pg_depend_reference_index           98304
  postgres   pg_depend                           81920
  postgres   pgbench_branches                    81920
  ...
Package: pgparse
Python wrapper forlibpg_queryInstallationpipinstallpgparseExample UsageThe following example shows how to create a dump and then read it in, and
iterate through the data of one of the tables.importpprintimportpgparsesql="SELECT * FROM pg_catalog.pg_class WHERE relname = 'foo'"print('Fingerprint:{}'.format(pgparse.fingerprint(sql)))print('Normalized:{!r}'.format(pgparse.normalize(sql)))parsed=pgparse.parse(sql)pprint.pprint(parsed)func="""\CREATE FUNCTION sales_tax(subtotal real) RETURNS real AS $$
        BEGIN
            RETURN subtotal * 0.06;
        END;
        $$ LANGUAGE plpgsql;
"""parsed=pgparse.parse_pgsql(func)pprint.pprint(parsed)
Package: pgpasslib
pgpasslib is a library for retrieving passwords from a PostgreSQL password
file, either from a location specified in thePGPASSFILEenvironment
variable or in the.pgpassfile in the current user’s home directory.Installationpgpasslib may be installed via the Python package index with the tool of
your choice. I prefer pip:pipinstallpgpasslibDocumentationhttps://pgpasslib.readthedocs.orgRequirementsThere are no requirements outside of the Python standard library.ExampleThe following example will attempt to get the password for PostgreSQL running
onlocalhost:5432to thepostgresdatabase as thepostgresuser.importpgpasslibpassword=pgpasslib.getpass('localhost',5432,'postgres','postgres')ifnotpassword:raiseValueError('Did not find a password in the .pgpass file')Version HistoryAvailable athttps://pgpasslib.readthedocs.org
Package: pgpc
PGPCOverviewPGPCis a parser combinator library. Its name is an acronym, which stands forPythonGenerator basedParserCombinator library.The library was heavily influenced by theParsecmonadic parser combinator library,
so the transition fromParsec(and other parser combinators) toPGPCshould be relatively easy.The original idea of the library is emulating thedo-notation with theyieldPython keyword.Quick startThe@topologydecorator over a function allows theyieldkeyword work with parsers in ado-notation-like fashion:install the package:pip install pgpcsave the following code intomain.py:frompgpc.scannerimportTextScannerfrompgpc.parserimportParser,topology,char,position,content@topologydefparse_hello_world():start=yieldposition()forcin"Hello":yieldchar(c)yieldchar(",")yieldchar(" ")forcin"World":yieldchar(c)yieldchar("!")end=yieldposition()source=yieldcontent()returnf"Parsed '{source[start.offset:end.offset]}' which started at{start}and ended at{end}"if__name__=='__main__':text="Hello, World!"hw_parser:Parser[str]=parse_hello_world()result=hw_parser(TextScanner(text))print(result)runmain.py:python main.py
Package: pgpdump
# python-pgpdump: a Python library for parsing PGP packetsThis is based on the C version published at:http://www.mew.org/~kazu/proj/pgpdump/The intent here is not on completeness, as we don’t currently decode every
packet type, but on being able to do what people actually have to 95% of the
time. Currently supported things include:Signature packetsPublic key packetsSecret key packetsTrust, user ID, and user attribute packetsASCII-armor decoding and CRC checkA single codebase with dependencies on only the standard python library is
compatible across Python 2.6, 2.7, and 3.2+, as well as with PyPy 1.8+.
Package: pgpdump3
This is based on the C version published at:http://www.mew.org/~kazu/proj/pgpdump/This version is a fork of the version by Dan McGee, dropping Python 2
compatibility.The intent here is not on completeness, as we don’t currently decode
every packet type, but on being able to do what people actually have to
95% of the time. Currently supported things include:Signature packetsPublic key packetsSecret key packetsTrust, user ID, and user attribute packetsASCII-armor decoding and CRC checkCompressed packet decompression automaticallyThis package is designed for Python 3 only.
Package: pgpipe
PgPipePgPipe is a Python package designed to efficiently synchronize data between two PostgreSQL databases.
It is specifically designed to handle large amounts of data by dividing the transfer process into smaller,
manageable chunks.FeaturesEfficient data transfer using the COPY commandDivide data into smaller chunks for improved performanceTrack transfer progress and allow restarting from checkpointsHandle foreign key dependencies by ensuring the correct order of table transfersInstallationTo install PgPipe, simply use pip:pipinstallpgpipeUsageThe PgPipe tool provides a command line interface (CLI) for synchronizing data between two PostgreSQL databases. The
following sections will guide you on how to use this tool effectively.Setting up the CLIPgPipe requires you to specify several settings to connect to your PostgreSQL databases and define the synchronization parameters. These settings can be provided through command line arguments or environment variables. Here is the list of command line arguments you can use:--log-level: Set the log level. Default is 'INFO'.--schemas: Specify the list of schemas. Default is 'public'.--chunk-size-offset: Set the chunk size for offset-based chunking. Default is 50000.--chunk-size-timedelta: Set the chunk size for time-range-based chunking in days. Default is 1.--db-schema-path: Specify the path to the DB schema file. Default is './db_schema.json'.--chunk-data-path: Specify the path to the chunk data file. Default is './chunk_data.json'.--source-dsn: Specify the source database DSN. If not set, it is read from the environment variable 'PGPIPE_SOURCE_DSN'.--destination-dsn: Specify the destination database DSN. If not set, it is read from the environment variable 'PGPIPE_DESTINATION_DSN'.Running the Synchronization StepsThe synchronization process is divided into three steps, each corresponding to a command:Create Database Schema: To execute this step, run thecreate_db_schemacommand. This will read the source database metadata and store it in the DB schema file specified by--db-schema-path.pgpipe create_db_schema --source-dsn="your_source_dsn" --db-schema-path="your_db_schema_path"Prepare Chunks: Run theprepare_chunkscommand to prepare chunk metadata for data transfer. The metadata will be saved to the chunk data file specified by--chunk-data-path.pgpipe prepare_chunks --source-dsn="your_source_dsn" --db-schema-path="your_db_schema_path" --chunk-data-path="your_chunk_data_path" --chunk-size-offset=50000 --chunk-size-timedelta=1Transfer Data: Thetransfer_datacommand starts the data transfer process, using the prepared metadata to guide the process.pgpipe transfer_data --source-dsn="your_source_dsn" --destination-dsn="your_destination_dsn" --chunk-data-path="your_chunk_data_path"If no command is specified, the CLI will print a help message with a list of available commands and settings.LicensePgPipe is released under the MIT License. See theLICENSEfile for more details.
Package: pgpipeline
All rights reserved.Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.Description: # Pgpipeline: automatic postgres pipeline for ScrapyA Scrapy pipeline module to persist items to a postgres table automatically.## Quick StartHere's an example showing automatic item pipeline, with a custom `JSONB` field.```python# settings.pyfrom sqlalchemy.dialects.postgresql import JSONBITEM_PIPELINES = {'pgpipeline.PgPipeline': 300,}PG_PIPELINE = {'connection': 'postgresql://localhost:5432/scrapy_db','table_name': 'demo_items','pkey': 'item_id','ignore_identical': ['item_id', 'job_id'],'types': {'some_data': JSONB},'onconflict': 'upsert'}```All columns, tables, and indices are automatically created.* `pkey`: a primary key for this item (other than database-generated `id`)* `ignore_identical`: these are a set of fields by which we identify duplicates and skip insert.* `types`: keys specified here will be using the type given, otherwise types are guessed.* `onconflict`: upsert|ignore|non-null - `ignore` will skip inserting on conflict and `upsert` will update. `non-null` will upsert only values that are not `None` and thus avoid removing existing values.## DevelopersSet up a development environment```$ pip install -r requirements.txt```### Development* Dependencies: list them in `requirements.txt`### Release* Dependencies: list them in `setup.py` under `install_requires`:```pythoninstall_requires=['peppercorn'],```Then:```$ make dist && make release```# ContributingFork, implement, add tests, pull request, get my everlasting thanks and a respectable place here :).### Thanks:To all [Contributors](https://github.com/jondot/pgpipeline/graphs/contributors) - you make this happen, thanks!# CopyrightCopyright (c) 2017 [Dotan Nahum](http://gplus.to/dotan) [@jondot](http://twitter.com/jondot). See [LICENSE](LICENSE.txt) for further details.Platform: UNKNOWN
Package: pgpm
pgpmis a package manager for Postgres database.
Its main features include deployment of postgres objects (schemas, tables, functions, etc.), tracking of DDL changes and execution of arbitrary scripts across multiple data sources.InstallationPrerequisites: Postgres 9.4+, python 2.7/3.4+.Installpsycopg2. This cannot be easily resolved withpipas the library relies on Postgres binaries. The step is not always trivial, though Google will certainly enlighten the path.Run$pipinstallpgpmInstall pgpm in every postgres database you’ll be using with pgpm (must be a superuser). E.g.:$pgpminstall"postgresql://postgres:postgres@localhost:5432/testdb"Note: connection string must conformlibpg format.Main conceptsPackageInpgpma package is a set of files in a directory with validconfig.json(see below) file. When package is being deployed,pgpmreads the configuration and executes sql scripts from files specified in the package configuration.
Note:config.jsonfile must be present in a root directory of the package in order to use deployment feature of pgpmConfigpgpmusesconfig.jsonfile for a configuration of a package. Configuration file must be a single valid JSON object with the following properties:Property nameRequired/OptionalValueDescriptionnamerequiredstringName of the package (for details see section ‘Name of the package’)descriptionoptionalstringDescription of the packagesubclassrequiredstring, accepts values:basicandversionedType of the package when deployed as a DB schema.
For details see section ‘Versioned or not versioned schema’scoperequiredstring, accepts values:SCHEMAandDATABASE,
defaults toSCHEMAWhether the the deployment should be bound to a specific schema or be for the whole DB.
For details see section ‘Scope of package’.versionrequiredstring, accepts values with the following format:
[0..9][0..9]_[0..9][0..9]_[0..9][0..9]Version of the package.
For details see section ‘Versioned or not versioned schema’.functions_pathoptionalstringA relative path (relative to the root directory of the package) to the folder with source code for stored procedures
For details see section ‘Object types’.tables_pathoptionalstringA relative path (relative to the root directory of the package) to the folder with sql scripts that change table definitions
For details see section ‘Object types’.types_pathoptionalstringA relative path (relative to the root directory of the package) to the folder with source code for types definitions
For details see section ‘Object types’.triggers_pathoptionalstringA relative path (relative to the root directory of the package) to the folder with source code for triggers (not trigger functions, they go tofunctions_path)
For details see section ‘Object types’.Name of the packageTODOTest infoVersioned or not versioned schemaTODOScope of packageTODOObject typesTODOGeneral configTODOMain featuresPackage deploymentTODODDL change loggingTODOpgpm schemaTODOTODOsProvide support for DDL evolutions and dependency management.
Package: pgpmsg
This is a fairly simply module which will decode OpenPGP messages (RFC 2440).
Such messages are generated by Gnu Privacy Guard and PGP. I haven’t made any
effort to do much cryptographic operations because in my case my purpose for
creating the module was to write an OpenLDAP backend that supported the kind of
LDAP access present in PGP. The one crypto thing it does do is calculate the
fingerprint/key_id for a public key. So, basically this module will let you read
in a message (which could be encrypted text or a pgp key) and will let you
iterate over the object and inspect its parts.
Package: pg_populator
Tool: pg_populatorDescription:This program populates an already existing PostgreSQL service with:
- N databases. The specific amount of databases is parametrized through the -n argument.
- N tables inside each database. The specific amount of tables is parametrized through the -t argument.
- Some data rows inside every table. The specific amount of rows is parametrized through the -r argument.This program will make sure that local postgreSQL service contains, at least, the amount of
databases and tables defined in the required command line arguments -n (–num_db) and -t (–num_tables) per database.
It will also add the amount of rows specified in the -r parameter.
This program will NOT delete any database, table or row.Requirements:A running postgreSQL database must be previously configured.
To make this software works, the user that runs this script must have the ~/.pgpass properly configured.
To get this done, I recommend you to use the scripts stored in the next repository:https://gitlab.com/jaqm/carto_test.gitHow to’s:Note: The next commands are meant to be run in the same that directory in which this Readme.1st file is stored.Disable any previously activated virtualenv:$ deactivateEnable virtualenv:$ source venvs/1_populate_db/bin/activateGet the program help:$ ./run -vRun the tool (example):Run the next command to create 10 databases with 20 tables and insert 100 rows each of them.$ ./run -n 10 -t 20 -r 100
Package: pgpq
pgpqConvert PyArrow RecordBatches to Postgres' native binary format.Usage"""Example for README.md"""fromtempfileimportmkdtempimportpsycopgimportpyarrow.datasetasdsimportrequestsfrompgpqimportArrowToPostgresBinaryEncoder# let's get some example datatmpdir=mkdtemp()withopen(f"{tmpdir}/yellow_tripdata_2023-01.parquet",mode="wb")asf:resp=requests.get("https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2023-01.parquet")resp.raise_for_status()f.write(resp.content)# load an arrow dataset# arrow can load datasets from partitioned parquet files locally or in S3/GCS# it handles buffering, matching globs, etc.dataset=ds.dataset(tmpdir)# create an encoder object which will do the encoding# and give us the expected Postgres table schemaencoder=ArrowToPostgresBinaryEncoder(dataset.schema)# get the expected Postgres destination schema# note that this is _not_ the same as the incoming arrow schema# and not necessarily the schema of your permanent table# instead it's the schema of the data that will be sent over the wire# which for example does not have timezones on any timestampspg_schema=encoder.schema()# assemble ddl for a temporary table# it's often a good idea to bulk load into a temp table to:# (1) Avoid indexes# (2) Stay in-memory as long as possible# (3) Be more flexible with types#     (you can't load a SMALLINT into a BIGINT column without casting)cols=[f'"{col_name}"{col.data_type.ddl()}'forcol_name,colinpg_schema.columns]ddl=f"CREATE TEMP TABLE data ({','.join(cols)})"withpsycopg.connect("postgres://postgres:postgres@localhost:5432/postgres")asconn:withconn.cursor()ascursor:cursor.execute(ddl)# type: ignorewithcursor.copy("COPY data FROM STDIN WITH (FORMAT BINARY)")ascopy:copy.write(encoder.write_header())forbatchindataset.to_batches():copy.write(encoder.write_batch(batch))copy.write(encoder.finish())# load into your actual table, possibly doing type casts# cursor.execute("INSERT INTO \"table\" SELECT * FROM data")