Saravanan666/pypi-org huggingface dataset:
Package: pg-seldump
This tool allows to extract data from aPostgreSQLdatabase with greater
flexibility that tools likepg_dumpallow.pg_seldumpreads one of more dump definitions from YAML files and selects
what tables or other database objects to save. It is possible to extract only
certain columns of the tables, only certain records, or to replace certain
values with a different expression, for instance to anonymize data.The output of the program is a text file which can be used bypsqlto
restore data into a database with a complete schema but with no data (or at
least no conflicting data), e.g. using:$ pg_seldump --dsn="dbname=sourcedb" datadump.yaml > dump.sql
...
$ psql -1X --set ON_ERROR_STOP=1 -f dump.sql "dbname=targetdb"Program usageUsage:pg_seldump [-h] [--version] [--dsn DSN] [--outfile OUTFILE] [--test]
           [-q | -v]
           config [config ...]

Create a selective dump of a PostgreSQL database.

positional arguments:
  config                yaml file describing the data to dump

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --dsn DSN             database connection string [default: '']
  --outfile OUTFILE, -o OUTFILE
                        the file where to save the dump [default: stdout]
  --test                test the configuration to verify it works as expected
  -q, --quiet           talk less
  -v, --verbose         talk moreTheconfigfiles must beYAMLfiles containing adb_objectslist of
entries. Each entry may have:Selectors (all the specified ones must match):name: name of the db object to dumpnames: list of names or regex of db objects to dumpschema: schema name of the db object to dumpschemas: list of schema names or regexp to match schema names of the
db object to dumpkind: kind of object to match. Can be:tablesequenceparitioned tablematerialized viewkinds: list of kind of objects to match (like forkind)adjust_score: adjustment for the match score to break rules tiesNoteSequences are selected automatically if they are used in default values by
dumped tables.Data modifiers:action: what to do with the matched object:dump: dump the object in the output (default)skip: don’t dump the objecterror: raise an error in case of match (useful to create strict
descriptions where all the db objects must be mentioned explicitly)no_columns: list of columns names to omitfilter: WHERE condition to include only a subset of the records in the dumpreplace: mapping from column names to SQL expressions to replace values
into the dump with somethings elseThe objects in the database are matched to the rules in the config files.
Every match will have a score according to how specific was the selector
matched the object.nameornameslist: 1000namesregexp: 500schemaorschemaslist: 100schemasregexp: 50kindorkinds: 10The rule with the highest score will apply. If two rules have exactly the same
score the program will report an error: you can useadjust_scoreto break
the tie.
Package: pg-sequence-increaser
pg-sequence-increaserSet all sequences to a higher value when promoting a PostgreSQL logical replica.RequirementsPython3libpq-dev(for psycopg2). On Mac OS,brew install postgresql, on Ubuntu, installlibpq-dev.InstallationProduction$pipinstallpg-sequence-increaserDevelopmentUsing virtualenv,$pipinstall-rrequirements.txtUsageThis tool implements two strategies:Getting the sequences and their desired values from the logical replica alone (called "max-id-on-replica"),getting the sequences and their desired values from the primary database, i.e. the publisher; this is called "primary-sequences".Max Id on ReplicaWe get the sequences frominformation_schema.default_value. Anydefault_valuethat hasnextval('some_sequence_name'), we assume that's a sequence and extract the table name, the associated column and the sequence name. Then, we runMAX(column_name)to get the last value inserted into that table, likely from the sequence; we then move the sequence forward to theMAX(column_name) + increment_by. This strategy has the advantage of using the replica as the source of truth and not requiring the publisher to be online.$pgseqmover--replica-url=postgres://localhost:5432/my_db--queries-onlywill print the queries you can then run manually. Those queries will increase all sequences by 1000.Primary SequencesWe get the sequences from the primary (publisher). After all, that's where we are replicating from in the first place. We usepg_sequencesview and get both the name and thelast_valuefrom there as well. We then set the sequences on the replica to what they are on the primary +increment_by. This strategy has the advantage of using the primary database as the source of truth in case replica lag is non-zero during promotion which is bad, but possible. This requires the primary to be available during promotion of the replica.$pgseqmover--replica-url=postgres://replica-db:5432/my_db--primary-url=postgres://primary-db:5432/my_db--queries-only--strategy=primary-sequenceswill print the queries you can then run manually. Those queries will increase all sequences by 1000.ArgumentsThis also accepts optional arguments:--dry-runwhich will show the queries it will run but not actually do anything; this is the default behavior,--debugwill showallqueries it's running; can be combined with--dry-run,--increment-byoverrides the default increase by value of 1000 to any value (even negative ones).--strategypicks the strategy to use for increasing sequences. Your options are:primary-sequencesandmax-id-on-replica, both explained above.--queries-onlywill only print the queries that will move the sequences and not do anything else. Less pretty version of--dry-run, but good forpgseqmover --queries-only > ~/Desktop/queries_to_run.sqluse case.
Package: pg-serializer
pg_serializer: Very fast JSON serializer using Django and PostgreSQLOverviewDjango REST framework is a powerful and flexible toolkit for building Web APIs.Django Rest Framework is a great framework for building REST APIs on top of Django and
it fits 99% of use cases.However, JSON serialization is done in Python and it can be very slow for
somelistendpoints that return a lot of records.pg_serializeraims to soften this 1% issue by providing a way to do JSON serialization
inside your PostgreSQL database. Indeed, PostgreSQL has a built-in support for JSON
since version 9.2, and it is far more faster than Python's one.Installationpipinstallpg-serializerBasic usagepg_serializerfeatures an automaticModelSerializerlike DRF.fromdjango.contrib.auth.modelsimportUserimportpg_serializerclassUserSerializer(pg_serializer.ModelSerializer):classMeta:model=Userfields=("id","username","email","first_name","last_name","is_staff","date_joined",)json_str=UserSerializer(User.objects.all()).jsonIt also has preliminary support for ForeignKey relation using the Django__separator.importpg_serializerclassOrderSerializer(pg_serializer.ModelSerializer):buyer=pg_serializer.StringField(source="buyer__username")classMeta:model=Orderfields=("transaction_id","transaction_time","buyer","is_gift","shipping_date","additional_data",)json_str=OrderSerializer(Order.objects.all()).jsonFull examples are available insidetests/models.pyandtests/serializers.py.Disclaimer ⚠️pg_serializeris not designed to replace DRF serializers, only to speed up
some endpoints when performance is becoming an issuepg_serializeris still in alpha development: bugs can occur and
it doesn't support all Django fields and relationsRoadmapreinforce the test suiteimplement (and document) custom datetime and decimal formattingdocument how to create custom serializer fieldsfull support for ArrayField and JSONFieldcomplex queryset operations (GROUP BY, ...) support
Package: pgserver
pgserver: pip installable self-contained postgres serverpip install pgserverpgserverlets you create and manage a Postgres within your Python app, with server binaries included.
Wheels are built for multiple platforms.Example use cases:Building a Postgres-backed yet still self-contained Python app, much like you would withsqlite.Developing and testing apps that depend on some external Postgres (as a dev dependency)Basic summary:Pip installable binaries: tested on Ubuntu and MacOS (apple silicon + x86), including pgvector extension.No sudo needed: Does not requirerootorsudo.Convenient startup:pgserver.get_server(MY_DATA_DIR)factory method to initialize data and server if needed, so you don't need to understandinitdb,pg_ctl, port conflicts, and skip debugging why you still cannot connect to the server.Convenient cleanup: server process cleanup is done for you, including when multiple independent processes callpgserver.get_server(MY_DATA_DIR)Context manager protocol to explicitly control cleanup timing in testing scenarios.For lower-level control, wrappers to all binaries, such asinitdb,pg_ctl,psql,pg_config. Includes header files in case you wish to build some other extension and use it against these binaries.# Example 1: postgres backed applicationimportpgserverpgdata=f'{MY_APP_DIR}/pgdata'db=pgserver.get_server(pgdata)# server ready for connection.print(db.psql('create extension vector'))db_uri=db.get_uri()# use uri with sqlalchemy / psycopg, etc# if no other process is using this server, it will be shutdown at exit,# if other process use same pgadata, server process will be shutdown when all stop.# Example 2: Testingimporttempfileimportpytest@pytest.fixturedeftmp_postgres():tmp_pg_data=tempfile.mkdtemp()withpgserver.get_server(tmp_pg_data,cleanup_mode='delete')aspg:yieldpgPostgres binaries in the package can be found in the directory pointed
to by thepgserver.pg_binglobal variable.Based onhttps://github.com/michelp/postgresql-wheel, with the following differences:+Wheels for multiple platforms (+MacOS x86, +MacOS apple silicon)+Server management (initialization and cleanup including multi-process )+pgvector extension included-postGIS (many dependencies => harder to build cross platform. pull requests taken)
Package: pgserviceparser
pgserviceparserA python package parsing the PostgreSQL connection service file.>>> import pgserviceparserFinding the PostgreSQL connection service file withconf_pathReturns the path found for the pg_service.conf on the system as string.>>> pgserviceparser.conf_path()
'/home/dave/.pg_service.conf'Listing all the services withservice_namesReturns all service names in a list.
Optionally you can pass a config file path. Otherwise it gets it byconf_path.>>> pgserviceparser.service_names()
['srvce_wandplaene', 'ktn_solothurn', 'daves_bakery']Receiving the configuration for a service withservice_configReturns the config from the given service name as a dict.
Optionally you can pass a config file path. Otherwise it gets it byconf_path.>>> pgserviceparser.service_config('daves_bakery')
{'host': 'localhost', 'port': '5432', 'dbname': 'bakery', 'user': 'dave', 'password': 'fischersfritz'}Getting the full configuration withfull_configReturns full pgservice config asconfigparser.ConfigParser().
Optionally you can pass a config file path. Otherwise it gets it byconf_path.>>> pgserviceparser.full_config()
<configparser.ConfigParser object at 0x7f4c6d66b580>
Package: pgsheets
pgsheets is a Python3 library for interacting with Google Sheets.
It makes use ofPandasDataFrames,
2-dimensional structures perfectly
suited for data analysis and representing a spreadsheet.This library can be integrated easily with your existing data to present dashboards, update documents, or provide quick data analysis.FeaturesGet / Set all or part of a Google SheetManage authorization with Google APIRetrieve/set formulas or valuesResize spreadsheetsOpen up a wealth of Pandas data tools to use on Google SheetsInstallationSimply install with pip:$pipinstallpgsheetsUsageSetting up a Google ProjectIf you haven’t already you will need to create a project in Google’s Developer Console and get your Client ID and Client Secret.Navigate to theGoogle Developer ConsoleCreate a project (you will be redirected to the project page)Click onAPIs & AuthClick onConsent screenand set a Product NameNow click oncredentials.ClickCreate new Client IDand selectInstalled Application>OtherGetting a TokenUsing your Googleclient idandclient secretwe can get a
authorization URL to present to a user:>>>frompgsheetsimportToken,Client>>>c=Client(my_client_id,my_client_secret)>>>c.getOauthUrl()'https://accounts.google.com/o/oauth2/auth?...'By visiting this URL a Google user can consent to your application
viewing and modifying their Google sheets. After consenting to this
an access code is returned, which we use to get a token:>>>my_token=c.getRefreshToken(access_code)>>>type(my_token)str>>>t=Token(c,my_token)You need to savemy_tokenfor future use.Editing a spreadsheetCreate a spreadsheet (and make sure you save it) and copy the url.
Now we can access the Spreadsheet:>>>importpandasaspd>>>frompgsheetsimportSpreadsheet>>>s=Spreadsheet(t,my_url)>>>s<Spreadsheettitle='test'key='.....'>>>>s.getTitle()'test'>>>s.getWorksheets()[<Worksheettitle='Sheet1'sheet_key='.....'>]>>>w=s.getWorksheet('Sheet1')>>>w.getTitle()'Sheet1'>>>w.setDataFrame(pd.DataFrame([['money','interest','years','result'],['1000','0.015','3','=A2 * (1+B2) ^ C2']]),copy_columns=False,copy_index=False,resize=True)>>>w.asDataFrame()interestyearsresultmoney10000.0153=R[0]C[-3]*(1+R[0]C[-2])^R[0]C[-1]>>>w.asDataFrame(values=True)interestyearsresultmoney10000.01531045.678375>>>df=w.asDataFrame()>>>df['checked']="TRUE">>>w.setDataFrame(df)>>>w.asDataFrame()interestyearsresultcheckedmoney10000.0153=R[0]C[-3]*(1+R[0]C[-2])^R[0]C[-1]TRUELimitationsThe library has only been tested in Python3.4.
It will almost certainly not work in Python2.Currently the following cannot be done with pgsheets:Create a spreadsheetAdd a workbook to a spreadsheetRename a spreadsheet or a workbookPrevent certain values from changing slightly e.g. ‘True’ becomes ‘TRUE’Finally the Google API has some limitations.
Ideally this code should not cause any changes to a worksheet:>>>w.setDataFrame(w.asDataFrame())Unfortunately, there are certain edge cases.
For example, with a Formula such as the following={1, 2}which displays across two cells:12There is no clear way to know
that the cell on the right wasn’t input as a ‘2’ by the user.
Thus the above code would cause the following output:={1, 2}2which displays as:#REF!2
Package: pgsm
State ManagerA simple pygame state manager with threaded loading using loading screens.This means you can use heavy initialization and entry code without having to worry about it. As long as your code needs to finish up, a loading state will be displayed.Set UpYou can choose to install eitherpygameor the community versionpygame-cepipinstallpygame# orpipinstallpygame-ceThe package is called pgsm (PyGameStateMachine)pipinstallpgsmUsageYou first write the states by inheriting fromStatethen you put it all together with theStateManager.Take this as a simple example with a play state and a pause state.We want to toggle between play and pause state when the user presses "p"importpygameaspgfromStateManagerimportState,StateManagerclassPlayState(State):defon_enter(self):print("Playing")defupdate(events,dt):# This is an example for handling events# and changing statesforeventinevents:ifevent.type==pg.KeyDownandevent.key==pg.K_p:self.exit("pause")defdraw(self,s):"""Whatever you want to draws is the surface or screen you may draw to"""classPauseState(State):defon_enter(self,_frm):print("Pausing")defupdate(events,dt):foreventinevents:ifevent.type==pg.KEYDOWNandevent.key==pg.K_p:self.exit("play")defdraw(self,s):""" Whatever you want to draw """# We use strings as keys to refer to our statessm=StateManager({"play":PlayState(),"pause":PauseState()},start="play")Now you can use your state machine like this.
The state machine will by default just draw to the main screen.SCREEN=pg.display.set_mode((900,600))whilerunning:dt=clock.tick(60)events=pg.event.get()foreventinevents:ifevent.type==pg.QUIT:running=Falseifevent.type==pg.KEYDOWNandevent.key==pg.K_ESCAPE:running=False# we could also have done the toggle hereSCREEN.fill("white")sm(events,dt)pg.display.flip()The default loading state draws a little spinner. If you want to change how that works, simply override theLoadingState.For example:fromStateManagerimportLoadingStateclassBlankLoadingState(LoadingState):def__init__(self,color):self.color=colordefupdate(self,*args):passdefdraw(self,s):s.fill(self.color)And use like thissm=StateManager({"play":PlayState(),"pause":PauseState()},start="play",loading_state=BlankLoadingState("black"))LicenseMIT License
Package: pgspecial
Meta-commands for PostgresThis package provides an API to execute meta-commands (AKA “special”, or
“backslash commands”) on PostgreSQL.Quick StartThis is a python package. It can be installed with:$ pip install pgspecialUsageOnce this library is included into your project, you will most likely use the
following imports:frompgspecial.mainimportPGSpecialfrompgspecial.namedqueriesimportNamedQueriesThen you will create and use an instance of PGSpecial:pgspecial=PGSpecial()forresultinpgspecial.execute(cur,sql):# Do somethingIf you want to import named queries from an existing config file, it is
convenient to initialize and keep around the class variable inNamedQueries:fromconfigobjimportConfigObjNamedQueries.instance=NamedQueries.from_config(ConfigObj('~/.config_file_name'))Contributions:If you’re interested in contributing to this project, first of all I would like
to extend my heartfelt gratitude. I’ve written a small doc to describe how to
get this running in a development setup.https://github.com/dbcli/pgspecial/blob/master/DEVELOP.rstPlease feel free to file an issue if you need help.Projects using it:This library is used by the following projects:pgcli: A REPL for Postgres.ipython-sql: %%sql magic for IPythonOmniDB: An web tool for database managementIf you find this module useful and include it in your project, I’ll be happy
to know about it and list it here.
Package: pgspot
pgspotSpot vulnerabilities in PostgreSQL extension scripts.pgspot checks extension scripts for following PostgreSQL security best
practices. In addition to checking extension scripts it can also be
used to check security definer functions or any other PostgreSQL SQL code.pgspot checks for the following vulnerabilities:search_path-based attacksunsafe object creationConsult thereferencefor detailed documentation of the vulnerabilities which
pgspot detects, and their potential mitigations.Useful linksPostgreSQL security recommendations for extensionsPostgreSQL security recommendations for SECURITY DEFINER functionsInstallationpip install pgspotRequirementspython >= 3.10pglast == 5.0libpg_query(through pglast)To install the runtime requirements, usepip -r requirements.txt.Usage> pgspot -h
usage: pgspot [-h] [-a] [--proc-without-search-path PROC] [--summary-only] [--plpgsql | --no-plpgsql] [--explain EXPLAIN] [--ignore IGNORE] [--sql-accepting SQL_FN] [FILE ...]

Spot vulnerabilities in PostgreSQL SQL scripts

positional arguments:
  FILE                  file to check for vulnerabilities

options:
  -h, --help            show this help message and exit
  -a, --append          append files before checking
  --proc-without-search-path PROC
                        whitelist functions without explicit search_path
  --summary-only        only print number of errors, warnings and unknowns
  --plpgsql, --no-plpgsql
                        Analyze PLpgSQL code (default: True)
  --explain EXPLAIN     Describe an error/warning code
  --ignore IGNORE       Ignore error or warning code
  --sql-accepting SQL_FN
                        Specify one or more sql-accepting functions> pgspot --ignore PS017 <<<"CREATE TABLE IF NOT EXISTS foo();"
PS012: Unsafe table creation: foo

Errors: 1 Warnings: 0 Unknown: 0SQL-accepting functionsIt is a common pattern that SQL-accepting functions exist, which take a
string-like argument which will be executed as SQL. This can "hide" some SQL
from pgspot, as the string-like argument masks the SQL. With the--sql-acceptingargument, pgspot can be told about such functions.Assuming a function namedexecute_sqlwhich takes a SQL string as its first
argument, and executes it. Withpgspot --sql-accepting=execute_sqlwe can
tell pgspotexecute_sqlmay accept SQL. pgspot will attempt to unpack and
evaluate all arguments to that function as SQL.
Package: pgsql2gist
Create maps using the Github Gist API as GeoJSON or TopoJSON, directly from PostGIS.dev-branchmasterObjectiveCreate a command line utility similar to pgsql2shp, but the output of which should be
a link to an anonymous private gist. The gist link should leverage Github’s
rendering of GeoJSON/TopoJSON, e.g. render a map.See issues for additional features to be implemented.Current Usage / CaveatsListed below is the –help usage information.:POST GeoJSON or TopoJSON features from PostGIS to a Github Gist.

Example usage: pgsql2gist --host localhost --user matt tilestache \
               "SELECT name, ST_AsGeoJSON(geom) AS geometry FROM neighborhoods LIMIT 5;"

Current SELECT Statement Requirements:
 - Geometry must be in EPSG:4326 WGS84 coordinate system
 - Geometry must be wrapped in ST_AsGeoJSON(), ST_AsTopoJSON()

usage: pgsql2gist [-f FILE] [-d DESCRIPTION] [-h HOST] [-p PORT] [-P PASSWORD]
                  [-u USER] [-g GEOM_COL] [-t TOPOLOGY_LAYER] [-v] [-?]
                  database SELECT

positional arguments:
  database              PostGIS database name.
  SELECT                SELECT Statement. NOTE: Geometry must be WGS84;
                        wrapped in ST_AsGeoJSON() or AsTopoJSON().

optional arguments:
  -f FILE, --file FILE  Filename. NOTE: Must end in 'geojson' or 'topojson'
                        extension. (default: upload.geojson)
  -d DESCRIPTION, --description DESCRIPTION
                        Description of upload (default: File uploaded using
                        pgsql2gist.)
  -h HOST, --host HOST  PostGIS database hostname. (default: None)
  -p PORT, --port PORT  PostGIS database port. (default: 5432)
  -P PASSWORD, --password PASSWORD
                        PostGIS user password. (default: None)
  -u USER, --user USER  PostGIS database user. (default: postgres)
  -g GEOM_COL, --geom-col GEOM_COL
                        Geometry column name as defined in SELECT statement.
                        (default: geometry)
  -t TOPOLOGY_LAYER, --topology-layer TOPOLOGY_LAYER
                        For TopoJSON Queries; Name of Topology Layer (default:
                        None)
  -v, --verbose         Verbose output. (default: False)
  -?, --helpResourcesGist API: (http://developer.github.com/v3/gists/)ThanksThe spiffy idea behind a db context manager comes from Migurski’s awesome TileStache
map server.http://tilestache.org
Package: pgsql-connection-handler
pgsql-connection-handlerA package that connects to one or more PostgreSQL databases and allows you
to retrieve a database connection when required. Depends on psycopg2-binary.QuickstartInstall the package:pip install pgsql-connection-handlerUse the package:frompg_connection_handlerimportdatabases,connect_databasesDATABASES={'default':{'dbname':'mydatabase','user':'guido','password':'SuperS3cr!t','host':'localhost','port':'5432','sslmode':'disable','application_name':'myapp',},'another_db':{'dbname':'mydatabase','user':'guido','password':'SuperS3cr!t','host':'localhost','port':'5432','sslmode':'disable','application_name':'myapp',},}connect_databases(DATABASES)db_connection=databases.get_connection()# Or get the connection based on the alias used in DATABASESdb_connection=databases.get_connection('db_alias')# Or get the database objectdb=databases.get()db_connection=db.connCompatiblityCompatible with Python 3.8 and above.VersioningThis project followssemantic versioning(SemVer).License, code of conduct and requirementsCheck the root of the repo for these files.