Saravanan666/pypi-org huggingface dataset:
Package: db-sheet
DBSheetUsing google spreadsheets as database.This package helps you make use of google sheets as databases. What all it supportsCreating TablesInserting RowsGet all recordsSearch for One RowSearch for particular number of RowsHow to initializefromdb_sheet.db_sheetimportDbSheetfromdb_sheet.sheet_tableimportWorkingTable,CreateTable,Tabledb=DbSheet(db_name="SHEETBACKEND",api_creds="client_secret.json",scope=['https://spreadsheets.google.com/feeds'])sheet=db.get_db_conn()Sheet is like database.Creating tablesfromdb_sheet.db_sheetimportDbSheetfromdb_sheet.sheet_tableimportWorkingTable,CreateTable,Tabledb=DbSheet(db_name="SHEETBACKEND",api_creds="client_secret.json",scope=['https://spreadsheets.google.com/feeds'])sheet=db.get_db_conn()table=Table({"Col1","col2"},"name_of_table")creator=CreateTable(table,sheet)creator.create_table()Inserting Rowsfromdb_sheet.sheet_tableimportWorkingTable,CreateTable,Tableworking_table=WorkingTable("tablename",sheet)working_table.insert({"value1","value2"}Get all recordsfromdb_sheet.sheet_tableimportWorkingTable,CreateTable,Tableworking_table=WorkingTable("tablename",sheet)working_table.get_all()Search for One Rowfromdb_sheet.sheet_tableimportWorkingTable,CreateTable,Tableworking_table=WorkingTable("tablename",sheet)working_table.get_one("column_name",value)Search for particular number of Rowsfromdb_sheet.sheet_tableimportWorkingTable,CreateTable,Tableworking_table=WorkingTable("tablename",sheet)working_table.find("column_name",value,total_count_u_need)Contribution:You can contibute by following below simple steps.Make sure your build is passingKeep your code clean and proper commenting with pep8.Thats it for now. Will add more since its just a begginging.
Suggestions are welcome. Please create an issue if you think this is **** and you can make it better :D@ Author: chowmean
Package: dbskit
# dbskit[![Build Status](https://api.travis-ci.org/listen-lavender/dbskit.svg?branch=master)](https://api.travis-ci.org/listen-lavender/dbskit)dbskit是一个简单的封装了mongo，mysql的数据库操作的工具集，支持线程和协程；orm的mongo和mysql操作是一致的，都遵循mong语法，例如query(select id, name from user where id = 2)需写成query({'id':2}, {'id':1, 'name':1})，操作的一致就可以轻松切换项目的数据库，只需要修改数据库的配置连接就可以了，当然sqlalchemy这种大型的orm已经支持的很好了，但是dbskit对于小型项目更直观，简单，方便控制.Maybe you have other choice to cross platform, like sqlalchemy, but dbskit is easy to use and control.## mysql (mongo syntax operation)>    - 支持线(协)程池>    - 基本的orm## mongo>    - 支持mongo的bulk操作>    - 基本的orm# Getting startedHere is a simple example orm operation for Dbskit:````python# coding=utf-8from dbskit.mysql import transferfrom dbskit.mysql.orm import *from dbskit.mysql.suit import dbpc, withMysqlclass MarkModel(Model):def __init__(self, **attributes):self.__mappings__['create_time'] = DatetimeField(ddl='datetime')self.__mappings__['update_time'] = DatetimeField(ddl='datetime')self.__mappings__['tid'] = IntField(ddl='int')attributes['create_time'] = attributes.get('create_time', datetime.datetime.now())attributes['update_time'] = attributes.get('update_time', datetime.datetime.now())for key in self.__mappings__:if not key in attributes:raise Exception('Need field %s. ' % key)attributes[key] = self.__mappings__[key].check_value(attributes[key])super(Model, self).__init__(**attributes)def __setstate__(self, state):self.__dict__ = statedef __getstate__(self):return self.__dict__def initDB():dbpc.addDB('test', 20, host='localhost',port=3306,user='root',passwd='',db='pholcus',charset='utf8',use_unicode=False,override=False)dbpc.addDB('test', 20, host='localhost',port=3306,user='root',passwd='',db='pholcus',charset='utf8',use_unicode=False,override=False)'''@comment('代理数据')'''class Proxy(MarkModel):__table__ = 'grab_proxy'ip = StrField(ddl='str', unique='daili')port = IntField(ddl='int(5)', unique='daili')location = StrField(ddl='varchar(30)')safetype = StrField(ddl='varchar(30)')protocol = StrField(ddl='varchar(30)')refspeed = FloatField(ddl='float')usespeed = FloatField(ddl='float')usenum = IntField(ddl='int(10)')status = IntField(ddl='tinyint(1)')initDB()@withMysql('test', resutype='DICT', autocommit=True)def test():print transfer({"$or" : [{"id_num" : {"$mod" : [5, 1]}}, {"winner" : True}]}, grand=None, parent='', index=index, condition=condition)a = Proxy.queryOne({'ip':'110.52.221.27'}, {'ip':1, 'port':1, 'usenum':1})print aProxy.update({'ip':'110.52.221.27'}, {'$inc':{'port':1}, '$set':{'usenum':2}})a = Proxy.queryOne({'ip':'110.52.221.27'}, {'ip':1, 'port':1, 'usenum':1})print aif __name__ == '__main__':test()````## InstallationTo install dbskit, simply:````bash$ pip install dbskit✨🍰✨````Satisfaction, guaranteed.## DocumentationTODO## Discussion and supportReport bugs on the *GitHub issue tracker <https://github.com/listen-lavender/dbskit/issues*.
Package: dbs-statement-downloader
DBS Statement DownloaderPython library that logs into the DBS iBanking frontend and download estatements using Selenium.Since logging into the frontend requires an OTP, text-forwarding (independent of this repository) is used to send the contents of the SMS to a Gmail service account. This OTP is then passed to the Selenium browser.Requests are then made to the undocumented DBS backend API (https://internet-banking.dbs.com.sg/api/v3/channels/estatements) to retrieve the relevant bank statements.InstallClone the repogitclonehttps://github.com/benjamin-awd/dbs-statement-downloader.gitInstall dependencies usingHomebrewandpoetrybrewinstallpoetry
poetryenvuse3.11
poetryshell
poetryinstallCreate .env file and update valuescp.env.template.envSeePrerequsitesfor information on Google Cloud/Gmail setupUsageThe downloader can be run directly:python3src/dbs/main.pyor with Docker compose:docker-composebuild&&docker-composeupPrerequisitesText-forwarding from iPhone/Android to an email account. On iOS, one possible method is to use anautomationto forward SMS messages from DBS to an email account.Google Cloud account, with the Gmail API enabled (see Google Clouddocs)Authorized Gmail account (see Gmail APIdocs)Secret stored on Google Secret Manager, which is defined in the .env file asSECRET_ID. The secret should be the contents of thetoken.jsonfile generated by running thequickstart.pyscript in this repository. This allows the OTP to be retrieved from Gmail.Note:quickstart.pyrequires acredentials.jsonfile to generate the Gmail access token, which can be generated with thesesteps.
Package: dbs-todo
dbs == dain-bread simple========================TODO list for minimalists# SPDX-License-Identifier: GPL-2.0-only# Copyright (c) 2022, Al Stone <ahs3@ahs3.net>A "task" consists of a file that looks something like this:Task: get something doneState: openProject: stuffPriority: hNote: this is a noteNote: this is another note(sans indenting, of course).  Each key:value is exactly one line (everythingup until the newline).  There are a few limits:-- State is one of 'open', 'active', 'done' or 'deleted' [0]-- Priority is one of 'h', 'm', 'l'It's all pretty self-explanatory.  I've tried to keep this as simple aspossible, but Python wants things packaged properly, and I wanted to haveboth a CLI version (dbs) and an ncurses version (dbsui) using the same codefor managing tasks.  So, yeah, needed to create a module to package anddistribute with.Installation is simple: clone the source tree, cd to the clone, then:$ make installOr:$ pip install dbs-todoThat's it.  Python3 is the only requirement.  Most distros will have theright Python modules by default.  I use this on Fedora and Debian, soyour mileage may vary.To add a task like the one above:$ dbs add 0001 stuff h "get something done"That's it.  It's automatically created as an open task.  You can find allopen tasks with:$ dbs lo('lo' == 'list open' -- extra credit if you can figure out the command for'list active' and 'list done').If you do this:$ dbsYou'll get a list of all current commands and any parameters they might need.There is a config file: it is always $HOME/.config/dbs/config. The only thingyou can specify is this:repo: <some directory path>If it does not exist, it will be created.  If the repo path does not exist,it will be created, also.  In the repo, there is a directory for each taskstate containing one file for each task in that state.  Task names must beunique across all states since we move the files around in the various repodirectories [1].With 0.6.0, I've added an ncurses-based UI.  It's crude, probably buggyeven.  It can be invoked with:$ dbsuiThere is a 'help' command ('?').  The left panel lists the projects thathave open tasks, and the right panel lists the tasks for the currentproject.  This is still experimental code so caveat emptor.Notes:[0] Nothing ever gets actually deleted unless you remove the files.[1] In order to sync across machines, I create ~/Dropbox/dbs and useDropbox to sync that directory across machines.
Package: dbstore
This distribution provides database backed storage using any PEP 249compliant database package in conjunction with flup.middleware.session.Storage and decorators for supplying WSGI apps with MySQLdb or psycopg2backed sessions are provided. Others backends are easy to hook up.@mysql_sessionizer(user='foo', passwd='bar', db='baz', host='localhost')def some_wsgi_app(environ, start_response):...You will need a database with a sessions table:CREATE TABLE sessions (id VARCHAR(32) NOT NULL,touched  FLOAT,pickle LONGTEXT,primary key (id));
Package: dbt
dbtenables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.This package installs thedbt Cloud CLIto invoke dbt commands from the command line that execute in adbt Cloudenvironment.Understanding dbtAnalysts using dbt can transform their data by simply writing select statements, while dbt handles turning these statements into tables and views in a data warehouse.These select statements, or "models", form a dbt project. Models frequently build on top of one another – dbt makes it easy tomanage relationshipsbetween models, andvisualize these relationships, as well as assure the quality of your transformations throughtesting.Join the dbt CommunityBe part of the conversation in thedbt Community SlackRead more on thedbt Community Discourse
Package: dbt2looker
dbt2lookerUsedbt2lookerto generate Looker view files automatically from dbt models.Want a deeper integration between dbt and your BI tool? You should also checkoutLightdash - the open source alternative to LookerFeaturesColumn descriptionssynced to lookerDimensionfor each column in dbt modelDimension groupsfor datetime/timestamp/date columnsMeasuresdefined through dbt columnmetadatasee belowLooker typesWarehouses: BigQuery, Snowflake, Redshift (postgres to come)QuickstartRundbt2lookerin the root of your dbt projectafter compiling looker docs.Generate Looker view files for all models:dbtdocsgenerate
dbt2lookerGenerate Looker view files for all models taggedproddbt2looker--tagprodInstallInstall from PyPi repositoryInstall from pypi into a fresh virtual environment.# Create virtual env
python3.7 -m venv dbt2looker-venv
source dbt2looker-venv/bin/activate

# Install
pip install dbt2looker

# Run
dbt2lookerBuild from sourceRequirespoetryand python >=3.7For development, it is recommended to use python 3.7:# Ensure you're using 3.7
poetry env use 3.7  
# alternative: poetry env use /usr/local/opt/python@3.7/bin/python3

# Install dependencies and main package
poetry install

# Run dbtlooker in poetry environment
poetry run dbt2lookerDefining measuresYou can define looker measures in your dbtschema.ymlfiles. For example:models:-name:pagescolumns:-name:urldescription:"Pageurl"-name:event_iddescription:unique event id for page viewmeta:measures:page_views:type:count
Package: dbt2looker-bigquery
dbt2lookerUsedbt2looker-bigqueryto generate Looker view files automatically from dbt models in Bigquery.This is a fork of dbt2looker that is specific to bigquery.
The intention is to allow one to define most of the simple and tedious lookml settings in dbt.
That way the lookml code gets less bloated, and can be more focused on advanced metrics and explores.Want a deeper integration between dbt and your BI tool?
You should also checkoutLightdash - the open source alternative to LookerFeaturesColumn descriptionssynced to lookerDimensionfor each column in dbt modelDefine Dimensionsdefine common lookml settings in dbt like label, group label, hiddenOpinionated Primary keyautomatically set the first column to be the primary key, and hide it.Dimension groupsfor datetime/timestamp/date columnsMeasuresdefined through dbt columnmetadatasee belowLooker typesWarehouses: BigQueryQuickstartRundbt2lookerin the root of your dbt projectafter compiling looker docs.Generate Looker view files for all models:dbtdocsgenerate
dbt2lookerGenerate Looker view files for all models taggedproddbt2looker--tagprod**Generate Looker view files for all exposed models **dbt2looker--exposed_onlyInstallInstall from PyPi repositoryInstall from pypi into a fresh virtual environment.# Create virtual env
python3.7 -m venv dbt2looker-venv
source dbt2looker-venv/bin/activate

# Install
pip install dbt2looker-bigquery

# Run
dbt2lookerBuild from sourceRequirespoetryand python >=3.7For development, it is recommended to use python 3.7:# Ensure you're using 3.7
poetry env use 3.7  
# alternative: poetry env use /usr/local/opt/python@3.7/bin/python3

# Install dependencies and main package
poetry install

# Run dbtlooker in poetry environment
poetry run dbt2lookerDefining measuresYou can define looker measures in your dbtschema.ymlfiles. For example:models:-name:pagescolumns:-name:urldescription:"Pageurl"-name:event_iddescription:unique event id for page viewmeta:looker:hidden:Truelabel:eventgroup_label:identifiersvalue_format_name:idlooker_measures:type:count_distincttype:count
Package: dbTable
dbTable is a innovative library built on the Python SQLite Library that
allows you to manipulate views on tables in Databases, select data from
them, and modify them in many ways. This library abstracts out all SQL
queries with native Python syntax, and has built-in lazy computation as
well. Furthermore, this library provides an interface to load tables
from SQLite into in-memory table structures like Pandas DataFrames or
Numpy Array for further analysis.Check out the full documentation atDOCUMENTATIONQuickstart Guide:InstallationMake sure you have numpy and tabulate installedpip install numpy
pip install tabulateInstall dbTable through PyPI (This’ll automatically install numpy and
tabulate)pip install dbTableYou can also install the built distribution in the dist/ folder, or for
portable cases, just download the dbTable.py file, and place it in your
project folderLoad dbTable into your interpreter/projectimportdbTableMake a connection to a databasedb=Database('test.db')# Replace with your database or use ":memory:" to load an in-memory databaseSelect a table:table1=db['table1']Exporting / ImportingTo convert a dbTable to any other form, use:new_df=db['table1'].to_df()# Returns a Pandas DataFrame Objectnew_table=db['table1'].to_table()# Returns a UCB Datascience Tablenew_ndarray=db['table1'].to_numpy_array()# Returns a Numpy ArrayTo save a DataFrame, UCB Table, or Numpy Array, use:db.store_table(new_df,'newtablename')Here’s a sampling of modifications and filters you can do with dbTabletable2=table1.select(['col1','col2'])# Selecting columns from databasetable3=table2.where(table2['col1']>20)# Filterstable4=table3.sort('col2',descending=False)# Sort datagrouped_table=table4.group(table4['col1'])table4['col2']=table4['col1']*10# Modify columnstable4['col3']=table4['col1']+table4['col2']table4['col3']*=2Have fun!Check out the full documentation atDOCUMENTATION
Package: dbt-accelerator
dbt_schema_md_generatorThis code automatically produces schema.yml and columns.md file with input from xlsx file.Please follow the Steps below:Input the program with your xlsx filename and model_name_to_appendEach Sheet Name will represent a Table and contains the Column_Name, Column_Description and tests if any.Each sheet must contain 'Column_Name' and 'Column_Description' as Headers in themIf you want to add Test you need to add 'Yes' respectively under the test column.Make Sure you have a Table_Descriptions Sheet present in your xlsx fileEnsure you have entries for all the Tables in Table_Descriptions SheetThe Sheet Name must match with the Column_Name given in the Table_Descriptions SheetPlease refer to the Sample_File provided for more clarity.
Package: dbt-accelrator
dbt_schema_md_generatorThis code automatically produces schema.yml and columns.md file with input from xlsx file.Please follow the Steps below:Input the program with your xlsx filename and model_name_to_appendEach Sheet Name will represent a Table and contains the Column_Name, Column_Description and tests if any.Each sheet must contain 'Column_Name' and 'Column_Description' as Headers in themIf you want to add Test you need to add 'Yes' respectively under the test column.Make Sure you have a Table_Descriptions Sheet present in your xlsx fileEnsure you have entries for all the Tables in Table_Descriptions SheetThe Sheet Name must match with the Column_Name given in the Table_Descriptions SheetPlease refer to the Sample_File provided for more clarity.
Package: dbt-adapters
dbt-tests-adapterThis package is responsible for:defining database connection methodscaching information from databasesdetermining how relations are definedThere are two major adapter types: base and sqlDirectoriesbaseDefines the base implementation Adapters can use to build out full functionality.sqlDefines a sql implementation for adapters that initially inherits the base implementation
and comes with some pre-made methods and macros that can be overwritten as needed per adapter.
(most common type of adapter.)Filescache.pyCached information from the database.factory.pyDefines how we generate adapter objectsprotocol.pyDefines various interfaces for various adapter objects. Helps mypy correctly resolve methods.reference_keys.pyConfigures naming scheme for cache elements to be universal.
Package: dbt-airflow-factory
DBT Airflow FactoryLibrary to convert DBT manifest metadata to Airflow tasksDocumentationRead the full documentation athttps://dbt-airflow-factory.readthedocs.io/InstallationUse the package managerpipto install the library:pipinstalldbt-airflow-factoryUsageThe library is expected to be used inside an Airflow environment with a Kubernetes image referencingdbt.dbt-airflow-factory's main task is to parsemanifest.jsonand create Airflow DAG out of it. It also reads config
files fromconfigdirectory and therefore is highly customizable (e.g., user can set path tomanifest.json).To start, create a directory with a following structure, wheremanifest.jsonis a file generated bydbt:.
├── config
│   ├── base
│   │   ├── airflow.yml
│   │   ├── dbt.yml
│   │   └── k8s.yml
│   └── dev
│       └── dbt.yml
├── dag.py
└── manifest.jsonThen, put the following code intodag.py:fromdbt_airflow_factory.airflow_dag_factoryimportAirflowDagFactoryfromosimportpathdag=AirflowDagFactory(path.dirname(path.abspath(__file__)),"dev").create()When uploaded to Airflow DAGs directory, it will get picked up by Airflow, parsemanifest.jsonand prepare a DAG to run.Configuration filesIt is best to look up the example configuration files intests directoryto get a glimpse of correct configs.You can useAirflow template variablesin yourdbt.ymlandk8s.ymlfiles, as long as they are inside
quotation marks:target:"{{var.value.env}}"some_other_field:"{{ds_nodash}}"Analogously, you can use"{{ var.value.VARIABLE_NAME }}"inairflow.yml, but only the Airflow variable getter.
Any other Airflow template variables will not work inairflow.yml.Creation of the directory with data-pipelines-cliDBT Airflow Factoryworks best in tandem withdata-pipelines-clitool.dpnot only prepares directory
for the library to digest, but also automates Docker image building and pushes generated directory to the cloud storage
of your choice.
Package: dbt-allure
dbt-allureEnhance your dbt testing with the dbt-allure plugin. It transforms dbt test results into detailed Allure reports, giving clear insights into your data transformations. Perfect for ensuring data integrity, this plugin makes test outcomes easy to understand and act upon. Dive deeper into testing with dbt by checking the dbt testing documentation.For more information on Allure, visitAllure Framework. To learn more about dbt and how it works, visit theofficial dbt documentation.InstallationEnsure you install the dbt-allure plugin with the same Python interpreter used by your dbt-core. The plugin can be installed via pip, poetry, or directly from a Git repository:Using pippipinstalldbt-allureUsing Poetrypoetryadddbt-allureUsageRunning dbt Commands to Generate Allure Test ResultsThe dbt-allure plugin can be utilized through programmatic invocations in Python or by using the provided dbt CLI wrapper.Programmatic Invocation:For more detailed information on programmatic invocation with dbt, refer to thedbt documentation on running dbt programmatically.fromdbt_allureimportallure_callbackfromdbt.cli.mainimportdbtRunnerdbt=dbtRunner(callbacks=[allure_callback,])dbt.invoke("test")CLI Wrapper (Bash):python-mdbt_alluretestGenerating Allure Test ReportsTo visualize your test results with Allure, generate and open the Allure report:exportTEST_RESULTS_DIR=target/allure-resultsexportTEST_REPORT_DIR=target/allure-report
alluregenerate$TEST_RESULTS_DIR-o$TEST_REPORT_DIR--clean
allureopen$TEST_REPORT_DIRPlugin ConfigurationThe default configuration file for the plugin is .dbt_allure.yml, which can be overridden by the environment variable DBT_ALLURE_CONFIG_PATH.Configuration optionsresults_dir_path: The path where Allure results will be stored.clean_results: A boolean flag to clean results_dir_path before a run. Note: When using programmatic invocation, cleanup occurs only upon importing the callback. It's recommended to set this to False and manage cleanup manually for programmatic usage.
Package: dbt-argo
dbt-argoA simple Python package to parse dbt dependencies to create an Argo Workflow job. It can also submit Argo files directly to the server.MotivationsSimplify CI steps to create Argo workflow for dbt projects.LimitationsThis version only support Google Cloud Storage as backend to storedbt artifactsto be used by Argo as artifacts.This version only supportCronWorkflowandWorkflowgenerationUsageBy default, it assumes thedbt_project.ymlfile is located at the root folder, if not theDBT_PROJECT_DIRcan be defined to locate the project path.Usage: dbt-argo [OPTIONS] COMMAND [ARGS]...

  Test

Options:
  --help  Show this message and exit.

Commands:
  cp      copy local folder to GCS Bucket
  create  Create a dbt Argo Workflow
  cron    Manage cron workflow
  submit  dbt-argo deploy {FILE}InspirationsHerafor Argo Python CLI + pydantic validation objectscoulerfor easy cluster submission (likekubectl apply -f ...)dbt > 1.5for easy manifest parsing directly in python scriptsArgo CLIfor cli optionskubectlfor simple CLI applicationcloudpathlibto deal with cloud file likepathlib
Package: dbt-artifact-parser
dbt-artifact-parserAn open source project to parse dbt artifacts which can be later used for alerting purposes.Supporting repository to setup the alerting service can be foundhereSetupWhile it is not necessary, we advise using a virtual environment with Python >= 3.8 before contributing to this project.Runmake initto setup the necessary dependencies and commit hooks.TestingRunmake testto run all tests.Upload new version to PyPipip install twine
python setup.py sdist
twine upload dist/*HelpTo see all the available commands for the repo, you can runmake help