Saravanan666/pypi-org huggingface dataset:
Package: glueetl
glueetlA command line tool to help deploy AWS Glue Jobs at ease :)Install$pipinstallglueetlHow to develop a Glue jobYou can develop a Glue job by following the steps below.1. Set up AWS Credentials and RegionBefore you can deploy a Glue job to AWS Glue, you must set up AWS Credentials and Region.$vim~/.aws/credentials[default]aws_access_key_id=<AWS_ACCESS_KEY_ID>aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>region=<REGION>2. Initialize a Glue job$mkdirsample
$cdsample
$glueetlinit.
├── README.md
├── config.yaml
└── script.pyconfig.yamlincludes job properties and currently it supports the following properties:job:name:sample-glue-jobrole_name:AWSGlueServiceRolescript_location:s3://glue-job-scripts/sample-glue-job/script.pymax_concurrent_runs:10command_name:glueetlmax_retries:0timeout:28800max_capacity:10connections:-first_connection-second_connectiondefault_arguments:argument1:value1argument2:value2non_overridable_arguments:argument1:value1argument2:value2trigger:name:trigger-sample-glue-jobschedule:cron(5 * * * ? *)tags:key1:value1key2:value2Please change default values in fileconfig.yamland write your job logic in filescript.py.3. Deploy a Glue job$cdsample
$glueetldeployYour job will be deployed to AWS Glue.4. Run a Glue jobYou can manually run your Glue job like this.$cdsample
$glueetlrun--arg1=value1--arg2=value2
Package: glue-genes
The glue-genes meta-packageglue genesis a custom version of theglue softwarefor the
visual exploration of genomics data, developed in partnership withThe Jackson Laboratory.Featuresglue genes provides all the core features of glue:Interactive, linked statistical graphicsA user interface for 'glueing' together multiple datasets with complicated relationshipsA highly scriptable and extendable environmentIn addition, glue genes provides:Data loaders for genomics data file formats, including multiresolution images and transcriptomics data.Viewers including: 2D Heatmap, Small Multiples, QTL ViewerMenubar plug-ins to facilitate analysis of single-cell and spatial transcriptomics dataInstallSeethe glue genes documentationfor installation help.We recommend installing into a new virtual environment. To install the current stable version use:pip install glue-genes[qt6]orpip install glue-genes[qt5]depending on which version of Qt you would like to use.To install the latest development version we recommend the following procedure in a new virtual environment:pip install "glue-genes[qt6]@git+https://git@github.com/gluesolutions/glue-genes.git"UsageAfter installation, glue (with the glue genes plug-ins loaded) can be invoked at the terminal:glueDevelopmentglue genes is under active development. Please file all bug reports as github issues.
Package: glue-geospatial
RequirementsNote that this plugin requiresglueto be
installed - seethis
pagefor
instructions on installing glue.In addition, this plugin requires therasterioandpyprojpackages to be
installed.InstallingIf you are using conda, you can easily install the
plugin and all the required dependencies with:conda install -c glueviz glue-geospatialAlternatively, if you don’t use conda, be sure to install the above
dependencies then install the plugin with:pip install glue-geospatialThis will auto-register the plugin with Glue.UsingAt the moment, this plugin provides a reader based on rasterio. You can
give glue any file that can be read by rasterio. For example, ifmydata.tifis a GeoTIFF file, you can do:glue mydata.tifand you can also load files from inside glue.TestingTo run the tests, do:py.test glue_geospatialat the root of the repository. This requires thepytestmodule to be installed.
Package: glue-ginga
glue-ginga is a plugin viewer for the Glue viewer that uses the Ginga
viewer widget for viewing 2D images. The Ginga widget provides an
alternative viewer with many customizable features and allows Ginga
users to use a familiar interface.In addition, it also has Glue global plugin for Ginga reference viewer,
which can be loaded as follows:ginga --modules=glue\_ginga.plugins.GlueFeaturesUse the Ginga viewer widget in GlueRead your custom Ginga key/mouse bindingsUse the Glue plugin in Ginga to transfer data between Ginga reference
viewer and Glue viewerInstallationInstalling from source:python setup.py installInstalling withpip:pip install git+https://github.com/ejeschke/glue-gingaSupportPleaseopen an
issue.LicenseGlue is licensed under theBSD
License
Package: gluegun
gluegunA small Python module containing quick utility functions for standard ETL processes.Installationpip install gluegunLinksSourceWikiIssuesSlackLicenseMITDependenciesNumPyPandasContributingThis project is maintained by thehotglueteam. We welcome contributions from the
community via issues and pull requests.If you wish to chat with our team, feel free to join ourSlack!
Package: glue-jupyter
Sometimes known as “Glupyter”AboutGlueis a Python library to explore
relationships within and among datasets. The main interface until now
has been based onQt, but theglue-jupyterpackage aims to provide a way to use Glue in Jupyter notebooks and
Jupyter lab instead. This is currently a work in progress and highly
experimental.For some notebooks with examples of usage of glue-jupyter, see thenotebooksdirectory.You can try out glue-jupyter online at mybinder:Notebooks with real data:Investigating star formation in the W5
region(example with linking a table and an image)Exploring the L1448 data in
3D(example of 3D volume rendering)Visualizing flight paths in the Boston
area(example with a single tabular dataset)Distance to the Pleiades with GAIA
dataInstallingFor now, installing should be done using pip:pip install git+https://github.com/glue-viz/glue-jupyter.gitOr from source:git clone https://github.com/glue-viz/glue-jupyter.git
cd glue-jupyter
pip install -e .TestingThe test suite can be run using:pytest glue_jupyter
Package: gluelib
gluelibThis is a simple library for interacting with the pastebin API. The code is fairly simple and self-documenting, and all other information needed is contained within this file. Install withpip3 install gluelib.(It's called gluelib because pastelib was taken)TerminologySession KeysWhen using this library, the first thing to do is generate a session key. This is what you will use instead of a username and password. CallgetSessionKeywith your pastebin username, password, and developer key, and you will receive a session key. This is used in all future API calls. Keep it in memory, and generate a new one for every session. You do not need to use your username and password again.Paste KeysThese are the parts of a pastebin URL that identify a specific paste. (e.g.zvwcdXyninhttps://pastebin.com/zvwcdXyn)FunctionsgetPaste(pasteKey, sessionKey, devKey)Returns the text of a paste given the above parameters.makePaste(pasteName, pasteText, sessionKey, devKey, expiryDate = "N", privacy = "0")Uploads a paste to pastebin with the above parameters. Seethe pastebin API pagefor more detail.deletePaste(pasteKey, sessionKey, devKey)Deletes a paste given the above parameters.listUserPastes(sessionKey, devKey, resultsLimit=50)Returns information about your pastes in XML format. Will only display lastresultslimitresults.getUserData(sessionKey, devKey)Returns information about your user profile in XML format.getSessionKey(username, password, devKey)Returns a session key when given a pastebin username, password, and developer key. See the section on session keys for more.outputBytesToString(outputBytes)This function formats a response from the pastebin API. It removes the prefix and suffix bytes, converts the response to a string, and returns it. All other functions in this library pass their output through it. You do not need to use this function in your code.
Package: glue-medical
RequirementsNote that this plugin requiresglueto be
installed - seethis
pagefor
instructions on installing glue.In addition, this plugin requires theGDCMandpydicompackages to be
installed. If you are using conda, you can easily install these with:conda install -c conda-forge gdcm pydicomInstallingTo install the latest developer version of the plugin from the git
repository, you can do:pip install https://github.com/glue-viz/glue-medical/archive/master.zipThis will auto-register the plugin with Glue.UsingAt the moment, this plugin provides a reader for DICOM files. You can
give glue either a DICOM file or a directory containing DICOM files. For
example, you can start glue using:glue mydata.dcmorglue directory_with_dicom_filesand you can also load files from inside glue. It is not yet possible to
load a directory from the ‘Open Data Set’ menu in glue.At the moment, if all the shapes of the DICOM files match inside the
directory, we combine the arrays into an array with a higher
dimensionality (for example we combine 2d arrays into a single 3d
array). In future, we will make it possible for users to control this
behavior.TestingTo run the tests, do:py.test glue_medicalat the root of the repository. This requires thepytestmodule to be installed.
Package: gluepkg
This is a commandline applicationAWS Glue Packaging Helper. To prepare packaging according to AWS Glue required
setup for external libraries dependencies.This is based on the specification laid down at:https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-python-libraries.htmlInstallpip install gluepkgUsagegluepkg --src_dir <src> --pkg_s3_path <target>ArgStatusDescriptionsrc_dirrequiredthe source directory path to be packagedpkg_s3_pathrequiredthe target object location in s3 (s3 URI)Basically it creates a zip file unto the desired target location of s3example:gluepkg --src_dir gluepkg/ --pkg_s3_path s3://bucket/folder/gluepkg.zipBuildBuild distributionpython setup.py bdist_wheel --universalDistribute to pypitwine upload dist/*project install (development)python setup.py installafterward you can run the commandgluepkg –src_dir <src> –pkg_s3_path <target>during development of this package
Package: glueplate
config frameworkWhat is Config framework?Your framework or library might need default settings and you want your user to change them.Many developer wrote this kind of code, and you are going to. GluePlate is it!FeaturesEasy to write - MergeYou and your user write easiry.Just import Glue as _ and write like nested dict.from glueplate import Glue as _


settings = _(
    GLUE_PLATE_ENVIRONMENT_VARIABLE_KEY = 'BASEPACKAGE_SETTINGS_MODULE',
    from_base = 'comming from base',
    to_be_override = 'I am base.',
)glueplate import variable namedsettingsfrom os.environ[‘GLUE_PLATE_BASE_MODULE’], this is a base settings.Then glueplate looks settings.GLUE_PLATE_ENVIRONMENT_VARIABLE_KEY and import variable namedsettingsfrom os.environ[settings.GLUE_PLATE_ENVIRONMENT_VARIABLE_KEY], this is a user customized settings. Customized settings needs only user needs override, or addition.from glueplate import Glue as _


settings = _(
    from_sub='comming from sub',
    to_be_override = 'I am sub.',
)They are merged and easy to use.from glueplate import config

assert('I am sub' == config.settings.to_be_override)
assert('comming from base' == config.settings.from_base)
assert('comming from sub' == config.settings.from_sub)Append to listYou may not want to override by you user, just add user’s additional data.glueplate provide special keyword prefix to append original settings variable.GLUE_PLATE_PLUS_BEFORE_User can append list before.# base
settings = _(
    list1 = [1,2,3],
)

# user customized
settings = _(
    GLUE_PLATE_PLUS_BEFORE_list1 = [5,4]
)

# config.settings.list1 == [5, 4, 1, 2, 3]GLUE_PLATE_PLUS_AFTER_Same as GLUE_PLATE_PLUS_BEFORE_ but append to backward.# base
settings = _(
    list1 = [1,2,3],
)

# user customized
settings = _(
    GLUE_PLATE_PLUS_AFTER_list1 = [5,4]
)

# config.settings.list1 == [1, 2, 3, 5, 4]Inherit other glueplateYou might use library using glueplate. Environment variableGLUE_PLATE_BASE_MODULEis only one on your process.Your base settings can indecate library’s gluplate settings module.GLUE_PLATE_PARENT_MODULESSpecify library’s gluplate settings module names as list.from glueplate import Glue as _


settings = _(
    GLUE_PLATE_PARENT_MODULES=[
        'parentpackage1.parentpackage1_settings',
        'parentpackage2.parentpackage2_settings'
    ]
)
Package: glue-qt
GlueGlue is a python project to link visualizations of scientific datasets
across many files. Click on the image for a quick demo:FeaturesInteractive, linked statistical graphics of multiple files.Support for manyfile
formatsincluding common image formats (jpg, tiff, png), ascii tables,
astronomical image and table formats (fits, vot, ipac), and HDF5.
Custom data loaders can also beeasily
added.Highlyscriptable and
extendable.InstallationFor installation documentation, visitglueviz.org.ContributingIf you are interested in contributing toglue, please read ourCode of ConductandContribution Guidelines.SupportPlease report problems toglueviz@gmail.com, oropen an
issue.LicenseGlue is licensed under theBSD
License.
Package: gluer
GluerDependency Injection for PythonInstallationGluer usestype annotations, so python 3.6+ is requiredpython3 -m pip install --user gluerUsageLet's follow awonderful explanationof DI:fromabcimportABC,abstractmethodfromgluerimportGluerclassDrink(ABC):# this doesn't *have* to be abstract@abstractmethoddefsip(self):passclassChild:def__init__(self,drink:Drink):self.drink=drinkdeftake_a_sip(self):self.drink.sip()print("ahhh")classJuice(Drink):defsip(self):print("*gulp*")classCoke(Drink):defsip(self):print("*fizzle*")if__name__=="__main__":gluer=Gluer()mom_watches=Truegluer.register(Child)gluer.register(Juiceifmom_watcheselseCoke).As(Drink)container=gluer.build()kid=container.resolve(Child)kid.take_a_sip()# *gulp* ahhh
Package: glue-samp
RequirementsNote that this plugin requiresglueto be
installed - seethis
pagefor
instructions on installing glue.InstallingIf you are using conda, you can easily install the
plugin and all the required dependencies with:conda install -c glueviz glue-sampAlternatively, if you don’t use conda, be sure to install the above
dependencies then install the plugin with:pip install glue-sampIn both cases this will auto-register the plugin with Glue.TestingTo run the tests, do:py.test glue_sampat the root of the repository. This requires thepytestmodule to be installed.
Package: gluestick
gluestickA small Python module containing quick utility functions for standard ETL processes.Installationpip install gluestickLinksSourceWikiIssuesSlackLicenseMITDependenciesNumPyPandasContributingThis project is maintained by thehotglueteam. We welcome contributions from the
community via issues and pull requests.If you wish to chat with our team, feel free to join ourSlack!
Package: glue-struct-transform
Jsonschema to Glue structA small library that works to read your provided jsonschema and convert it to a glue struct model, in this way you can integrate it in your pipeline or uses it just to get a output and insert manually in your glue table schema.Installationpip install glue-struct-transformGet StartedHow to convert a full json schema to a glue struct schema using this libimportjsonfromglue_struct_transformimportGlueStructTransform# Read or set your json schemaschema=open(f'YOUR_JSON_SCHEMA.json',)schemaLoad=json.load(schema)# Pass your json schema in dict type to the functionresult=GlueStructTransform.json_schema_to_glue_struct(schemaLoad)How to convert a object into the json schema to a glue struct schema using this libimportjsonfromglue_struct_transformimportGlueStructTransform# Read or set your json schemaschema=open(f'YOUR_JSON_SCHEMA.json',)schemaLoad=json.load(schema)# Pass your json schema in dict type to the functionresult=GlueStructTransform.json_schema_to_glue_struct(schemaLoad,objectField="objectKey",fullSchema=False)How to convert a full json to a glue struct schema using this libimportjsonfromglue_struct_transformimportGlueStructTransform# Read or set your json schemapayload=open(f'YOUR_JSON_FILE.json',)payloadLoad=json.load(payload)# Pass your json schema in dict type to the functionresult=GlueStructTransform.json_schema_to_glue_struct(payloadLoad)How to convert a object into the json file to a glue struct schema using this libimportjsonfromglue_struct_transformimportGlueStructTransform# Read or set your json schemapayload=open(f'YOUR_JSON_FILE.json',)payloadLoad=json.load(payload)# Pass your json schema in dict type to the functionresult=GlueStructTransform.json_schema_to_glue_struct(payloadLoad,objectField="objectKey",fullBody=False)Conversions implementation statusFromFrom TypeGlue Schema typeImplementation statusVersionJson Schemastringstringimplemented0.1.0Json Schemanumberdoubleimplemented0.1.0Json Schemaintegerintimplemented0.1.0Json Schemaobjectstructimplemented0.1.0Json Schemaarrayarrayimplemented0.1.0Json Schemabooleanbooleanimplemented0.1.0Json SchemanullRaises exceptionimplemented0.1.2Jsonstrstringimplemented0.3.0Jsonfloatdoubleimplemented0.3.0Jsonintintimplemented0.3.0Jsondictstructimplemented0.3.0Jsonlistarrayimplemented0.3.0JsonnullRaises exceptionimplemented0.3.0
Package: gluettalax
GLUEttalaxGlue ETL without constipationusage: gluettalax <command> [parameters]

Commands:
 add_partition <db> <table> [--partition1=value...] [--location=path]
    Create a new Glue partition.
    Example: add_partition datalake usage --year=2019 --month=09

 add_partitions <db> <table> [s3_path]
    Create new Glue partitions in a given location.
    Example: add_partition datalake usage s3://example/usage/year=2020/month=10

 del_partition <db> <table> [--partition1=value...]
    Delete a Glue partition.
    Example: del_partition datalake usage --year=2019 --month=09

 help [command]
    Display information about commands.

 list_crawlers [pattern] [--noheaders]
    List Glue crawlers.
    Example: list_crawlers 'test*' --noheaders

 list_jobs [pattern] [--noheaders]
    List Glue jobs.
    Example: list_jobs 'test*'

 list_partitions <db> <table> [pattern] [--noheaders]
    List the partitions in a table.
    Example: list_partitions datalake usage

 list_runs [job_name] [--lines=num] [--noheaders]
    Print Glue jobs history.
    Example: list_runs my_batch_job --lines 10

 list_tables [pattern] [--noheaders]
    List Glue tables.
    Example: list_tables 'test*' --noheaders

 run_crawler <crawler_name> [--async] [--timeout=seconds]
    Run a crawler. If not async, wait until execution is finished.
    Example: run_crawler my_usage_crawler --async

 run_job <job_name> [--async] [--param1=value...]
    Run a Glue job. if not async, wait until execution is finished.
    Example: cmd_run_job --DATALAKE_BUCKET=test --THE_DATE=20191112 --HOUR=15

Command aliases:
 lsc -> list_crawlers
 lsj -> list_jobs
 runc -> run_crawler
 lsr -> list_runs
 runj -> run_job
 lsp -> list_partitions
 addp -> add_partition
 rmp -> del_partition
 lst -> list_tables
Package: gluetube
gluetubeA simple python script scheduler with a shared local key-value database. Meant to enable easy automation and integration of infrastructure and systems. Like cron, but with more bells and whistles.installationThere are two ways to deploy gluetube.dockervirtual machine / bare metaldockerdocker volume create gluetube-cfgdocker volume create gluetube-dbdocker run -d --init --name gluetube -v gluetube-cfg:/home/gluetube/.gluetube/etc -v gluetube-db:/home/gluetube/.gluetube/db ctomkow/gluetubedocker exec -it gluetube bashgt db --rekeyVMadduser gluetubepip install --user gluetubegt --configuregt db --initgt daemon --backgroundgt db --rekeyexample usagegt --helpgt summarygt schedule 1 --nowpipeline developmentYou are meant to develop your own pipelines in python for gluetube. The following is a brief description of how to get your development environment setup. These instructions assume you useVS codeanddocker.docker pull ctomkow/gluetubedocker volume create gluetube-dbdocker volume create gluetube-cfgdocker run -itd --name gluetube --net=host -v gluetube-db:/home/gluetube/.gluetube/db -v gluetube-cfg:/home/gluetube/.gluetube/etc ctomkow/gluetube:latestOpen VS code. In the docker-> containers section, right-click on the running gluetube container andattach visual studio codeto running container.Within the container terminal, enter the pipelines directory.cd pipelinesClone a repository.git clone <url> .Note the.at the end of the git clone. This is important because we don't want the project folder as a subdirectory.Now you can develop your pipelines (.py files) while having a live gluetube daemon running within the container. All the gluetube cli commands are available to test out your pipeline code in a production-like environment.gt --helproadmapgluetube roadmap
Package: glue-vispy-viewers
RequirementsNote that this plugin requiresGlueandPyOpenGLto be installed - seethis pagefor
instructions on installing Glue. PyOpenGL should get installed
automatically when you install the plugin (see below).While this plugin uses VisPy, for now we bundle our own version of VisPy
since we rely on some recently added features, so you do not need to
install VisPy separately.InstallingIf you use the Anaconda Python Distribution, you can install this plugin
with:conda install -c glueviz glue-vispy-viewersTo install the latest stable version of the plugin, you can do:pip install glue-vispy-viewersor you can install the latest developer version from the git repository
using:pip install git+https://github.com/glue-viz/glue-vispy-viewers.gitThis will auto-register the plugin with Glue. Now simply start up Glue,
open a data cube, drag it onto the main canvas, then select ‘3D viewer’.TestingTo run the tests, do:py.test glue_vispy_viewersat the root of the repository. This requires thepytestmodule to be installed.Using the isosurface viewerThe isosurface viewer is currently still unstable - to enable it, put
the following in a file calledconfig.pyfile in your current
working directory:from glue_vispy_viewers.isosurface import setup as setup_isosurface
setup_isosurface()
Package: glueviz
This package does not contain any code, but instead is a meta-package that
depends on the core glue code as well as stable plugins that we want to
provide to users by default.It can be installed with:pip install gluevizThis currently installs:glue-core, the main glue packageglue-vispy-viewers,
which adds 3D viewersIn future, as more plugin packages become stable, we will include them
in this meta-package.Issues should be openedin the repository for the main glue package.
Package: glue-wwt
RequirementsNote that this plugin requiresglueto be
installed - seethis
pagefor
instructions on installing glue.InstallingIf you are using conda, you can easily install the
plugin and all the required dependencies with:conda install -c glueviz glue-wwtAlternatively, if you don’t use conda, be sure to install the above
dependencies then install the plugin with:pip install glue-wwtIn both cases this will auto-register the plugin with Glue.
Now simply start up Glue,
open a tabular dataset, drag it onto the main canvas, then select
‘WorldWideTelescope (WWT)’.TestingTo run the tests, do:py.test glue_wwtat the root of the repository. This requires thepytestmodule to be installed.