Saravanan666/pypi-org huggingface dataset:
Package: aws-ecs-gcp-workload-identity-federation
aws-ecs-gcp-workload-identity-federationAWS to GCP Workload Identity Federation in Amazon ECS.This is a package for working withWorkload Identity Federationon Amazon ECS (EC2 or Fargate).When you import this package, the ECS task role credentials are set as environment variables, so you can use them without any special awareness.Quick startRefer to thedocumentation, set up Workload Identity federation and enable access to GCP resources from AWS.Download credential configuration file (e.g.config-aws-provider.json) and point it to the credential configuration file path.$exportGOOGLE_APPLICATION_CREDENTIALS=/path/to/config-aws-provider.jsonFor Amazon ECS, by task definitions, you can pass environment variables to a container (documentation).ExampleFor example, when executing a query from AWS to BigQuery, it will be as follows.importaws_ecs_gcp_workload_identityfromgoogle.cloudimportbigqueryproject_id='my-workload-identity'bqclient=bigquery.Client(project=project_id)sql="""SELECT name, ageFROM `my-data-project.my_dataset.my_table`"""query_job=bqclient.query(sql)results=query_job.result()forrowinresults:print("{},{}".format(row.name,row.age))ContributionFork (https://github.com/ohsawa0515/aws-ecs-gcp-workload-identity-federation/fork)Create a feature branchCommit your changesRebase your local changes against the master branchCreate new Pull RequestLicenseSeeLICENSE.
Package: aws-ecs-service
Aws Ecs ServiceA custom ecs service resource which does not have bugs that traditional ecs service resource does.RemarksThe project is written byLaimonas Sutkusand is owned byiDenfy. This is an open source
library intended to be used by anyone.iDenfyaims
to share its knowledge and educate market for better and more secure IT infrastructure.Release history1.3.0Add md files.1.2.0Dependency update 1.60.0 - 2.0.0.1.1.0Force CDK dependency update to 1.44.0.1.0.7Fix dependencies.1.0.6Various bugs fixed related to custom resource. Added more logging
for better debugging. Improved kwargs fixer. Improved cfn
response sender.1.0.5Implement a fail-safe for json dumps.1.0.4Human error fix.1.0.3Fix kwargs by converting string-integers to integers.1.0.2Mitigate CloudFormation bug.1.0.1Do not have global boto3 resources/clients.1.0.0Initial project. Not tested. No extensive readme.
Package: aws-ecs-task-render
Amazon ECS task definition rendering CLIA package for ECS task definition rendering.
From the idea ofGitHub action, I've started to work on this little tool that is usable locally or from your runners with AWS role attached.
With this tool is possible to chose a profile to use, define the task definition family name and all the image uri to update in a single command.Table of ContentInstallationUsageLicense SummaryInstallationYou can install the package usingpip install aws-ecs-task-render.UsageOnce installed, it's possible to use the CLI simply running the command:aws-ecs-task-render -p your_profile -f task_family_name -if image_name=new_image_uriIt's possible to update more than one image at time adding other key=value to the-ifcommand like that:aws-ecs-task-render -p your_profile -f task_family_name -if image_name_1=new_image_uri_1 image_name_2=new_image_uri_2This tool can be used only to update an already existing container definition, it cannot add a new one.
It doesn't generate a new task definition, it only render it returning a cleaned json output usable for a new task definition creation.License SummaryThis code is made available under the GPLv3 license.
Package: awsee
AWSeeCommand line tool that helps handle multiple AWS Account Credentials (its Roles and MFAs), when working with AWSCLI sessions.It opens and controls AWS Sessions using a command-line tool, allowing performing one or all of the following tasks:Start session using a MFA TokenAssume a RoleOpen a session in the same or another terminal windowKeep track of session information (expiration)"Injects" alias/DOSKEY commands in your opened session, that's configurable. Entries in file, examples:alias s3='aws s3 ls'alias info='awsee -i'Plus, it also tries (if allowed by your AWS Polices) automatically bring and save locally information about your MFA-Devices to be used later on.When not possible to retrieve account information automactically (due to security reasons), you still can register it locally using this tool. You can save and keep them updated, to help to deal with your AWS Sessions, like:MFA-DevicesRolesSave and mantain your credential profiles associated with their respective Roles and MFA Devices.
You can use the command "synchronize" to remotely bring this information (if allowed by your AWS Polices), check help documentation.The complete help/manual is available inside the AWSee tool itself, check it out:# To show a short version...$:~awsee-h# Show how to use, see below:$:~awsee-u
awsee[TOKEN-MFA][-pprofile][-rrole][-ntor--new-term]Examples:awsee857946OpenAWSSessionwithdefaultprofilewithyoursix-digitMFATokenawsee857946-pinfraOpenAWSSessionwiththepassedprofileusingyoursix-digitMFATokenawsee857946-pinfra-rprodOpenAWSSessionwiththepassedprofileusingyoursix-digitMFATokenandimmediatellyassumethepassedroleatthisnewsessionawseeOpenAWSSessionwithdefaultprofile,itwillcheckifneedsMFAawsee-pdevOpenAWSSessionwiththepassedprofile,asksforMFAifneededawsee-rdeveloperAssumeaRoleinsidethecurrentAWSSessionawsee-pdev-rdeveloperOpenAWSSessionwiththepassedprofile,andfollowingassumetheroleawsee-pprod-radminOpenAWSSessionwiththepassedprofile,andfollowingassumetherolawsee-ntor--new-terminalWhenopeninganAWSSession,willdoitinanewterminalawsee-hor--helpShowthisandallauxiliaryfunctions(use--helpformoreinformation)awsee-uor--usageShowonlyhowtouse,nottheauxiliaryfunctionawsee[FUNCTION]Executeanauxiliaryfunction,seebelowforallavailableoptions# Use the awsee --help to see all the functions and their usage# More things...# At the folder ~/.awsee/ you'll find two files that you can customize a few more things:# ~/.awsee/awsee.ini### --> Colors on/off### --> Emoticons on/off### --> At Windows, the New Terminal to be used (CMD, CMDER or GitBash)# ~/.awsee/alias/alias_awsee.txt### --> Add/Remove here your ALIAS/DOSKEYS (Linux/Windows) for new opened sessions, things like:### ----> DOSKEY ec2=aws ec2 describe-instances --query Reservations[*].Instances[*].InstanceId --output table### ----> DOSKEY s3=aws s3 ls### ----> DOSKEY ssmfrodo=aws ssm start-session --target "i-0452a94919225f816"For more information and details, check here:AWSee
Package: aws-empty-bucket
AWS Empty BucketA custom S3 bucket with an ability to completely delete itself
(even if it contains files within).RemarksThe project is written byLaimonas Sutkusand is owned byiDenfy. This is an open source
library intended to be used by anyone.iDenfyaims
to share its knowledge and educate market for better and more secure IT infrastructure.Related technologyThis project utilizes the following technology:AWS(Amazon Web Services).AWS CDK(Amazon Web Services Cloud Development Kit).AWS CloudFormation.AWS S3(Amazon Web Services Simple Storage Service).AssumptionsThis library project assumes the following:You have knowledge in AWS (Amazon Web Services).You have knowledge in AWS CloudFormation and AWS S3.You are managing your infrastructure with AWS CDK.You are writing AWS CDK templates with a python language.InstallThe project is built and uploaded to PyPi. Install it by using pip.pipinstallaws-empty-bucketOr directly install it through source../build.sh-icDescriptionNatively S3 buckets can not be deleted if they contain files. If you were to
delete a bucket through CloudFormation, you would get a similar error message:The bucket you tried to delete is not empty(Service: Amazon S3; Status Code: 409; Error Code: BucketNotEmpty;
Request ID:; S3 Extended Request ID:)This gets especially annoying if a developer is spinning up and tearing down
the infrastructure many times a day. Wouldn't it be awesome if S3 buckets could
just be simply deleted in any case?With this project you can create S3 buckets that can be deleted even if they
contain filed inside. A project exposes a classEmptyS3Bucketwhich can
be used exactly the same as a classBucketprovided by AWS CDK. Next time
you delete your stack, you will not see that error message again.ExamplesTo create an S3 Bucket that can be easily deleted create anEmptyS3Bucketinstance in your stack. An example is given below:fromaws_cdkimportcore,aws_s3fromaws_empty_bucket.empty_s3_bucketimportEmptyS3BucketclassMainStack(core.Stack):def__init__(self,scope:core.App)->None:super().__init__(scope=scope,id='MyCoolStack')self.empty_bucket=EmptyS3Bucket(self,'MyCoolBucketThatCanBeDeleted',access_control=aws_s3.BucketAccessControl.PRIVATE,bucket_name='mybucket',)To delete inner S3 Bucket files, a custom resource with a lambda function as
as a backend is created too.EmptyS3Bucketexposes two properties:backendandcustom_resource. If you need to access them use the following:fromaws_empty_bucket.empty_s3_bucketimportEmptyS3Bucketempty_bucket=EmptyS3Bucket(...)function=empty_bucket.backendresource=empty_bucket.custom_resourceRelease history2.4.0Add md files.2.3.0Dependency update 1.60.0 - 2.0.0.2.2.0Force AWS CDK dependency update to 1.44.0.2.1.0Dependencies update.2.0.1Do not use singleton lambdas since we are using roles for specific buckets.2.0.0Make a custom S3 bucket with inner-file deletion capabilities.1.0.0Initial project. Not tested. No extensive readme.
Package: aws-empty-ecr-repository
AWS Empty ECR RepositoryA custom ECR Repository resource which can delete itself event if it contains docker images within.RemarksThe project is written byLaimonas Sutkusand is owned byiDenfy. This is an open source
library intended to be used by anyone.iDenfyaims
to share its knowledge and educate market for better and more secure IT infrastructure.Release history1.3.0Add md files.1.2.0Force dependency update to 1.60.0 and add upper bound 2.0.0.1.1.0Force AWS CDK dependency version update to 1.44.0.1.0.2Use ECR API instead of ECS API.1.0.1Delete README copied info.1.0.0Initial. No extensive readme.
Package: aws-emr-cost-calculator
EMR Cost CalculatorFeatures at a glanceCalculates exact costs of an EMR cluster (EMR + EC2 costs)Multiple EMR clusters cost calculation for a given periodSpot prices and all other prices are exact and retrieved every time from AWS Pricing APIIf a cluster is still running, costs incurred up to current time are displayedWhy the need for this scriptGiven that Amazon doesn't provide a straightforward solution to calculate the cost of an EMR workflow, this module aims to calculate the cost of an EMR workflow given a period of days, or the cost of a single cluster given the cluster id. The simple way to do that would be to use the information given by the JobFLow method of the boto.emr module. However, this method doesn't return any information about the Task nodes of a cluster, and whether or not spot instances were used. This cost calculator takes care of both. OnDemand instance prices are retrieved using the AWS pricing API. In case spot instances were used, the price is retrieved using the AWS EC2 API.How it worksThis module is usingdocoptto parse command line arguments.It currently supports two operations:Get the total cost of an EMR workflow for a given period of daysaws-emr-cost-calculator total --created_after=<YYYY-MM-DD> --created_before=<YYYY-MM-DD>Get the cost of an EMR cluster given the cluster idaws-emr-cost-calculator cluster --cluster_id=<j-xxxxxxxxxxxx>Authentication to AWS API is done using credentials of AWS CLI which are configured by executingaws configureInstallTo install all requirements it's best to usepip install -r requirements.txtUsers with python<2.7.9 won't be able to run the code if requests[security] isn't installed (which is listed in requirements.txt)Python 3.7 is tested, lower 3x versions will probably work though.LicenseDistributed under the MIT license. SeeLICENSEfor more information.
Package: aws-emr-cost-calculator2
EMR Cost CalculatorThis project is a fork of [https://github.com/marko-bast/aws-emr-cost-calculator].Original features:Calculates exact costs of an EMR cluster (EMR + EC2 costs)Multiple EMR clusters cost calculation for a given periodSpot prices and all other prices are exact and retrieved every time from AWS Pricing APIIf a cluster is still running, costs incurred up to current time are displayedIn addition to the original project:calculate cost for EMR Instance Fleetscompute cost of EBSWhy the need for this scriptGiven that Amazon doesn't provide a straightforward solution to calculate the cost of an EMR workflow, this module aims to calculate the cost of an EMR workflow given a period of days, or the cost of a single cluster given the cluster id. The simple way to do that would be to use the information given by the JobFLow method of the boto.emr module. However, this method doesn't return any information about the Task nodes of a cluster, and whether or not spot instances were used. This cost calculator takes care of both. OnDemand instance prices are retrieved using the AWS pricing API. In case spot instances were used, the price is retrieved using the AWS EC2 API.EBS cost is fixed to 0.1 $ for month.InstallTo install or upgrade the package it's best to use pip:pip install -U aws-emr-cost-calculator2How it worksThis module is usingdocoptto parse command line arguments.It currently supports two operations:Get the cost of an EMR cluster given the cluster idaws-emr-cost-calculator2 cluster --cluster_id=<j-xxxxxxxxxxxx>Authentication to AWS API is done using credentials of AWS CLI which are configured by executingaws configureFor example (if you want to use a different profile):aws-emr-cost-calculator2 cluster --cluster_id=<j-xxxxxxxxxxxx> --profile=<your profile>Remember to include your default region in the default section of your ./aws/config file.Example run$ aws-emr-cost-calculator cluster --cluster_id=j-K3C155R34111 --profile=myorg
CORE.EC2    :   0.40
CORE.EMR    :   0.11
CORE.EBS    :   0.01
MASTER.EC2  :   0.40
MASTER.EBS  :   0.01
MASTER.EMR  :   0.11
TASK.EC2    :   2.80
TASK.EMR    :   3.12
TOTAL       :   6.95How release a new moduleTo release a new module you have to use:pypitwineFor example, to deploy the 0.1.1 version:python setup.py sdist
twine upload dist/*0.1.1* -r pypiLicenseDistributed under the MIT license. SeeLICENSEfor more information.
Package: aws-emr-launcher
aws-emr-launcheraws-emr-launcheris a generic python library that enables to provision emr clusters with yaml config files (Configuration as Code).Get started by installing the packages.pipinstallpyyaml==5.2

pipinstallaws-emr-launcherFeaturesConfiguration as Code to launch EMR clusters.Support to launch EMR clusters in multiple AWS regions.Provision for environment specific configuration files.Works with lambda layers.DocumentationDocumentation for this project is available inGitHub Wiki.
Package: aws-eni-identifier
AWS ENI IdentifierIdentify to which AWS service network interface is associatedInstallationpipinstallaws-eni-identifierUsageaws-eni-identifier does not connect to AWS by itself, so you will need to load data with aws-cliLogin to aws:awsssologin--profilemy-profileUse pipe:awsec2describe-network-interfaces|aws-eni-identifierOr save to file with aws-cli and read it:awsec2describe-network-interfaces>ni.json
aws-eni-identifier-ini.jsonShow extra columnsawsec2describe-network-interfaces|aws-eni-identifier\--add-columnAttachment.Status\--add-columnAvailabilityZoneFilterFind unused network interfaces:awsec2describe-network-interfaces\--filters"Name=status,Values=available"|aws-eni-identifierFind AWS resource by IP address (you can use public or private IP address)exportIP='51.21.223.193';awsec2describe-network-interfaces\--query"NetworkInterfaces[?PrivateIpAddresses[?PrivateIpAddress=='${IP}' || Association.PublicIp=='${IP}']]"|aws-eni-identifierDetermine what is using specific AWS network interfaceawsec2describe-network-interfaces\--network-interface-idseni-0068ac3f8786de59a|aws-eni-identifierYou can find more information about filters and queries inAWS documentationDevelopingInstall the package:poetryinstallRun tests:poetryrunpytest
Package: aws-eni-tagger
AWS ENI TaggerThis CLI tool adds meaningful tags to your AWS Elastic Network Interfaces (ENIs) based on information obtained fromaws-eni-idetifier.Before:After:Installationpipinstallaws-eni-taggerUsageaws-eni-tagger--profilemy-aws-profileArguments:The CLI takes the following arguments:--profile: AWS profile name. Required.--owner-service-tag-name: Key for tag with information about the ENI owner service. Default is 'OwnerService'.--owner-service-name-tag-name: Key for tag with information about the ENI owner service name. Default is 'OwnerServiceName'.--overwrite: If set, the tags with the keys with name specified above/config will be overwritten. Default is False.For example:aws-eni-tagger--profilemy_aws_profile--overwriteThis will run the aws-eni-tagger with the specified AWS profile and will overwrite existing tags.DevelopingInstall the package:poetryinstallRun tests:poetryrunpytest
Package: awsenv-cli
Usage of awsenvInstallboto3to your base envAdd the following awsenv function to your shell resource config (e. g. .zshrc or .oh-my-zsh/custom/awsenv.zsh)awsenv(){eval$(awsenv-cli$@)}Add aconfig.jsonc(note the c for comment) with the following strutucture under~/.awsenv/config.json{"AWS_DEFAULT_REGION":"eu-central-1","AWS_DEFAULT_OUTPUT":"json","AWS_PAGER":"","defaultUser":"<your-name-you-want-to-login-with-by-default,must-match-a-name-below-in-users>","defaultRole":"<some-role-name>","defaultAccount":"<some-account-id>","defaultDuration":3600,"users":{"<your-name-1>":{"AWS_ACCESS_KEY_ID":"<your-access-key>","AWS_SECRET_ACCESS_KEY":"<your-secret-access-key>","SERIAL_NUMBER":"<your-mfa-serial>"}},"predefinedShortcuts":{"<some-shortcut-1>":{"user":"<your-name>","role":"<some-role-name>","account":"<some-account-id>","duration":43200},"<some-shortcut-2>":{"user":"<your-name>","role":"<some-role-name>","account":"<some-account-id>","duration":3600}}}Development of awsenv
Package: awsenviron
python-awsenviron================[![Supported Versions](https://img.shields.io/pypi/pyversions/awsenviron.svg)](https://pypi.python.org/pypi/awsenviron)[![Build Status](https://travis-ci.org/douglasfarinelli/python-awsenviron.svg?branch=master)](https://travis-ci.org/douglasfarinelli/python-awsenviron)[![Coverage Status](https://coveralls.io/repos/github/douglasfarinelli/python-awsenviron/badge.svg?branch=master)](https://coveralls.io/github/douglasfarinelli/python-awsenviron?branch=master)[![PyPI version](https://badge.fury.io/py/awsenviron.svg)](https://pypi.python.org/pypi/awsenviron)The `awsenviron` reads the key, value pair from [AWS Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html) and adds them to environment variable. This was inspired by [python-dotenv](https://github.com/theskumar/python-dotenv) and [12-factor](https://12factor.net/config) principles.Installation============```bashpip install awsenviron```or```bashpipenv install awsenviron```Usage=====```pythonawsenviron.load_from_parameter_store(path='<your-path>')```Now, you can access the variables either from system environment variable:```pythonimport osDATABASE_URI = os.environ.get('DATABASE_URI')```Authentication==============awsenviron use `boto3` to authentication, [click here](http://boto3.readthedocs.io/en/latest/guide/configuration.html) to see the methods.
Package: aws-es-auth-proxy
# aws-es-auth-proxyA proxy that permits to connect to the AWS managed Elasticsearch cluster
protected with the AWS authentication accessing kibana and other stuffs
without authentication.# Installation with PIP` pip installaws-es-auth-proxy`# How to run the applicationTo lunch the application you have to set some parameter in the
command line. The only required parameter isaws_service_endpointwhich require the AWS ElasticSearch endpoint.
The AWS credential is loaded automatically following the boto3 priority
(http://boto3.readthedocs.io/en/latest/guide/configuration.html#configuring-credentials)For example:`aws-es-auth-proxy--aws_service_endpointsearch-mydomain-id.eu-west-1.es.amazonaws.com`For security purpose the service starts automatically listening
only in localhost on port 8080.To access to kibana you have to connect with your browser to`http://127.0.0.0.1:8080`
Package: aws-etl-utils
AWS ETL UtilsSet of utilities to constructdata pipeline units in Amazon Web Services.Redshift module: RedshiftETLBuilderMethods to build ETL in Redshift.FeaturesRedshiftETLBuilder currently has these features:Connect to a given redshift clusterPasses parameters to sql scriptsExecute sql scriptsTestUnit tests: python -m unittest discoverVersionThis utility uses bumpversion to update version on '.bumpversion.cfg', 'setup.py' and '.properties' files.bumpversion --current-version 0.1.0 [major, minor, patch]
Package: aws-event-check
AWS Event CheckAnnotation based library to check aws event.Checking event to avoid internal call by lambda function.Supporting AWS Cloud Watch trigger events.Supporting AWS S3 trigger event.Supporting AWS API Gateway event.ExamplesS3 trigger check :Checking for event Records.from aws.event import s3_trigger_event_check


@s3_trigger_event_check
def handler(event, context):
    try:
        pass
    except Exception as e:
        raise eCloud Watch Event :Checking for event id.from aws.event import cloud_watch_trigger_event_check


@cloud_watch_trigger_event_check
def handler(event, context):
    try:
        pass
    except Exception as e:
        raise eAPI Gateway Event :Checking only http method.from aws.event import api_gateway_trigger_event_check


@api_gateway_trigger_event_check
def handler(event, context):
    try:
        pass
    except Exception as e:
        raise eMIT License : Copyright (c) 2019 Abhimanyu Haralukallu