Saravanan666/pypi-org huggingface dataset:
Package: airless
AirlessAirless is a package that aims to build a serverless and lightweight orchestration platform, creating workflows of multiple tasks being executed onGoogle Cloud FunctionsWhy not just useApache Airflow?Airflow is the industry standard when we talk about job orchestration and worflow management. However, in some cases, we believe it may not be the best solution. I would like to highlight 3 main cases we face that Airflow struggles to handle.ServerlessAt the beginning of a project we want to avoid dealing with infrastructure since it demands time and it has a fixed cost to reserve an instance to run Airflow. Since we didn't have that many jobs, it didn't make sense to have an instance of Airflow up 24-7.When the project starts to get bigger and, if we use Airflow's instance to run the tasks, we start facing performance issues on the workflow.In order to avoid this problems we decided to build a 100% serverless platform.Parallel processingThe main use case we designed Airless for is for data scrappers. The problem with data scrappers is that normally you want them to process a lot of tasks in parallel, for instance, first you want to fetch a website and collect all links in that page and send them forward for another task to be executed and then that task does the same and so on and so forth.Building this workflow that does not know before hand how many tasks are going to be executed is something hard be built on Airflow.Data sharing between tasksIn order to built this massive parallel processing workflow that we explained on the previous topic, we need to be able to dynamically create and send data to the next task. So use the data from the first task as a trigger and an input data for the next tasks.How it worksAirless builts its workflows based onGoogle Cloud Functions,Google Pub/SubandGoogle Cloud Scheduler.Everything starts with the Cloud Scheduler, which is a serverless product from Google Cloud that is able to publish a message to a Pub/Sub with a cron schedulerWhen a message is published to a Pub/Sub it can trigger a Cloud Function and get executed with that message as an inputThis Cloud Functions is able to publish as many messages as it wants to as many Pub/Sub topics as it wantsRepeat from 2PreparationEnvironment variablesENVGCP_PROJECTPUBSUB_TOPIC_ERRORLOG_LEVELPUBSUB_TOPIC_EMAIL_SENDPUBSUB_TOPIC_SLACK_SENDBIGQUERY_DATASET_ERRORBIGQUERY_TABLE_ERROREMAIL_SENDER_ERROREMAIL_RECIPIENTS_ERRORSLACK_CHANNELS_ERROR
Package: airline
airline|Lightweight wide event logging to bring more observability to lambda functions.This is very strongly inspired byhoneycomband theirbeelinelibrary.How?Use the decorators!importairlineairline.init(dataset='your_app_name')@airline.evented()defsome_function(a,b)# do thingsorimportairlinefromairline.awslambdaimportairline_wrapperairline.init(dataset='function_or_app_name')@airline_wrapperdefhandler(event,context):# do thingsWide event logging?The idea is to build up the full context of a function/script into one wide
event that gets emitted at the end.  This puts all the context in one log
message, making it very easy to run analytics on, find like errors, and a lot
of other things.  A full blown observability platform like honeycomb would be
more informative, and allows for the notion of spans and distributed tracing
(i.e. across different (micro) services and the like).But this is a start.Take this:importtimeimportrandomimportairlineairline.init(dataset='example')@airline.evented()defmain(a,b):airline.add_context_field("a",a)airline.add_context_field("b",b)withairline.timer('processing_a'):subfunction1(a)withairline.timer('processing_b'):subfunction1(b)defsubfunction1(input):time.sleep(random.uniform(0,len(input)))main("foo","example_long_thing")And emit this at the end{"time":"2020-03-09T09:49:43.376126Z","dataset":"example","client":"airline/0.1.0","data":{"a":"foo","b":"example_long_thing","duration_ms":9438.041,"timers":{"processing_a_ms":255.11,"processing_b_ms":9182.859}}}
Package: airlock
# airlockAirlock is a lightweight, web-security-concious wrapper forwebapp2on
Google App Engine. It provides oauth2 integration for identity management
with Google Accounts, sessions, and user management.## ComparisonAirlock is a drop-in replacement for severalwebapp2andprotorpcobjects. Specifically, it wrapsremote.Service,webapp2.WSGIApplication,
andwebapp2.RequestHandlerto provide authentication and session features
via oauth2 and theoauth2clientlibrary.original | airlock variant |——– | ————— |protorpc.remote.Service|airlock.Service|webapp2.RequestHandler|airlock.Handler|webapp2.WSGIApplication|airlock.WSGIApplication|## User featuresOauth2 integration with Google Accounts (sign in and sign out).Anonymous user/session support.## Security featuresA standard configuration format for specifying the security characteristics of an application.Provides a framework for setting the following headers:
* Content security policy.
* HSTS policy.
* XSRF.## Usage1. Download client secrets.
1. In appengine config, use airlock.set_config
1. Use airlock’s subclasses.
1. Set up aUsermodel.
Package: airmail
AirmailA CLI tool for deploying projects to AWSIntroduction/PhilosophyThe point of Airmail is to make deploying projects into AWS a little easier. It was inspired as a binding layer betweenTerraformedinfrastructure and deploying applications toAWS ECS. At NYMag we wanted to manage infrastructure with Terraform and then allow applications to be more declarative about how they run without caring about the infrastructure. A developer should be able to change easily declare where and how their application will run and then be able to easily configure resources in Terraform to support that. Airmail is designed to deploy codewith the assumption that the underlying infrastructure is there to support the project.How ToAirmail needs to be run in a project with a.deploydirectory. It will look inside this directory for configuration files that will tell the tool how to deploy to ECS.<project dir>
├── app                     # The directory of your application
├── .deploy                 # The directory holding the config
│   ├── config.yml          # Holds the primary config declarations
│   └── <env>.env           # Environment variable configuration for the container
└── ...Config FileTheconfig.ymlfile contains all the information that Airmail needs to build the service and task definitions to deploy to ECS.For an example file click here.CommandsA list of commands and corresponding arguments/environment variablescan be found here.AWS ConfigurationAirmail assumes your local env is configured perBoto3configuration. The tool uses Boto3 to execute requests to AWS and does not do anything to setup your local environment.Environment VariablesYou can use a few environment variables to control how Airmail is run.AWS_PROFILE: will run the Boto3 commands under the local profile you have configuredAIRMAIL_ENV: automatically chooses which environment to run commands for. Good for CI/CD so the prompt is not triggered.AIRMAIL_DRY_RUN: will run all of the command except the actual call to AWSAIRMAIL_CONFIG_FILE(default:config.yml): specifies the file to read from in the.deploydirectory for application configuration. The file must be a valid YAML file.AIRMAIL_VERBOSE: will log in verbose mode. Good for debugging.Local DevelopmentClone and runpython3 setup.py installor downloadWatchcodeand runwatchcodein the root of the project./
Package: airmailer
AirmailerSend e-mails with either plain SMTP or AWS SES.Free software: Apache Software License 2.0Documentation:https://airmailer.readthedocs.io.InstallationFirst, installairmailer:$pipinstallairmailerDocumentationSeehttps://airmailer.readthedocs.io.History0.1.0 (2021-11-30)First release on PyPI.
Package: airobots
Airobots全端自动化测试框架Airobots，整合了Airtest、RobotFramework、Selenium、Appium、HTTPRunner、Locust框架的方法，从而实现一套框架，能够进行全端的测试，统一执行入口和报告的输出。本质上，Airobots可看成是以上优秀开源测试框架的集成库，整合后以实现方法的共用，比如在Selenium和Appium中使用Airtest的图像识别功能。实现一个框架同时支持Android、IOS、WEB、API的自动化测试及性能测试(HTTPRunner提供的基于Locust的压力生成器)，并且统一了测试报告输出格式，使用美观大气的Allure作为测试报告，可详细记录测试过程日志、测试步骤和截图。安装框架依赖包, 执行pip install airobots -i https://mirrors.aliyun.com/pypi/simplewindows系统下, 如果安装失败, 可能需要安装C++编译工具: visualcppbuildtools_full.exe, 具体错误请留意控制台报错信息。运行WEB测试，需要安装ChromeDriver，请自行下载安装，或安装node之后执行npm install -g chromedriver安装执行测试Allure 报告(推荐)airobots -t api ./API/Case/Path/ --alluredir=Results             # API测试
airobots -t web ./Web/Case/Path/ --alluredir=Results             # Web测试
airobots -t android ./Android/Case/Path/ --alluredir=Results     # Android测试
airobots -t ios ./IOS/Case/Path/ --alluredir=Results             # IOS测试HTML 报告airobots -t api ./API/Case/Path/ --html=Results/report.html          # API测试
airobots -t web ./Web/Case/Path/ --html=Results/report.html          # Web测试
airobots -t android ./Android/Case/Path/ --html=Results/report.html  # Android测试
airobots -t ios ./IOS/Case/Path/ --html=Results/report.html          # IOS测试查看Allure报告allure serve ./Results安装AllureLinuxsudo apt-add-repository ppa:qameta/allure
sudo apt-get update 
sudo apt-get install allureMac OS X对于Mas OS，可通过Homebrew进行自动安装brew install allureWindows对于Windows，可从Scoop命令行安装程序获得Allure。要安装Allure，请下载并安装Scoop，然后在Powershell中执行scoop install allure演示项目:https://github.com/BSTester/AirobotsDemo
Package: airo-models
airo-modelsCurated URDFs and 3D models of the robots and gripper used at airo.Installationairo-modelsis available on PyPi and can be installed with pip:pip install airo-modelsUsageExample of loading a URDF from airo-models, customizing it and writing it to a temporary file:importairo_modelsrobotiq_urdf_path=airo_models.get_urdf_path("robotiq_2f_85")robotiq_urdf=airo_models.urdf.read_urdf(robotiq_urdf_path)# Make the robotiq gripper staticairo_models.urdf.replace_value(robotiq_urdf,"@type","revolute","fixed")airo_models.urdf.delete_key(robotiq_urdf,"mimic")airo_models.urdf.delete_key(robotiq_urdf,"transmission")# Write it to a temporary file to read later with Drake's AddModelFromFilerobotiq_static_urdf_path=airo_models.urdf.write_urdf_to_tempfile(robotiq_urdf,robotiq_urdf_path,prefix="robotiq_2f_85_static_")To check which models are available:fromairo_models.filesimportAIRO_MODEL_NAMESprint(AIRO_MODEL_NAMES)>>>['ur3e','ur5e','robotiq_2f_85']DevelopmentLocal installationClone this repoCreate the conda environmentconda env create -f environment.yamlInitialize the pre-commit hookspre-commit installRun the tests withpytest .ReleasingReleasing to PyPi is done automatically by github actions when a new tag is pushed to the main branch.Update the version inpyproject.toml.git add pyproject.tomlgit commit -m ""git pushgit tag -a v0.1.0 -m "airo-models v0.1.0"git push origin v0.1.0This was set up followingthis guidefirst and thenthis guide.
Package: aironsuit
AIronSuitAIronSuit (Beta) is a Python library for automatic model design/selection and visualization purposes built to work withtensorflowas a backend. It aims to accelerate
the development of deep learning approaches for research/development purposes by providing components relying on cutting
edge approaches. It is flexible and its components can be
replaced by customized ones from the user. The user mostly focuses on defining the input and output,
and AIronSuit takes care of its optimal mapping.Key features:Automatic model design/selection withhyperopt.Parallel computing for multiple models across multiple GPUs when using a k-fold approach.Built-in model trainer that saves training progression to be visualized withTensorBoard.Machine learning tools fromAIronTools:model_constructor,block_constructor,layer_constructor, preprocessing utils, etc.Flexibility: the user can replace AIronSuit components by a customized one. For instance,
the model constructor can be easily replaced by a customized one.Installationpip install aironsuitExample# Databricks notebook sourceimportnumpyasnpfromhyperopt.hpimportchoicefromhyperoptimportTrialsfromtensorflow.keras.datasetsimportmnistfromtensorflow.keras.optimizersimportAdamimportosfromaironsuit.suitimportAIronSuitfromairontools.preprocessingimporttrain_val_splitfromairontools.constructors.models.unsupervisedimportImageVAEfromairontools.toolsimportpath_managementHOME=os.path.expanduser("~")OS_SEP=os.path.sep# COMMAND ----------# Example Set-Up #model_name='VAE_NN'working_path=os.path.join(HOME,'airon',model_name)+OS_SEPnum_classes=10batch_size=128epochs=30patience=3max_evals=3max_n_samples=Noneprecision='float32'# COMMAND ----------# Make/remove pathspath_management(working_path,modes=['rm','make'])# COMMAND ----------# Load and preprocess data(train_dataset,target_dataset),_=mnist.load_data()ifmax_n_samplesisnotNone:train_dataset=train_dataset[-max_n_samples:,...]target_dataset=target_dataset[-max_n_samples:,...]train_dataset=np.expand_dims(train_dataset,-1)/255# Split data per parallel modelx_train,x_val,_,meta_val,_=train_val_split(input_data=train_dataset,meta_data=target_dataset)# COMMAND ----------# VAE Model constructordefvae_model_constructor(latent_dim):# Create VAE model and compile itvae=ImageVAE(latent_dim)vae.compile(optimizer=Adam())returnvae# COMMAND ----------# Hyper-parameter spacehyperparam_space={'latent_dim':choice('latent_dim',np.arange(3,6))}# COMMAND ----------# Invoke AIronSuitaironsuit=AIronSuit(model_constructor=vae_model_constructor,force_subclass_weights_saver=True,force_subclass_weights_loader=True,path=working_path)# COMMAND ----------# Automatic Model Designprint('\n')print('Automatic Model Design\n')aironsuit.design(x_train=x_train,x_val=x_val,hyper_space=hyperparam_space,max_evals=max_evals,epochs=epochs,trials=Trials(),name=model_name,seed=0,patience=patience)aironsuit.summary()delx_train# COMMAND ----------# Get latent insightsaironsuit.visualize_representations(x_val,metadata=meta_val,hidden_layer_name='z',)More Examplessee usage examples inaironsuit/examples
Package: airontools
AIronToolsAIronTools (Beta) is a Python library that provides the user with higher level state-of-the-art deep learning tools built to work withtensorflowas a backend. The main goal of this repository is to enable fast model
design for both POCs and production.Key features:Out-of-the-box models ready to be used.Block constructor to build customised blocks/models.Layer constructor to build customised layers such as sequential, convolutional, self-attention or dense, and combinations of them.Preprocessing tools.On the fly non-topological hyper-parameter optimization. For now only the dropout regularization is compatible with this feature, in the future others such as l1 and l2 regularization will be compatible too.Latent representations for visualization purposes.Installationpip install airontoolsCustom Keras subclass to build a variational autoencoder (VAE) with airontools and compatible withaironsuitimportnumpyasnpimporttensorflowastffromairontools.constructors.models.unsupervised.vaeimportVAEfromnumpy.randomimportnormaltabular_data=np.concatenate([normal(loc=0.5,scale=1,size=(100,10)),normal(loc=-0.5,scale=1,size=(100,10)),])model=VAE(input_shape=tabular_data.shape[1:],latent_dim=3,)model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001))model.fit(tabular_data,epochs=10,)print("VAE evaluation:",float(model.evaluate(tabular_data)["loss"]))More examplessee usage examples inaironsuit/examples
Package: airoscriptng
Airoscript-ng python complete implementationFree software: GNU GENERAL PUBLIC LICENSE 2Documentation:http://airoscript-ng.readthedocs.org/en/masterFeaturesDynamic aircrack-ng API generation (under airoscriptng.aircrack)Threaded executionHackability assesmentScanning provides a list of best wireless hacking techniquesSession control (for better process control)Wireless monitor interfaces are nicely handled and reused if neccesaryXMLRPC server implementationTODOBetter parameter parsing & format for aircrack-ng parameters file, read them from manpagesImplement attacks on airoscript-ng classImplement cracking (and control of it, once cracked stop all attacks against that network)Build a user interface (probably more than one)History0.0.2 (2015-08-01)First usable thing, still no attacks“Hackability” property for apsIntegrated clients on AP objectExternal plugin supportReaver supportXMLRPC WorkingGeneral cleanup0.0.1 (2014-12-26)First release, monitor mode and scanning working
Package: airouter
AI Routerpip install airouterimportairouteroutputs=airouter.StreamedCompletion.create(model='gpt-3.5-turbo-1106',temperature=0.0,max_tokens=100,messages=[{'role':'user','content':'write a story about the big Large Language Models convention'}])importairouteroutputs=airouter.StreamedCompletion.create(model='text-bison-32k',temperature=0.0,max_tokens=100,messages=[{'role':'user','content':'write a story about the big Large Language Models convention'}])importairouteroutputs=airouter.StreamedCompletion.create(models=['gpt-3.5-turbo-1106','text-bison-32k'],temperature=0.0,max_tokens=100,messages=[{'role':'user','content':'write a story about the big Large Language Models convention'}])importairouteroutputs=airouter.StreamedCompletion.create_interactive(models=['gpt-3.5-turbo-1106','text-bison-32k'],temperature=0.0,max_tokens=100,messages=[{'role':'user','content':'write a story about the big Large Language Models convention'}])
Package: airpg
airpg: Automatically accessing the inverted repeats of archived plastid genomesA Python package for automatically accessing the inverted repeats of thousands of plastid genomes stored on NCBI NucleotideINSTALLATIONTo get the most recent stable version ofairpg, run:pip install airpgOr, alternatively, if you want to get the latest development version ofairpg, run:pip install git+https://github.com/michaelgruenstaeudl/airpg.gitEXAMPLE USAGETutorial 1: Very short survey (runtime ca. 5 min.; for the impatient)Survey of all plastid genomes of flowering plants submitted to NCBI Nucleotide within the past 10 days.Tutorial 2: Short survey (runtime ca. 15 min.; for testing)Survey of all plastid genomes of flowering plants submitted to NCBI Nucleotide within the current month.Tutorial 3: Medium survey (runtime ca. 5 hours)Survey of all plastid genomes of flowering plants submitted to NCBI Nucleotide in 2019 only. Note: The results of this survey are available on Zenodo via DOI10.5281/zenodo.4335906Tutorial 4: Full survey (runtime ca. 19 hours; with explanations)Survey of all plastid genomes of flowering plants submitted to NCBI Nucleotide from January 2000 until, and including, December 2020. Note: The results of this survey are available on Zenodo via DOI10.5281/zenodo.4335906CHANGELOGSeeCHANGELOG.mdfor a list of recent changes to the software.
Package: airphin
AirphinAirphin is a tool for migrating Airflow DAGs to DolphinScheduler Python API.InstallationFor now, it just for test and without publish to pypi but will be adding in the future.
You could still install locally by yourself.python-mpipinstall--upgradeairphinQuick StartHere will give a quick example to show how to migrate base on standard input.# Quick test the migrate rule for standard input# Can also add option `--diff` to see the diff detail of this migrateairphintest"from airflow.operators.bash import BashOperatortest = BashOperator(task_id='test',bash_command='echo 1',)"And you will see the migrated result in the standard output. Airphin can only migrate standard input, it can
also migrate file, directory and even can use in your python code. For more detail, please seeour usage.DocumentationThe documentation host on read the doc and is available athttps://airphin.readthedocs.io.Support StatementFor now, we support following statement from Airflow's DAG filesDAGBefore MigrationAfter Migrationfrom airflow import DAGfrom pydolphinscheduler.core.process_definition import ProcessDefinitionwith DAG(...) as dag: passwith ProcessDefinition(...) as dag: passOperatorsDummy OperatorBefore MigrationAfter Migrationfrom airflow.operators.dummy_operator import DummyOperatorfrom pydolphinscheduler.tasks.shell import Shellfrom airflow.operators.dummy import DummyOperatorfrom pydolphinscheduler.tasks.shell import Shelldummy = DummyOperator(...)dummy = Shell(..., command="echo 'airflow dummy operator'")Shell OperatorBefore MigrationAfter Migrationfrom airflow.operators.bash import BashOperatorfrom pydolphinscheduler.tasks.shell import Shellbash = BashOperator(...)bash = Shell(...)Spark Sql OperatorBefore MigrationAfter Migrationfrom airflow.operators.spark_sql_operator import SparkSqlOperatorfrom pydolphinscheduler.tasks.sql import Sqlspark = SparkSqlOperator(...)spark = Sql(...)Python OperatorBefore MigrationAfter Migrationfrom airflow.operators.python_operator import PythonOperatorfrom pydolphinscheduler.tasks.python import Pythonpython = PythonOperator(...)python = Python(...)
Package: airpiano
What is airpiano?takefuji (2022) airpiano for playing piano in the air [Source Code].https://doi.org/10.24433/CO.8862673.v1This is under review.airpiano has been downloaded by 23337 downloads worldwide as of Feb.12, 2024.When the paper is accepted, the contents of the air piano will be disclosed.
airpiano is a Python program to play airpiano composed of 123 lines code.
You can play 10 music notes by th right hand of five fingers.
airpiano is based on the-state-of-the-art hand gesture recognition using mediapipe.
airpiano uses musicalbeeps for generating music sounds by five fingers.
The paper on airpiano is under submission.
If the paper is accepted, the source code will be disclosed.Hand landmarks detected by MediaPipe are composed of 21 points in 2D image coordinates.https://mediapipe.dev/images/mobile/hand_landmarks.pngairpiano DEMO"twinkle twinkle little star"https://youtu.be/7KCmVyRpSCo"Mary had a little lamb"https://youtu.be/3J5xjW66MP8"Tulip"https://youtu.be/H7lwHb69ZgM"buzz buzz buzz"https://youtu.be/oG3_ZsW3XrQ"The Cuckoo"https://youtu.be/evyfpph-WOYHow to install airpianoPython3.7 or Python3.8 is recommended for running airpiano.
In order to run airpiano, the following libraries must be installed.$ pip install musicalbeeps$ pip install mediapipe$ pip install airpiano$ pip install airpiano --force-reinstall --no-cache-dir --no-binary :all:How to play airpianoMoving thumb finger plays C5 note.Moving index finger plays D5 note.Middle finger plays E5 note.Ring finger plays F5 note.Little finger plays G5 note.Moving thumb finger deeply plays A5 note.Moving thumb finger deeply and index finger plays B5 note.Moving thumb finger deeply and middle finger plays C6 note.Moving thumb finger deeply and ring finger plays D6 note.Moving thumb finger deeply and little finger plays E6 note.$ airpiano
Package: air-pollution-check
air_pollution_checkA package for looking into the air pollution data, which can transfer the data into dataframe, add in the overall air pollution to the dataframe, visualize the air pollution data, and output the pollution level comparison from the past to the future for a specific location.Installation$pipinstallair_pollution_checkUsageTODOContributingInterested in contributing? Check out the contributing guidelines. Please note that this project is released with a Code of Conduct. By contributing to this project, you agree to abide by its terms.Licenseair_pollution_checkwas created by Yazhao Huang. It is licensed under the terms of the MIT license.Creditsair_pollution_checkwas created withcookiecutterand thepy-pkgs-cookiecuttertemplate.
Package: airportAI-simulator
Airport simulator for evaluating airport gate assignment problem. Part of ariportAI project.PrerequisitesPython 3 is required to run this application. Cuda toolkit must be installed. If you are using Conda type:$ conda install cudatoolkitRefer tonumba requirements.InstallationInstalling from git:$ git clone https://github.com/burnpiro/airport-ai.git
$ cd airport-ai/simulator
$ pip install .Installing from pypi:$ pip install airportAI-simulatorUsageImportSimulatormodule and createSimulationobject.fromSimulatorimportSimulationsim=Simulation()# stepping simulationsim.step()# get agents and theier propertiesagents=sim.agentspos=agents[0].get_pos()# get flight scheduleflights=sim.get_flights()# get gates statusgates=sim.get_gates()