Package: covalent-ec2-plugin
Covalent EC2 Executor PluginCovalent is a Pythonic workflow tool used to execute tasks on advanced computing hardware.This plugin allows tasks to be executed in an AWS EC2 instance (which is auto-created) when you execute your workflow with covalent.1. InstallationTo use this plugin with Covalent, simply install it usingpip:pip install covalent-ec2-plugin2. Usage ExampleThis is a toy example of how a workflow can be adapted to utilize the EC2 Executor. Here we train a Support Vector Machine (SVM) and spin up an EC2 automatically to execute thetrain_svmelectron. We also note we requireDepsPipto install the dependencies on the EC2 instance.fromnumpy.randomimportpermutationfromsklearnimportsvm,datasetsimportcovalentasctdeps_pip=ct.DepsPip(packages=["numpy==1.23.2","scikit-learn==1.1.2"])executor=ct.executor.EC2Executor(instance_type="t2.micro",volume_size=8,#GiBssh_key_file="~/.ssh/id_rsa",key_name="key_name"# EC2 Key Pair)# Use executor plugin to train our SVM model.@ct.electron(executor=executor,deps_pip=deps_pip)deftrain_svm(data,C,gamma):X,y=dataclf=svm.SVC(C=C,gamma=gamma)clf.fit(X[90:],y[90:])returnclf@ct.electrondefload_data():iris=datasets.load_iris()perm=permutation(iris.target.size)iris.data=iris.data[perm]iris.target=iris.target[perm]returniris.data,iris.target@ct.electrondefscore_svm(data,clf):X_test,y_test=datareturnclf.score(X_test[:90],y_test[:90])@ct.latticedefrun_experiment(C=1.0,gamma=0.7):data=load_data()clf=train_svm(data=data,C=C,gamma=gamma)score=score_svm(data=data,clf=clf)returnscore# Dispatch the workflowdispatch_id=ct.dispatch(run_experiment)(C=1.0,gamma=0.7)# Wait for our result and get result valueresult=ct.get_result(dispatch_id=dispatch_id,wait=True).resultprint(result)During the execution of the workflow one can navigate to the UI to see the status of the workflow, once completed however the above script should also output a value with the score of our model.0.86666666666666673. ConfigurationThere are many configuration options that can be passed in to the classct.executor.EC2Executoror by modifying thecovalent config fileunder the section[executors.ec2]For more information about all of the possible configuration values visit ourread the docs (RTD) guidefor this plugin.4. Required AWS ResourcesIn order to run your workflows with covalent there are a few notable resources that need to be provisioned first.For more information regarding which cloud resources need to be provisioned visit ourread the docs (RTD) guidefor this plugin.The required resources include an EC2 Key Pair (which corresponds to thekey_nameconfig value), and optionally a VPC & Subnet that can be used instead of the EC2 executor automatically creating it.Getting Started with CovalentFor more information on how to get started with Covalent, check out the projecthomepageand the officialdocumentation.Release NotesRelease notes for this plugin are available in theChangelog.CitationPlease use the following citation in any publications:W. J. Cunningham, S. K. Radha, F. Hasan, J. Kanem, S. W. Neagle, and S. Sanand.Covalent.Zenodo, 2022.https://doi.org/10.5281/zenodo.5903364LicenseCovalent is licensed under the Apache License 2.0. See theLICENSEfile or contact thesupport teamfor more details.