Package: sagify
LLMs and Machine Learning done easily.sagifyA command-line utility to train and deploy LLMs and Machine Learning/Deep Learning models onAWS SageMakerin a few simple steps! It hides all the details of Sagemaker so that you can focus 100% on Machine Learning, and not in low level engineering tasks.For detailed reference to Sagify commands please go to:Read the DocsInstallationPrerequisitessagify requires the following:Python (3.7, 3.8, 3.9, 3.10, 3.11)Dockerinstalled and runningConfiguredawscliInstall sagifyAt the command line:pip install sagifyGetting started -  LLM Deployment with no codeMake sure to configure your AWS account by following the instructions at sectionConfigure AWS AccountFinally, run the following command:sagifycloudfoundation-model-deploy--model-idmodel-txt2img-stabilityai-stable-diffusion-v2-1-base--model-version1.*-n1-eml.p3.2xlarge--aws-regionus-east-1--aws-profilesagemaker-devYou can change the values for ec2 type (-e), aws region and aws profile with your preferred ones.Once the Stable Diffusion model is deployed, you can use the generated code snippet to query it. Enjoy!Getting started -  No code deploymentCreate a file with namehuggingface_config.jsonwith the following content:{
   "transformers_version": "4.6.1",
   "pytorch_version": "1.7.1",
   "hub": {
     "HF_MODEL_ID": "gpt2",
     "HF_TASK": "text-generation"
   }
 }Then, make sure to configure your AWS account by following the instructions at sectionConfigure AWS AccountFinally, run the following command:sagify cloud lightning-deploy --framework huggingface -n 1 -e ml.c4.2xlarge --extra-config-file huggingface_config.json --aws-region us-east-1 --aws-profile sagemaker-devYou can change the values for ec2 type (-e), aws region and aws profile with your preferred ones.Once the Hugging Face model is deployed, you can go tohttps://console.aws.amazon.com/sagemaker/home?region=us-east-1#/endpoints(make sure you're on your preferred region) and find your deployed endpoint. For example:Then, you can click on your deployed endpoint and copy the endpoint url. For example:Postman is a good app to call the deployed endpoint. Here's an example on how to set up the AWS signature in order to call the endpoint:Finally, you can call the endpoint from Postman:Getting started - Custom Training and DeploymentStep 1: Clone Machine Learning demo repositoryYou're going to clone and train a Machine Learning codebase to train a classifier for the Iris data set.Clone repository:git clone https://github.com/Kenza-AI/sagify-demo.gitCreate environment:mkvirtualenv -p python3.7 sagify-demoormkvirtualenv -p python3.8 sagify-demoDon't forget to activate the virtualenv after the creation of environment by executingworkon sagify-demo.Install dependencies:make requirementsStep 2: Initialize sagifysagify initType insagify-demofor SageMaker app name,Nin questionAre you starting a new project?,srcfor questionType in the directory where your code livesand make sure to choose your preferred Python version, AWS profile and region. Finally, typerequirements.txtin questionType in the path to requirements.txt.A module calledsagifyis created under the directory you provided. The structure is:sagify_base/
    local_test/
        test_dir/
            input/
                config/
                    hyperparameters.json
                data/
                    training/
            model/
            output/
        deploy_local.sh
        train_local.sh
    prediction/
        __init__.py
        nginx.conf
        predict.py
        prediction.py
        predictor.py
        serve
        wsgi.py
    training/
        __init__.py
        train
        training.py
    __init__.py
    build.sh
    Dockerfile
    executor.sh
    push.shStep 3: Integrate sagifyAs a Data Scientist, you only need to conduct a few actions. Sagify takes care of the rest:Copy a subset of training data undersagify_base/local_test/test_dir/input/data/training/to test that training works locallyImplementtrain(...)function insagify_base/training/training.pyImplementpredict(...)function insagify_base/prediction/prediction.pyOptionally, specify hyperparameters insagify_base/local_test/test_dir/input/config/hyperparameters.jsonHence,Copyiris.datafiles fromdatatosagify_base/local_test/test_dir/input/data/training/Replace theTODOsin thetrain(...)function insagify_base/training/training.pyfile with:input_file_path=os.path.join(input_data_path,'iris.data')clf,accuracy=training_logic(input_file_path=input_file_path)output_model_file_path=os.path.join(model_save_path,'model.pkl')joblib.dump(clf,output_model_file_path)accuracy_report_file_path=os.path.join(model_save_path,'report.txt')withopen(accuracy_report_file_path,'w')as_out:_out.write(str(accuracy))and at the top of the file, add:importosimportjoblibfromiris_trainingimporttrainastraining_logicReplace the body ofpredict(...)function insagify_base/prediction/prediction.pywith:model_input=json_input['features']prediction=ModelService.predict(model_input)return{"prediction":prediction.item()}and replace the body ofget_model()function inModelServiceclass in the same file with:ifcls.modelisNone:importjoblibcls.model=joblib.load(os.path.join(_MODEL_PATH,'model.pkl'))returncls.modelStep 4: Build Docker imageIt's time to build the Docker image that will contain the Machine Learning codebase:sagify buildIf you rundocker images | grep sagify-demoin your terminal, you'll see the created Sagify-Demo image.Step 5: Train modelTime to train the model for the Iris data set in the newly built Docker image:sagify local trainModel filemodel.pkland report filereport.txtare now undersagify_base/local_test/test_dir/model/Step 6: Deploy modelFinally, serve the model as a REST Service:sagify local deployRun the following curl command on your terminal to verify that the REST Service works:```bash
curl -X POST \
http://localhost:8080/invocations \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-H 'Postman-Token: 41189b9a-40e2-abcf-b981-c31ae692072e' \
-d '{
    "features":[[0.34, 0.45, 0.45, 0.3]]
}'
```It will be slow in the first couple of calls as it loads the model in a lazy manner.Voila! That's a proof that this Machine Learning model is going to be trained and deployed on AWS SageMaker successfully. Now, go to theUsagesection inSagify Docsto see how to train and deploy this Machine Learning model to AWS SageMaker!Hyperparameter OptimizationGiven that you have configured your AWS Account as described in the previous section, you're now ready to perform Bayesian Hyperparameter Optimization on AWS SageMaker! The process is similar to training step.Step 1: Define Hyperparameter Configuration FileDefine the Hyperparameter Configuration File. More specifically, you need to specify in a local JSON file the ranges for the hyperparameters, the name of the objective metric and its type (i.e.MaximizeorMinimize). For example:{"ParameterRanges":{"CategoricalParameterRanges":[{"Name":"kernel","Values":["linear","rbf"]}],"ContinuousParameterRanges":[{"MinValue":0.001,"MaxValue":10,"Name":"gamma"}],"IntegerParameterRanges":[{"Name":"C","MinValue":1,"MaxValue":10}]},"ObjectiveMetric":{"Name":"Precision","Type":"Maximize"}}Step 2: Implement Train functionReplace theTODOsin thetrain(...)function insagify_base/training/training.pyfile with your logic. For example:fromsklearnimportdatasetsiris=datasets.load_iris()# Read the hyperparameter config json fileimportjsonwithopen(hyperparams_path)as_in_file:hyperparams_dict=json.load(_in_file)fromsklearnimportsvmclf=svm.SVC(gamma=float(hyperparams_dict['gamma']),# Values will be read as strings, so make sure to convert them to the right data typeC=float(hyperparams_dict['C']),kernel=hyperparams_dict['kernel'])fromsklearn.model_selectionimporttrain_test_splitX_train,X_test,y_train,y_test=train_test_split(iris.data,iris.target,test_size=0.3,random_state=42)clf.fit(X_train,y_train)fromsklearn.metricsimportprecision_scorepredictions=clf.predict(X_test)precision=precision_score(y_test,predictions,average='weighted')# Log the objective metric name with its calculated value. In tis example is Precision.# The objective name should be exactly the same with the one specified in the hyperparams congig json file.# The value must be a numeric (float or int).fromsagify.api.hyperparameter_tuningimportlog_metricname="Precision"log_metric(name,precision)fromjoblibimportdumpdump(clf,os.path.join(model_save_path,'model.pkl'))print('Training complete.')Step 3: Build and Push Docker imagesagify buildMake sure sagify is in yourrequirements.txtfile.sagify pushStep 4: Call The CLI CommandAnd, finally, call the hyperparameter-optimization CLI command. For example:sagify cloud hyperparameter-optimization -i s3://my-bucket/training-data/ -o s3://my-bucket/output/ -e ml.m4.xlarge -h local/path/to/hyperparam_ranges.jsonStep 5: Monitor ProgressYou can monitor the progress via the SageMaker UI console. Here is an example of a finished Hyperparameter Optimization job:CommandsInitializeNameInitializes a sagify moduleSynopsissagify initDescriptionThis command initializes a sagify module in the directory you provide when asked after you invoke theinitcommand.Examplesagify initConfigureDescriptionUpdates an existing configuration value e.g.python versionorAWS region.Synopsissagify configure [--aws-region AWS_REGION] [--aws-profile AWS_PROFILE] [--image-name IMAGE_NAME] [--python-version PYTHON_VERSION]Optional Flags--aws-region AWS_REGION:AWSregion whereDockerimages are pushed andSageMakeroperations (train,deploy) are performed.--aws-profile AWS_PROFILE:AWSprofile to use when interacting withAWS.--image-name IMAGE_NAME:Dockerimage name used when building for use withSageMaker. This shows up as anAWS ECRrepository on yourAWSaccount.--python-version PYTHON_VERSION:Pythonversion used when buildingSageMaker'sDockerimages. Currently supported versions:3.6.Examplesagify configure --aws-region us-east-2 --aws-profile default --image-name sage-docker-image-name --python-version 3.6BuildNameBuilds a Docker imageSynopsissagify buildDescriptionThis command builds a Docker image from code under the directory sagify is installed in. AREQUIREMENTS_FILEneeds to be specified duringsagify initor later viasagify configure --requirements-dirfor all required dependencies to be installed in the Docker image.Examplesagify buildLocal TrainNameExecutes a Docker image in train modeSynopsissagify local trainDescriptionThis command executes a Docker image in train mode. More specifically, it executes thetrain(...)function insagify_base/training/training.pyinside an already built Docker image (see Build command section).Examplesagify local trainLocal DeployNameExecutes a Docker image in serve modeSynopsissagify local deployDescriptionThis command executes a Docker image in serve mode. More specifically, it runs a Flask REST app in Docker image and directs HTTP requests to/invocationsendpoint. Then, the/invocationsendpoint calls thepredict(...)function insagify_base/prediction/prediction.py(see Build command section on how to build a Docker image).Examplesagify local deployPushNamePushes a Docker image to AWS Elastic Container ServiceSynopsissagify push [--aws-profile PROFILE_NAME] [--aws-region AWS_REGION] [--iam-role-arn IAM_ROLE] [--external-id EXTERNAL_ID]DescriptionThis command pushes an already built Docker image to AWS Elastic Container Service. Later on, AWS SageMaker will consume that image from AWS Elastic Container Service for train and serve mode.Only one ofiam-role-arnandaws_profilecan be provided.external-idis ignored when noiam-role-arnis provided.Optional Flags--iam-role-arn IAM_ROLEor-i IAM_ROLE: AWS IAM role to use for pushing to ECR--aws-region AWS_REGIONor-r AWS_REGION: The AWS region to push the image to--aws-profile PROFILE_NAMEor-p PROFILE_NAME: AWS profile to use for pushing to ECR--external-id EXTERNAL_IDor-e EXTERNAL_ID: Optional external id used when using an IAM roleExamplesagify pushCloud Upload DataNameUploads data to AWS S3Synopsissagify cloud upload-data --input-dir LOCAL_INPUT_DATA_DIR --s3-dir S3_TARGET_DATA_LOCATIONDescriptionThis command uploads content underLOCAL_INPUT_DATA_DIRto S3 underS3_TARGET_DATA_LOCATIONRequired Flags--input-dir LOCAL_INPUT_DATA_DIRor-i LOCAL_INPUT_DATA_DIR: Local input directory--s3-dir S3_TARGET_DATA_LOCATIONor-s S3_TARGET_DATA_LOCATION: S3 target locationExamplesagify cloud upload-data -i ./training_data/ -s s3://my-bucket/training-data/Cloud TrainNameTrains your ML/DL model using a Docker image on AWS SageMaker with input from S3Synopsissagify cloud train --input-s3-dir INPUT_DATA_S3_LOCATION --output-s3-dir S3_LOCATION_TO_SAVE_OUTPUT --ec2-type EC2_TYPE [--hyperparams-file HYPERPARAMS_JSON_FILE] [--volume-size EBS_SIZE_IN_GB] [--time-out TIME_OUT_IN_SECS] [--aws-tags TAGS] [--iam-role-arn IAM_ROLE] [--external-id EXTERNAL_ID] [--base-job-name BASE_JOB_NAME] [--job-name JOB_NAME] [--metric-names COMMA_SEPARATED_METRIC_NAMES] [--use-spot-instances FLAG_TO_USE_SPOT_INSTANCES]DescriptionThis command retrieves a Docker image from AWS Elastic Container Service and executes it on AWS SageMaker in train modeRequired Flags--input-s3-dir INPUT_DATA_S3_LOCATIONor-i INPUT_DATA_S3_LOCATION: S3 location to input data--output-s3-dir S3_LOCATION_TO_SAVE_OUTPUTor-o S3_LOCATION_TO_SAVE_OUTPUT: S3 location to save output (models, reports, etc). Make sure that the output bucket already exists. Any not existing key prefix will be created by sagify.--ec2-type EC2_TYPEor-e EC2_TYPE: ec2 type. Refer tohttps://aws.amazon.com/sagemaker/pricing/instance-types/Optional Flags--hyperparams-file HYPERPARAMS_JSON_FILEor-h HYPERPARAMS_JSON_FILE: Path to hyperparams JSON file--volume-size EBS_SIZE_IN_GBor-v EBS_SIZE_IN_GB: Size in GB of the EBS volume (default: 30)--time-out TIME_OUT_IN_SECSor-s TIME_OUT_IN_SECS: Time-out in seconds (default: 24 * 60 * 60)--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for training withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role--base-job-name BASE_JOB_NAMEor-n BASE_JOB_NAME: Optional prefix for the SageMaker training job--job-name JOB_NAME: Optional name for the SageMaker training job. NOTE: if a--base-job-nameis passed along with this option, it will be ignored.--use-spot-instances FLAG_TO_USE_SPOT_INSTANCES: Optional flag that specifies whether to use SageMaker Managed Spot instances for training. It should be used only for training jobs that take less than 1 hour. More information:https://docs.aws.amazon.com/sagemaker/latest/dg/model-managed-spot-training.html(default: False).--metric-names COMMA_SEPARATED_METRIC_NAMES: Optional comma-separated metric names for tracking performance of training jobs. Example:Precision,Recall,AUC. Then, make sure you log these metric values using thelog_metricfunction in thetrainfunction:...fromsagify.api.hyperparameter_tuningimportlog_metriclog_metric("Precision:, precision)log_metric("Accuracy",accuracy)...When the training jobs finishes, they will be stored in the CloudWatch algorithm metrics logs of the SageMaker training job:Examplesagify cloud train -i s3://my-bucket/training-data/ -o s3://my-bucket/output/ -e ml.m4.xlarge -h local/path/to/hyperparams.json -v 60 -t 86400 --metric-names Accuracy,PrecisionCloud Hyperparameter OptimizationNameExecutes a Docker image in hyperparameter-optimization mode on AWS SageMakerSynopsissagify cloud hyperparameter-optimization --input-s3-dir INPUT_DATA_S3_LOCATION --output-s3-dir S3_LOCATION_TO_SAVE_MULTIPLE_TRAINED_MODELS --ec2-type EC2_TYPE [--hyperparams-config-file HYPERPARAM_RANGES_JSON_FILE] [--max-jobs MAX_NUMBER_OF_TRAINING_JOBS] [--max-parallel-jobs MAX_NUMBER_OF_PARALLEL_TRAINING_JOBS] [--volume-size EBS_SIZE_IN_GB] [--time-out TIME_OUT_IN_SECS] [--aws-tags TAGS] [--iam-role-arn IAM_ROLE] [--external-id EXTERNAL_ID] [--base-job-name BASE_JOB_NAME] [--job-name JOB_NAME] [--wait WAIT_UNTIL_HYPERPARAM_JOB_IS_FINISHED] [--use-spot-instances FLAG_TO_USE_SPOT_INSTANCES]DescriptionThis command retrieves a Docker image from AWS Elastic Container Service and executes it on AWS SageMaker in hyperparameter-optimization modeRequired Flags--input-s3-dir INPUT_DATA_S3_LOCATIONor-i INPUT_DATA_S3_LOCATION: S3 location to input data--output-s3-dir S3_LOCATION_TO_SAVE_OUTPUTor-o S3_LOCATION_TO_SAVE_OUTPUT: S3 location to save output (models, reports, etc). Make sure that the output bucket already exists. Any not existing key prefix will be created by sagify.--ec2-type EC2_TYPEor-e EC2_TYPE: ec2 type. Refer tohttps://aws.amazon.com/sagemaker/pricing/instance-types/--hyperparams-config-file HYPERPARAM_RANGES_JSON_FILEor-h HYPERPARAM_RANGES_JSON_FILE: Local path to hyperparameters configuration file. Example:{"ParameterRanges":{"CategoricalParameterRanges":[{"Name":"kernel","Values":["linear","rbf"]}],"ContinuousParameterRanges":[{"MinValue":0.001,"MaxValue":10,"Name":"gamma"}],"IntegerParameterRanges":[{"Name":"C","MinValue":1,"MaxValue":10}]},"ObjectiveMetric":{"Name":"Precision","Type":"Maximize"}}Optional Flags--max-jobs MAX_NUMBER_OF_TRAINING_JOBSor-m MAX_NUMBER_OF_TRAINING_JOBS: Maximum total number of training jobs to start for the hyperparameter tuning job (default: 3)--max-parallel-jobs MAX_NUMBER_OF_PARALLEL_TRAINING_JOBSor-p MAX_NUMBER_OF_PARALLEL_TRAINING_JOBS: Maximum number of parallel training jobs to start (default: 1)--volume-size EBS_SIZE_IN_GBor-v EBS_SIZE_IN_GB: Size in GB of the EBS volume (default: 30)--time-out TIME_OUT_IN_SECSor-s TIME_OUT_IN_SECS: Time-out in seconds (default: 24 * 60 * 60)--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for training withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role--base-job-name BASE_JOB_NAMEor-n BASE_JOB_NAME: Optional prefix for the SageMaker training job--job-name JOB_NAME: Optional name for the SageMaker training job. NOTE: if a--base-job-nameis passed along with this option, it will be ignored.--wait WAIT_UNTIL_HYPERPARAM_JOB_IS_FINISHEDor-w WAIT_UNTIL_HYPERPARAM_JOB_IS_FINISHED: Optional flag to wait until Hyperparameter Tuning is finished. (default: don't wait)--use-spot-instances FLAG_TO_USE_SPOT_INSTANCES: Optional flag that specifies whether to use SageMaker Managed Spot instances for training. It should be used only for training jobs that take less than 1 hour. More information:https://docs.aws.amazon.com/sagemaker/latest/dg/model-managed-spot-training.html(default: False).Examplesagify cloud hyperparameter-optimization -i s3://my-bucket/training-data/ -o s3://my-bucket/output/ -e ml.m4.xlarge -h local/path/to/hyperparam_ranges.json -v 60 -t 86400Cloud DeployNameExecutes a Docker image in serve mode on AWS SageMakerSynopsissagify cloud deploy --s3-model-location S3_LOCATION_TO_MODEL_TAR_GZ --num-instance NUMBER_OF_EC2_INSTANCES --ec2-type EC2_TYPE [--aws-tags TAGS] [--iam-role-arn IAM_ROLE] [--external-id EXTERNAL_ID] [--endpoint-name ENDPOINT_NAME]DescriptionThis command retrieves a Docker image from AWS Elastic Container Service and executes it on AWS SageMaker in serve mode. You can update an endpoint (model or number of instances) by specifying the endpoint-name.Required Flags--s3-model-location S3_LOCATION_TO_MODEL_TAR_GZor-m S3_LOCATION_TO_MODEL_TAR_GZ: S3 location to to model tar.gz--num-instances NUMBER_OF_EC2_INSTANCESorn NUMBER_OF_EC2_INSTANCES: Number of ec2 instances--ec2-type EC2_TYPEore EC2_TYPE: ec2 type. Refer tohttps://aws.amazon.com/sagemaker/pricing/instance-types/Optional Flags--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for deploying withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role--endpoint-name ENDPOINT_NAME: Optional name for the SageMaker endpointExamplesagify cloud deploy -m s3://my-bucket/output/model.tar.gz -n 3 -e ml.m4.xlargeCloud Batch TransformNameExecutes a Docker image in batch transform mode on AWS SageMaker, i.e. runs batch predictions on user defined S3 dataSynopsissagify cloud batch-transform --s3-model-location S3_LOCATION_TO_MODEL_TAR_GZ --s3-input-location S3_INPUT_LOCATION --s3-output-location S3_OUTPUT_LOCATION --num-instance NUMBER_OF_EC2_INSTANCES --ec2-type EC2_TYPE [--aws-tags TAGS] [--iam-role-arn IAM_ROLE] [--external-id EXTERNAL_ID] [--wait WAIT_UNTIL_BATCH_TRANSFORM_JOB_IS_FINISHED] [--job-name JOB_NAME]DescriptionThis command retrieves a Docker image from AWS Elastic Container Service and executes it on AWS SageMaker in batch transform mode, i.e. runs batch predictions on user defined S3 data. SageMaker will spin up REST container(s) and call it/them with input data(features) from a user defined S3 path.Things to do:You should implement the predict function that expects a JSON containing the required feature values. It's the same predict function used for deploying the model as a REST service. Example of a JSON:{"features":[5.1,3.5,1.4,0.2]}The input S3 path should contain a file or multiple files where each line is a JSON, the same JSON format as the one expected in the predict function. Example of a file:{"features":[[5.1,3.5,1.4,0.2]]}{"features":[[4.9,3.0,1.4,0.2]]}{"features":[[4.7,3.2,1.3,0.2]]}{"features":[[4.6,3.1,1.5,0.2]]}Required Flags--s3-model-location S3_LOCATION_TO_MODEL_TAR_GZor-m S3_LOCATION_TO_MODEL_TAR_GZ: S3 location to to model tar.gz--s3-input-location S3_INPUT_LOCATIONor-i S3_INPUT_LOCATION: s3 input data location--s3-output-location S3_OUTPUT_LOCATIONor-o S3_OUTPUT_LOCATION: s3 location to save predictions--num-instances NUMBER_OF_EC2_INSTANCESorn NUMBER_OF_EC2_INSTANCES: Number of ec2 instances--ec2-type EC2_TYPEore EC2_TYPE: ec2 type. Refer tohttps://aws.amazon.com/sagemaker/pricing/instance-types/Optional Flags--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for deploying withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role--wait WAIT_UNTIL_BATCH_TRANSFORM_JOB_IS_FINISHEDor-w WAIT_UNTIL_BATCH_TRANSFORM_JOB_IS_FINISHED: Optional flag to wait until Batch Transform is finished. (default: don't wait)--job-name JOB_NAME: Optional name for the SageMaker batch transform jobExamplesagify cloud batch-transform -m s3://my-bucket/output/model.tar.gz -i s3://my-bucket/input_features -o s3://my-bucket/predictions -n 3 -e ml.m4.xlargeCloud Create Streaming InferenceNOTE: THIS IS AN EXPERIMENTAL FEATUREMake sure that the following 2 policies are attached to the role you created in section "Configure AWS Account":NameCreates streaming inference pipelinesSynopsissagify cloud create-streaming-inference --name WORKER_NAME --endpoint-name ENDPOINT_NAME --input-topic-name FEATURES_INPUT_TOPIC_NAME --output-topic-name PREDICTIONS_OUTPUT_TOPIC_NAME --type STREAMING_INFERENCE_TYPEDescriptionThis command creates a worker as a Lambda function that listens to features in theFEATURES_INPUT_TOPIC_NAME, calls the the endpointENDPOINT_NAMEand, finally, forwards predictions toPREDICTIONS_OUTPUT_TOPIC_NAME.Required Flags--name WORKER_NAME: The name of the Lambda function--endpoint-name ENDPOINT_NAME: The name of the endpoint of the deployed model--input-topic-name FEATURES_INPUT_TOPIC_NAME: Topic name where features will be landed--output-topic-name PREDICTIONS_OUTPUT_TOPIC_NAME: Topic name where model predictions will be forwarded--type STREAMING_INFERENCE_TYPE: The type of streaming inference. At the moment, onlySQSis supported!Examplesagify cloud create-streaming-inference --name recommender-worker --endpoint-name my-recommender-endpoint-1 --input-topic-name features --output-topic-name model-predictions --type SQSCloud Delete Streaming InferenceNOTE: THIS IS AN EXPERIMENTAL FEATUREMake sure that the following 2 policies are attached to the role you created in section "Configure AWS Account":NameDeletes streaming inference pipelinesSynopsissagify cloud delete-streaming-inference --name WORKER_NAME --input-topic-name FEATURES_INPUT_TOPIC_NAME --output-topic-name PREDICTIONS_OUTPUT_TOPIC_NAME --type STREAMING_INFERENCE_TYPEDescriptionThis command deletes the worker (i.e. Lambda function), input topicFEATURES_INPUT_TOPIC_NAMEand output topicPREDICTIONS_OUTPUT_TOPIC_NAME.Required Flags--name WORKER_NAME: The name of the Lambda function--input-topic-name FEATURES_INPUT_TOPIC_NAME: Topic name where features will be landed--output-topic-name PREDICTIONS_OUTPUT_TOPIC_NAME: Topic name where model predictions will be forwarded--type STREAMING_INFERENCE_TYPE: The type of streaming inference. At the moment, onlySQSis supported!Examplesagify cloud delete-streaming-inference --name recommender-worker --input-topic-name features --output-topic-name model-predictions --type SQSCloud Lightning DeployNameCommand for lightning deployment of pre-trained ML model(s) on AWS SageMaker without codeSynopsissagify cloud lightning-deploy --framework FRAMEWORK --num-instances NUMBER_OF_EC2_INSTANCES --ec2-type EC2_TYPE --aws-profile AWS_PROFILE --aws-region AWS_REGION --extra-config-file EXTRA_CONFIG_FILE [--model-server-workers MODEL_SERVER_WORKERS] [--s3-model-location S3_LOCATION_TO_MODEL_TAR_GZ] [--aws-tags TAGS] [--iam-role-arn IAM_ROLE] [--external-id EXTERNAL_ID] [--endpoint-name ENDPOINT_NAME]DescriptionThis command deploys a pre-trained ML model without code.Required Flags--framework FRAMEWORK: Name of the ML framework. Valid values:sklearn,huggingface,xgboost--num-instances NUMBER_OF_EC2_INSTANCESorn NUMBER_OF_EC2_INSTANCES: Number of ec2 instances--ec2-type EC2_TYPEore EC2_TYPE: ec2 type. Refer tohttps://aws.amazon.com/sagemaker/pricing/instance-types/--aws-profile AWS_PROFILE: The AWS profile to use for the lightning deploy command--aws-region AWS_REGION: The AWS region to use for the lightning deploy command--extra-config-file EXTRA_CONFIG_FILE: Json file with ML framework specific argumentsFor SKLearn, you have to specify theframework_versionin the EXTRA_CONFIG_FILE and specify the S3 location to model tar.gz (i.e. tar gzip your sklearn pickled fileOptional Flags--s3-model-location S3_LOCATION_TO_MODEL_TAR_GZor-m S3_LOCATION_TO_MODEL_TAR_GZ: Optional S3 location to model tar.gz--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for deploying withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role--endpoint-name ENDPOINT_NAME: Optional name for the SageMaker endpointExample for SKLearnCompress your pre-trained sklearn model to a GZIP tar archive with command!tar czvf model.tar.gz $your_sklearn_model_name.sagify cloud lightning-deploy --framework sklearn -n 1 -e ml.c4.2xlarge --extra-config-file sklearn_config.json --aws-region us-east-1 --aws-profile sagemaker-dev -m s3://my-bucket/output/model.tar.gzThesklearn_config.jsonmust contain the following flagframework_version. Supported sklearn version(s): 0.20.0, 0.23-1.Example ofsklearn_config.json:{
      "framework_version": "0.23-1"
    }Example for HuggingFace by specifying theS3_LOCATION_TO_MODEL_TAR_GZCompress your pre-trained HuggingFace model to a GZIP tar archive with command!tar czvf model.tar.gz $your_hg_model_name.sagify cloud lightning-deploy --framework huggingface -n 1 -e ml.c4.2xlarge --extra-config-file huggingface_config.json --aws-region us-east-1 --aws-profile sagemaker-dev -m s3://my-bucket/output/model.tar.gzThehuggingface_config.jsonmust contain the following flagspytorch_versionortensorflow_version(not both), andtransformers_version. For more info:https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/sagemaker.huggingface.html#hugging-face-model.Example ofhuggingface_config.json:{
      "transformers_version": "4.6.1",
      "pytorch_version": "1.7.1"
    }Example for HuggingFace without specifying theS3_LOCATION_TO_MODEL_TAR_GZsagify cloud lightning-deploy --framework huggingface -n 1 -e ml.c4.2xlarge --extra-config-file huggingface_config.json --aws-region us-east-1 --aws-profile sagemaker-devThehuggingface_config.jsonmust contain the following flagspytorch_versionortensorflow_version(not both),transformers_versionandhub. For more info:https://sagemaker.readthedocs.io/en/stable/frameworks/huggingface/sagemaker.huggingface.html#hugging-face-model.Example ofhuggingface_config.json:{
      "transformers_version": "4.6.1",
      "pytorch_version": "1.7.1",
      "hub": {
        "HF_MODEL_ID": "gpt2",
        "HF_TASK": "text-generation"
      }
    }Example for XGBoostCompress your pre-trained XGBoost model to a GZIP tar archive with command!tar czvf model.tar.gz $your_xgboost_model_name.sagify cloud lightning-deploy --framework xgboost -n 1 -e ml.c4.2xlarge --extra-config-file xgboost_config.json --aws-region us-east-1 --aws-profile sagemaker-dev -m s3://my-bucket/output/model.tar.gzThexgboost_config.jsonmust contain the following flagframework_version. Supported xgboost version(s): 0.90-2, 1.0-1, and later.Example ofxgboost_config.json:{
      "framework_version": "0.23-1"
    }Cloud Foundation Model DeployNameCommand for deployment of Foundation models on SageMaker without codeSynopsissagifycloudfoundation-model-deploy--model-idMODEL_ID--model-versionMODEL_VERSION--num-instancesNUMBER_OF_EC2_INSTANCES--ec2-typeEC2_TYPE--aws-profileAWS_PROFILE--aws-regionAWS_REGION[--aws-tagsTAGS][--iam-role-arnIAM_ROLE][--external-idEXTERNAL_ID][--endpoint-nameENDPOINT_NAME]DescriptionThis command deploys a Foundation model without code.Required Flags--model-id MODEL_ID: Model id of the Foundation model. For more, see the list of Foundation modelshttps://sagemaker.readthedocs.io/en/stable/doc_utils/pretrainedmodels.html.--model-version MODEL_VERSION: Model verion of the Foundation model (default: 1.* which fetches the latest of this major version)--num-instances NUMBER_OF_EC2_INSTANCESorn NUMBER_OF_EC2_INSTANCES: Number of ec2 instances--ec2-type EC2_TYPEore EC2_TYPE: ec2 type. Refer tohttps://aws.amazon.com/sagemaker/pricing/instance-types/--aws-profile AWS_PROFILE: The AWS profile to use for the lightning deploy command--aws-region AWS_REGION: The AWS region to use for the lightning deploy commandOptional Flags--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for deploying withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role--endpoint-name ENDPOINT_NAME: Optional name for the SageMaker endpointLLM Start InfrastructureNameCommand to start LLM infrastructureSynopsissagifyllmstart--all--chat-completions--image-creations--embeddings[--configEC2_CONFIG_FILE]--aws-profileAWS_PROFILE--aws-regionAWS_REGION[--aws-tagsTAGS][--iam-role-arnIAM_ROLE][--external-idEXTERNAL_ID]DescriptionIt spins up the endpoints for chat completions, image creation and embeddings.Required Flags--all: Start infrastructure for all services. If this flag is used the flags--chat-completions,--image-creations,--embeddingsare ignored.--chat-completions: Start infrastructure for chat completions.--image-creations: Start infrastructure for image creations.--embeddings: Start infrastructure for embeddings.--config EC2_CONFIG_FILE: Path to config file to override foundation models, ec2 instance types and/or number of instances.--aws-profile AWS_PROFILE: The AWS profile to use for the lightning deploy command--aws-region AWS_REGION: The AWS region to use for the lightning deploy commandOptional Flags--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for deploying withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM roleLLM Stop InfrastructureNameCommand to stop LLM infrastructureSynopsissagifyllmstop--all--chat-completions--image-creations--embeddings--aws-profileAWS_PROFILE--aws-regionAWS_REGION[--aws-tagsTAGS][--iam-role-arnIAM_ROLE][--external-idEXTERNAL_ID]DescriptionIt stop all or some of the services that are running.Required Flags--all: Start infrastructure for all services. If this flag is used the flags--chat-completions,--image-creations,--embeddingsare ignored.--chat-completions: Start infrastructure for chat completions.--image-creations: Start infrastructure for image creations.--embeddings: Start infrastructure for embeddings.--aws-profile AWS_PROFILE: The AWS profile to use for the lightning deploy command--aws-region AWS_REGION: The AWS region to use for the lightning deploy commandOptional Flags--aws-tags TAGSor-a TAGS: Tags for labeling a training job of the formtag1=value1;tag2=value2. For more, seehttps://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.--iam-role-arn IAM_ROLEor-r IAM_ROLE: AWS IAM role to use for deploying withSageMaker--external-id EXTERNAL_IDor-x EXTERNAL_ID: Optional external id used when using an IAM role