Package: proximl
proxiML Python SDK and Command Line ToolsProvides programmatic access toproxiML platform.InstallationPython 3.8 or above is required.pip install proximlAuthenticationPrerequisitesYou must have a validproxiML account. On theaccount settings pageclick theCreatebutton in theAPI Keyssection. This will automatically download acredentials.jsonfile. This file can only be generated once per API key. Treat this file as a password, as anyone with access to your API key will have the ability to create and control resources in your proxiML account. You can deactivate any API key by clicking theRemovebutton.Creating resources on the proxiML platform requires a non-zero credit balance. To purchase credits or sign-up for automatic credit top-ups, visit thebilling page.MethodsCredentials FileThe easiest way to authenticate is to place the credentials file downloaded into the.proximlfolder of your home directory and ensure only you have access to it. From the directory that thecredentials.jsonfile was downloaded, run the following command:mkdir -p ~/.proximl
mv credentials.json ~/.proximl/credentials.json
chmod 600 ~/.proximl/credentials.jsonEnvironment VariablesYou can also use environment variablesPROXIML_USERandPROXIML_KEYand set them to their respective values from thecredentials.jsonfile.export PROXIML_USER=<'user' field from credentials.json>
export PROXIML_KEY=<'key' field from credentials.json>
python create_job.pyEnvironment variables will override any credentials stored in~/.proximl/credentials.jsonRuntime VariablesAPI credentials can also be passed directly to the ProxiML object constructor at runtime.import proximl
proximl = proximl.ProxiML(user="user field from credentials.json",key="key field from credentials.json>")
await proximl.jobs.create(...)Passing credentials to the ProxiML constructor will override all other methods for setting credentials.ConfigurationBy default, all operations using the proxiML SDK/CLI will use the Personalprojectfor proxiML account the API keys were generated from. To change the active project, run the configure command:proximl configureThis command will output the currently configured active project (UNSETdefaults toPersonal) and allows you to specify any project you have access to as the new active project.Current Active Project: Personal
Select Active Project: (My Other Project, Personal, Project Shared With Me) [Personal]:Once you select a project, it will store the results of your selection in theconfig.jsonfile in thePROXIML_CONFIG_DIRfolder (~/.proximlby default). Once the active project is set, all subsequent operations will use the selected project.This setting can also be overridden at runtime using the environment variablePROXIML_PROJECT:PROXIML_PROJECT=<PROJECT ID> python create_job.pyor by instantiating the proximl client with theprojectkeyword argument:import proximl
proximl = proximl.ProxiML(project="PROJECT ID")
await proximl.jobs.create(...)You must specify the project ID (not name) when using the runtime options. The project ID can be found by runningproximl project list.UsagePython SDKThe proxiML SDK utilizes theasyncio libraryto ease the concurrent execution of long running tasks. An example of how to create a dataset from an S3 bucket and immediately run a training job on that dataset is the following:from proximl.proximl import ProxiML
import asyncio


proximl_client = ProxiML()

# Create the dataset
dataset = asyncio.run(
    proximl_client.datasets.create(
        name="Example Dataset",
        source_type="aws",
        source_uri="s3://proximl-examples/data/cifar10",
    )
)

print(dataset)

# Watch the log output, attach will return when data transfer is complete
asyncio.run(dataset.attach())

# Create the job
job = asyncio.run(
    proximl_client.jobs.create(
        name="Example Training Job",
        type="training",
        gpu_type="GTX 1060",
        gpu_count=1,
        disk_size=10,
        workers=[
            "PYTHONPATH=$PYTHONPATH:$PROXIML_MODEL_PATH python -m official.vision.image_classification.resnet_cifar_main --num_gpus=1 --data_dir=$PROXIML_DATA_PATH --model_dir=$PROXIML_OUTPUT_PATH --enable_checkpoint_and_export=True --train_epochs=10 --batch_size=1024",
        ],
        data=dict(
            datasets=[dict(id=dataset.id, type="existing")],
            output_uri="s3://proximl-examples/output/resnet_cifar10",
            output_type="aws",
        ),
        model=dict(git_uri="git@github.com:proxiML/test-private.git"),
    )
)
print(job)

# Watch the log output, attach will return when the training job stops
asyncio.run(job.attach())

# Cleanup job and dataset
asyncio.run(job.remove())
asyncio.run(dataset.remove())See more examples in theexamples folderCommand Line InterfaceThe command line interface is rooted in theproximlcommand. To see the available options, run:proximl --helpTo list all jobs:proximl job listTo list all datasets:proximl dataset listTo connect to a job that requires theconnection capability:proximl job connect <job ID or name>To watch the realtime job logs:proximl job attach <job ID or name>To create and open a notebook job:proximl job create notebook "My Notebook Job"To create a multi-GPU notebook job on a specific GPU type with larger scratch directory space:proximl job create notebook --gpu-type "RTX 3090" --gpu-count 4 --disk-size 50 "My Notebook Job"To run the model training code in thetrain.pyfile in your local~/model-codedirectory on the training data in your local~/datadirectory:proximl job create training --model-dir ~/model-code --data-dir ~/data "My Training Job" "python train.py"Stop a job by job ID:proximl job stop fe52527c-1f4b-468f-b57d-86db864cc089Stop a job by name:proximl job stop "My Notebook Job"Restart a notebook job:proximl job start "My Notebook Job"Remove a job by job ID:proximl job remove fe52527c-1f4b-468f-b57d-86db864cc089