Luke Stanley commited on
Commit
b51ce5c
1 Parent(s): f2e80c9

Add more serverless GPU endpoint setup instruction detail

Browse files
Files changed (1) hide show
  1. serverless.md +4 -2
serverless.md CHANGED
@@ -41,9 +41,11 @@ After testing and ensuring that everything works as expected, you can build the
41
 
42
  # Runpod previsioning:
43
  You'll need an account on Runpod with credit.
44
- You'll need a serverless GPU endpoint setting up using your Docker image.
45
-
46
  It has a Flashboot feature that seems like Firecracker with GPU support, it might be using Cloud Hypervisor under the hood, currently Firecracker has no GPU support. Fly.io also has something similar, with Cloud Hypervisor.
 
 
47
 
48
  ## Runpod Integration in `app.py`
49
 
 
41
 
42
  # Runpod previsioning:
43
  You'll need an account on Runpod with credit.
44
+ You'll need a serverless GPU endpoint setting up using your Docker image setup here:
45
+ https://www.runpod.io/console/serverless
46
  It has a Flashboot feature that seems like Firecracker with GPU support, it might be using Cloud Hypervisor under the hood, currently Firecracker has no GPU support. Fly.io also has something similar, with Cloud Hypervisor.
47
+ You'll need the secret saved somewhere securely. This will likely end up as a securely treated env var for use by app.py later.
48
+ You'll also need the endpoint ID.
49
 
50
  ## Runpod Integration in `app.py`
51