Deploying on AWS lambda with PyTorch 1.0

Seanny123: <p>I’m trying to deploy a PyTorch model (currently implemented in 0.4, but simple to convert to 1.0) on AWS Lambda <strong>[1]</strong>. What’s the best way to accomplish this? From my searches, I’ve found a few options.</p>
<h2>Deploy with C++ (for high speed)</h2>
<p>It is possible to include a PyTorch model in C++ and compile it to a binary, as demonstrated <a href="https://pytorch.org/tutorials/advanced/cpp_export.html#depending-on-libtorch-and-building-the-application" rel="nofollow noopener">in this tutorial</a>.</p>
<h2>Deploy via ONNX to Caffe2 (obsolete)</h2>
<p>Previously, there was the option to <a href="https://machinelearnings.co/serving-pytorch-models-on-aws-lambda-with-caffe2-onnx-7b096806cfac" rel="nofollow noopener">export the model into the ONNX format and execute it via Caffe2</a>. However, Caffe2 is now merging with PyTorch, which makes this option obsolete?</p>
<h2>Compile things from source to save space</h2>
<p>Another option I saw was to compile CPU-only PyTorch from source, to reduce the dependency size. However, is this now obsolete, given the pre-built Pytorch CPU pre-built binaries are only 90 MB?</p>
<h2>Other options?</h2>
<p>What other options are there for deploying a PyTorch model to AWS lambda? I noticed <a href="https://github.com/pytorch/glow" rel="nofollow noopener">Glow</a> was recently released, but I don’t understand the state of the project, nor it’s intended use.</p>
<hr>
<p><strong>[1]</strong> For those who aren’t aware. AWS Lambda allows you to only charge for a server when you’re using it. This is in contrast to an EC2 instance, which is constantly running. However, as a trade-off you’re limited by the deployed package size (250 MB). Thus, it’s impractical to use PyTorch with all of it’s dependencies.</p>

Sophia_Wright: <p>hello <a class="mention" href="/u/seanny123">@Seanny123</a> , you mention that it is simple to convert PyTorch 0.4 code to 1.0. Could you share any links to articles/posts that describe the steps to convert PyTorch code from 0.4 to 1.0. Thanks,</p>

Seanny123: <p>Sorry <a class="mention" href="/u/sophia_wright">@Sophia_Wright</a>, I meant it was easy for me to convert. As in, the architecture and my experiences made it manageable. Your circumstances may be quite different than mine and I don’t know of any articles to help you along ):</p>

mmcclean: <p><a class="mention" href="/u/seanny123">@Seanny123</a> did you manage to find a solution to deploying PyTorch 1.0 on AWS Lambda?</p>

Seanny123: <p>Still haven’t deployed anything, unfortunately.</p>

surbhinahtajain: <p>Hi,</p>
<p>I had the option to use the beneath layers for utilizing pytorch on AWS Lambda:</p>
<p>arn:aws:lambda:AWS_REGION:934676248949:layer:pytorchv1-py36:1 PyTorch 1.0.1<br>
arn:aws:lambda:AWS_REGION:934676248949:layer:pytorchv1-py36:2 PyTorch 1.1.0</p>

surbhi_jain: <p>Hi,</p>
<p>I had the option to use the underneath layers for utilizing pytorch on <a href="https://sevenmentors.odoo.com/aws-training-in-pune" rel="noopener nofollow ugc">AWS Lambda</a>:</p>
<p>arn:aws:lambda:AWS_REGION:934676248949:layer:pytorchv1-py36:1 PyTorch 1.0.1<br>
arn:aws:lambda:AWS_REGION:934676248949:layer:pytorchv1-py36:2 PyTorch 1.1.0</p>