Is it possible to do operator profiling within the naked binary for PyTorch Mobile in Android?

username: Chaojian_Li
cooked: <p>Hello all,<br>
The naked binary for benchmarking pytorch mobile models (<a href="https://pytorch.org/tutorials/recipes/mobile_perf.html?highlight=mobile" class="inline-onebox" rel="noopener nofollow ugc">Pytorch Mobile Performance Recipes — PyTorch Tutorials 1.10.1+cu102 documentation</a>) provides a good way to know the model performance without deploying a real App.</p>
<p>I was wondering whether it can do the operator profiling like PyTorch Profiler (<a href="https://pytorch.org/blog/introducing-pytorch-profiler-the-new-and-improved-performance-tool/" class="inline-onebox" rel="noopener nofollow ugc">Introducing PyTorch Profiler - the new and improved performance tool | PyTorch</a>)?</p>
<p>So that maybe we can know which operator is the bottleneck of the whole model?</p>
<p>Thank you!</p>

username: Linbin
cooked: <p>I think we do have op profiling. cc <a class="mention" href="/u/kimishpatel">@kimishpatel</a>.</p>

username: Chaojian_Li
cooked: <p>Thank you <a class="mention" href="/u/linbin">@Linbin</a> for the suggestion!</p>
<p>Hi <a class="mention" href="/u/kimishpatel">@kimishpatel</a>, could you give me some hints on how to do the operator profiling in the naked PyTorch Mobile binary benchmark like PyTorch Profiler?</p>
<p>I also tried some other solutions like <a href="https://github.com/facebook/FAI-PEP" rel="noopener nofollow ugc">Facebook AI Performance Evaluation Platform</a>, which provides operator profiling for caffe2 models, but it seems that it has been out-of-date for the current PyTorch repo.</p>
<p>Thank you in advance for your time and efforts!</p>

username: Chaojian_Li
cooked: <p>Hi <a class="mention" href="/u/linbin">@Linbin</a> and <a class="mention" href="/u/kimishpatel">@kimishpatel</a><br>
Thank you for your help!<br>
Is <a href="https://github.com/pytorch/pytorch/blob/master/binaries/speed_benchmark_torch.cc" rel="noopener nofollow ugc">this</a> the source file for the generated speed_benchmark_torch binary?<br>
If so, it seems that op profiling has not been supported yet.<br>
Do you have any plan to support the op profiling? Or could you share some hints with me on how to do that myself?</p>
<p>Thank you!</p>

username: kimishpatel
cooked: <p>Are you running with <code>BUILD_LITE_INTERPRETER</code>? Do you know?</p>

username: Chaojian_Li
cooked: <aside class="quote no-group" data-username="kimishpatel" data-post="5" data-topic="144249">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kimishpatel/48/39320_2.png" class="avatar"> kimishpatel:</div>
<blockquote>
<p>BUILD_LITE_INTERPRETER</p>
</blockquote>
</aside>
<p>Hi <a class="mention" href="/u/kimishpatel">@kimishpatel</a>,</p>
<p>Thank you for reply!<br>
Yes, I build with <code>BUILD_LITE_INTERPRETER</code>.</p>
<p>The following are the steps I used to build it</p>
<pre><code class="lang-auto">cd pytorch

export ANDROID_ABI=arm64-v8a # I am using Pixel3 for it
export ANDROID_NDK=/path/to/Android/Sdk/ndk/21.3.6528147/

rm -rf build_android

BUILD_PYTORCH_MOBILE=1 BUILD_LITE_INTERPRETER=1 ./scripts/build_android.sh \
-DBUILD_BINARY=ON \
-DBUILD_CAFFE2_MOBILE=OFF \
-DCMAKE_PREFIX_PATH=$(python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())') \
-DPYTHON_EXECUTABLE=$(python -c 'import sys; print(sys.executable)')
</code></pre>
<p>Is it the key to do operator profiling? Or any other steps are needed to do it?</p>
<p>Thank you!</p>