Use Executorch's Module Extension API on arm64 Mac

username: simonsasse
cooked: <p>I want to run a small MLP in the Executorch runtime (eventually on android, currently on arm64 Mac).<br>
For the start I just cloned the Executorch Repo and set everything up.<br>
I can build and run the <code>executor_runner</code> without problems.<br>
I then started to edit the <code>executor_runner</code> so that it matches my needs.<br>
Problem is, as soon as I start using the Module Extension API, which would ease things, I get the below error when calling Cmake.<br>
I added the <code>module.h</code> to the <code>targets.bzl</code> and <span class="hashtag-raw">#included</span> it in the <code>executor_runner.cpp</code>.</p>
<pre><code class="lang-auto">Undefined symbols for architecture arm64:
  "executorch::extension::Module::Module(std::__1::basic_string&lt;char, std::__1::char_traits&lt;char&gt;, std::__1::allocator&lt;char&gt;&gt; const&amp;, executorch::extension::Module::LoadMode, std::__1::unique_ptr&lt;executorch::runtime::EventTracer, std::__1::default_delete&lt;executorch::runtime::EventTracer&gt;&gt;)", referenced from:
      _main in executor_runner.cpp.o
ld: symbol(s) not found for architecture arm64
c++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [executor_runner] Error 1
make[1]: *** [CMakeFiles/executor_runner.dir/all] Error 2
make: *** [all] Error 2
</code></pre>
<p>I am new to Executorch and also not a pro in c++ and Cmake.</p>
<p>Executorch: release/0.4<br>
MacBook Air M3 (arm64)</p>

username: lucylq
cooked: <p>Hi <a class="mention" href="/u/simonsasse">@simonsasse</a>,</p>
<p>Thanks for trying out ExecuTorch!</p>
<p>The <code>targets.bzl</code> file is for building with Buck; you’ll have to edit the <code>CMakeLists.txt</code> file to build with CMake. You could take a look at the module test as an example: <a href="https://github.com/pytorch/executorch/tree/main/extension/module/test" class="inline-onebox" rel="noopener nofollow ugc">executorch/extension/module/test at main · pytorch/executorch · GitHub</a></p>
<p>Could you explain more about what you’re trying to do with the Module API / <code>executor_runner</code>? <code>executor_runner</code> uses all the lower-level APIs to load and run a model, whereas Module is a higher-level API that does the same thing. So you probably want to use one or the other.</p>
<p>cc <a class="mention" href="/u/shoumikhin">@shoumikhin</a> for Module API.</p>

username: simonsasse
cooked: <p>Hi <a class="mention" href="/u/lucylq">@lucylq</a>, thank you so much for the quick reply.</p>
<p>I want to use the Module API instead of the lower level APIs used in the executor_runner.</p>
<p>I wrote my own run script before (outside the executorch directory) and ran into the same issue. So I thought I might just try if my Cmake build is missing some reference or so . That’s why I started to edit the executor_runner. It’s basically taken from this tutorial:<br>
<a href="https://pytorch.org/executorch/stable/extension-module.html#running-an-executorch-model-using-the-module-extension-in-c" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/executorch/stable/extension-module.html#running-an-executorch-model-using-the-module-extension-in-c</a></p>
<pre><code class="lang-auto">#include &lt;executorch/extension/module/module.h&gt;
#include &lt;executorch/extension/tensor/tensor.h&gt;

#include &lt;cassert&gt;
#include &lt;iostream&gt;

using namespace ::executorch::extension;

int main(int argc, char **argv)
{
    // Create a Module.
    Module module("example/path.pte");

    // Wrap the input data with a Tensor.
    float input[1 * 3];
    auto inTensor = from_blob(input, {1, 3});

    // ......

    // Perform forward
    const auto result = module.forward(inTensor);

    // Check for success or failure.
   // ......
}
</code></pre>

username: lucylq
cooked: <p>Hi <a class="mention" href="/u/simonsasse">@simonsasse</a>, thanks for the additional context.</p>
<p>So executor_runner is built in the <a href="https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L808-L835" rel="noopener nofollow ugc">root CMakeLists.txt file</a>, set to ON by default.</p>
<p>And Module is built in the root CMakeLists.txt file if <a href="https://github.com/pytorch/executorch/blob/main/CMakeLists.txt#L692" rel="noopener nofollow ugc"><code>EXECUTORCH_BUILD_EXTENSION_MODULE=ON</code></a>, it is OFF by default.</p>
<p>I think the missing pieces are:</p>
<ol>
<li>Set <code>EXECUTORCH_BUILD_EXTENSION_MODULE=ON</code> in the root CMakeLists.txt, to build the Module library. This will build the targets <a href="https://github.com/pytorch/executorch/blob/main/extension/module/CMakeLists.txt#L49" rel="noopener nofollow ugc"><code>extension_module</code>, <code>extension_module_static</code></a>.</li>
<li>Link <code>extension_module</code> or <code>extension_module_static</code> as a library to your target, eg., <code>target_link_libraries({your_target_name} extension_module)</code></li>
</ol>
<p><a href="https://github.com/pytorch/executorch/pull/7557" rel="noopener nofollow ugc">See this example</a>. It might be better to create a separate CMakeLists.txt file rather than putting it into the root, but hopefully this helps.</p>