Undefined reference when building from modified source code

mhbassel: <p>Hello Everyone!</p>
<p>I am trying to build PyTorch from source after modifiing this file<br>
<a href="https://github.com/pytorch/pytorch/blob/master/torch/csrc/jit/serialization/import.cpp" rel="noopener nofollow ugc">import.cpp</a> to let it use a function imported from a C file.</p>
<p>I imported it like this:</p>
<pre><code class="lang-auto">extern "C" {
    #include "decryption_utils.h" // a C header, so wrap it in extern "C"
}
</code></pre>
<p>The C file is in the same directory with its header.</p>
<p>But when I run:</p>
<pre><code class="lang-auto">python setup.py install
</code></pre>
<p>I am getting the following error when linking:</p>
<pre><code class="lang-auto">Building wheel torch-2.0.0a0+git8bc38ae
-- Building version 2.0.0a0+git8bc38ae
cmake --build . --target install --config Release
[1/164] Linking CXX executable bin/basic
FAILED: bin/basic 
: &amp;&amp; /usr/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -rdynamic -pthread -Wl,-rpath-link,/usr/lib/x86_64-linux-gnu caffe2/CMakeFiles/basic.dir/__/aten/src/ATen/test/basic.cpp.o -o bin/basic  -Wl,-rpath,/home/moh/pytorch/build/lib:  lib/libgtest_main.a  /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so  /usr/lib/x86_64-linux-gnu/libpthread.so  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch.so" -Wl,--as-needed  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch_cpu.so" -Wl,--as-needed  lib/libprotobuf.a  lib/libc10.so  lib/libgtest.a  -pthread &amp;&amp; :
/usr/bin/ld: /home/moh/pytorch/build/lib/libtorch_cpu.so: undefined reference to `decrypt_fcontent'
collect2: error: ld returned 1 exit status
[2/164] Linking CXX executable bin/half_test
FAILED: bin/half_test 
: &amp;&amp; /usr/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -rdynamic -pthread -Wl,-rpath-link,/usr/lib/x86_64-linux-gnu caffe2/CMakeFiles/half_test.dir/__/aten/src/ATen/test/half_test.cpp.o -o bin/half_test  -Wl,-rpath,/home/moh/pytorch/build/lib:  lib/libgtest_main.a  /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so  /usr/lib/x86_64-linux-gnu/libpthread.so  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch.so" -Wl,--as-needed  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch_cpu.so" -Wl,--as-needed  lib/libprotobuf.a  lib/libc10.so  lib/libgtest.a  -pthread &amp;&amp; :
/usr/bin/ld: /home/moh/pytorch/build/lib/libtorch_cpu.so: undefined reference to `decrypt_fcontent'
collect2: error: ld returned 1 exit status
[3/164] Linking CXX executable bin/broadcast_test
FAILED: bin/broadcast_test 
: &amp;&amp; /usr/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -rdynamic -pthread -Wl,-rpath-link,/usr/lib/x86_64-linux-gnu caffe2/CMakeFiles/broadcast_test.dir/__/aten/src/ATen/test/broadcast_test.cpp.o -o bin/broadcast_test  -Wl,-rpath,/home/moh/pytorch/build/lib:  lib/libgtest_main.a  /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so  /usr/lib/x86_64-linux-gnu/libpthread.so  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch.so" -Wl,--as-needed  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch_cpu.so" -Wl,--as-needed  lib/libprotobuf.a  lib/libc10.so  lib/libgtest.a  -pthread &amp;&amp; :
/usr/bin/ld: /home/moh/pytorch/build/lib/libtorch_cpu.so: undefined reference to `decrypt_fcontent'
collect2: error: ld returned 1 exit status
[4/164] Linking CXX executable bin/MaybeOwned_test
FAILED: bin/MaybeOwned_test 
: &amp;&amp; /usr/bin/c++ -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -O3 -DNDEBUG -DNDEBUG -rdynamic -pthread -Wl,-rpath-link,/usr/lib/x86_64-linux-gnu caffe2/CMakeFiles/MaybeOwned_test.dir/__/aten/src/ATen/test/MaybeOwned_test.cpp.o -o bin/MaybeOwned_test  -Wl,-rpath,/home/moh/pytorch/build/lib:  lib/libgtest_main.a  /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so  /usr/lib/x86_64-linux-gnu/libpthread.so  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch.so" -Wl,--as-needed  -Wl,--no-as-needed,"/home/moh/pytorch/build/lib/libtorch_cpu.so" -Wl,--as-needed  lib/libprotobuf.a  lib/libc10.so  lib/libgtest.a  -pthread &amp;&amp; :
/usr/bin/ld: /home/moh/pytorch/build/lib/libtorch_cpu.so: undefined reference to `decrypt_fcontent'
collect2: error: ld returned 1 exit status
[5/164] Linking CXX executab
...
</code></pre>
<p>Apparently it fails because of my (imported) function <code>decrypt_fcontent</code> and I am not sure how to solve it, like should I define the header somewhere or so?</p>
<p>Any help will be appreciated a lot!<br>
Thank you<br>
Moh</p>

mhbassel: <p>This is the header:</p>
<pre><code class="lang-auto">#ifndef DECRYPTION_UTILS_H_
#define DECRYPTION_UTILS_H_

int decrypt_fcontent(char *fpath, char* dcontent);

#endif
</code></pre>

mhbassel: <p>After modifying <code>pytorch/BUILD.bazel</code> and <code>pytorch/caffe2/serialize/CMakeLists.txt</code> (I am using them there as well) to include the new files, the problem solved.</p>
<p>Also, since I’m using some openssl functions, I needed to link with <code>-lssl -lcrypto</code>, and that by adding them to the LINK_LIBRARIES in <code>pytorch/build/build.ninja</code>.</p>

mhbassel: <p>Adding <strong>-lssl -lcrypto</strong> to <strong>pytorch/build/build.ninja</strong> is really just a way to make it work, not recomended.</p>
<p>In order to build PyTorch from source with custom modifications, and assuming those modifications are using OpenSSL, then do the following:</p>
<ol>
<li>First, make sure OpenSSL is installed</li>
<li>Then either:
<ul>
<li>add your implementations directly in your target PyTorch files and just include the OpenSSL library, or</li>
<li>if you have your implementations in your own files, then just include those files in your target PyTorch files</li>
</ul>
</li>
<li>If you followed the second approach, then you need to add your files in: <strong>build_variables.bzl</strong>
</li>
<li>For linking, add these to <strong>torch/CMakeLists.txt</strong>:</li>
</ol>
<pre><code class="lang-auto">target_link_libraries(torch_cpu PUBLIC ssl crypto)
target_link_libraries(torch_cuda PUBLIC ssl crypto)
</code></pre>
<ol start="5">
<li>And build, done.</li>
</ol>