Need input on numa_test.py::NUMATest::test_numa failure

pramenku: <p>I am getting below message and numa test failed, Can someone please guide what going worng here?</p>
<p>numactl --hardware<br>
available: 4 nodes (0-3)<br>
node 0 cpus: 0 1 2 3 4 5 6 7 32 33 34 35 36 37 38 39<br>
node 0 size: 64351 MB<br>
node 0 free: 1532 MB<br>
node 1 cpus: 8 9 10 11 12 13 14 15 40 41 42 43 44 45 46 47<br>
node 1 size: 64505 MB<br>
node 1 free: 551 MB<br>
node 2 cpus: 16 17 18 19 20 21 22 23 48 49 50 51 52 53 54 55<br>
node 2 size: 64480 MB<br>
node 2 free: 17961 MB<br>
node 3 cpus: 24 25 26 27 28 29 30 31 56 57 58 59 60 61 62 63<br>
node 3 size: 64504 MB<br>
node 3 free: 32454 MB<br>
node distances:<br>
node   0   1   2   3<br>
0:  10  16  16  16<br>
1:  16  10  16  16<br>
2:  16  16  10  16<br>
3:  16  16  16  10</p>
<h2><a name="failure-log-1" class="anchor" href="#failure-log-1"></a>Failure log</h2>
<p>…/…/…/…/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py::NUMATest::test_numa FAILED [ 15%]</p>
<p>=================================== FAILURES ===================================<br>
______________________________ NUMATest.test_numa ______________________________</p>
<p>self = &lt;caffe2.python.numa_test.NUMATest testMethod=test_numa&gt;</p>
<pre><code>def test_numa(self):
    net = build_test_net("test_numa")
</code></pre>
<blockquote>
<pre><code>  workspace.RunNetOnce(net)
</code></pre>
</blockquote>
<p>/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py:46:</p>
<hr>
<p>/root/.local/lib/python2.7/site-packages/caffe2/python/workspace.py:232: in RunNetOnce<br>
StringifyProto(net),</p>
<hr>
<p>func = &lt;built-in method run_net_once of PyCapsule object at 0x7f685e201090&gt;<br>
op_id_fetcher = &lt;bound method Workspace._last_failed_op_net_position of &lt;caffe2.python.caffe2_pybind11_state_hip.Workspace object at 0x7f635c40a378&gt;&gt;<br>
net_name = ‘test_numa’<br>
args = (‘\n\ttest_numa\x12&gt;\x12\routput_blob_0\x1a\x00"\x0cConstantFill*\t\n\x05shape0\x01*\x0c\n\x05value\x15\xc3\xf5H@2\x04…x128\n\routput_blob_1\x12\x11output_blob_1_gpu\x1a\x00"\x0cCopyCPUToGPU2\x04\x08\x01\x10\x00\x1a\x10async_scheduling’,)<br>
kwargs = {}, op_id = 2<br>
net_tracebacks = {0: [(‘/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py’, 21, ‘build_test_net’), (‘/root/.local/lib…t.py’, 221, ‘runtest’), (‘/usr/local/lib/python2.7/dist-packages/_pytest/runner.py’, 122, ‘pytest_runtest_call’), …]}<br>
tb = [(‘/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py’, 32, ‘build_test_net’), (‘/root/.local/lib/pyt…st.py’, 221, ‘runtest’), (‘/usr/local/lib/python2.7/dist-packages/_pytest/runner.py’, 122, ‘pytest_runtest_call’), …]<br>
line = (‘/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py’, 32, ‘build_test_net’)</p>
<pre><code>def CallWithExceptionIntercept(func, op_id_fetcher, net_name, *args, **kwargs):
    try:
</code></pre>
<blockquote>
<pre><code>      return func(*args, **kwargs)
</code></pre>
</blockquote>
<p>E           RuntimeError: [enforce fail at operator.cc:243] op. Cannot create operator of type ‘CopyCPUToGPU’ on the device ‘CUDA’. Verify that implementation for the corresponding device exist. It might also happen if the binary is not linked with the operator implementation code. If Python frontend is used it might happen if dyndep.InitOpsLibrary call is missing. Operator def: input: “output_blob_0” output: “output_blob_0_gpu” name: “” type: “CopyCPUToGPU” device_option { device_type: 1 device_id: 0 }<br>
E           frame <span class="hashtag-raw">#0:</span> c10::ThrowEnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string&lt;char, std::char_traits, std::allocator &gt; const&amp;, void const*) + 0xa3 (0x7f6816937783 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libc10.so)<br>
E           frame <span class="hashtag-raw">#1:</span> caffe2::CreateOperator(caffe2::OperatorDef const&amp;, caffe2::Workspace*, int) + 0xd41 (0x7f6818916161 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#2:</span> caffe2::dag_utils::prepareOperatorNodes(std::shared_ptr&lt;caffe2::NetDef const&gt; const&amp;, caffe2::Workspace*) + 0x321 (0x7f6818903a91 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#3:</span> caffe2::AsyncNetBase::AsyncNetBase(std::shared_ptr&lt;caffe2::NetDef const&gt; const&amp;, caffe2::Workspace*) + 0x1ad (0x7f68188eb4dd in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#4:</span>  + 0x1b9507c (0x7f68188f507c in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#5:</span>  + 0x1b951e2 (0x7f68188f51e2 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#6:</span> caffe2::CreateNet(std::shared_ptr&lt;caffe2::NetDef const&gt; const&amp;, caffe2::Workspace*) + 0x3a2 (0x7f68188e7552 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#7:</span> caffe2::CreateNet(caffe2::NetDef const&amp;, caffe2::Workspace*) + 0x92 (0x7f68188e6f92 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#8:</span> caffe2::Workspace::RunNetOnce(caffe2::NetDef const&amp;) + 0x21 (0x7f681893f681 in /root/.local/lib/python2.7/site-packages/caffe2/python/…/…/torch/lib/libcaffe2.so)<br>
E           frame <span class="hashtag-raw">#9:</span>  + 0x4705d (0x7f685dee405d in /root/.local/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state_hip.so)<br>
E           frame <span class="hashtag-raw">#10:</span>  + 0x6c0bb (0x7f685df090bb in /root/.local/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state_hip.so)<br>
E           frame <span class="hashtag-raw">#11:</span> PyEval_EvalFrameEx + 0x6f3e (0x4c2e1e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#12:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#13:</span> PyEval_EvalFrameEx + 0x6076 (0x4c1f56 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#14:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#15:</span> PyEval_EvalFrameEx + 0x6076 (0x4c1f56 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#16:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#17:</span> PyEval_EvalFrameEx + 0x6076 (0x4c1f56 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#18:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#19:</span> /usr/bin/python2() [0x4d57a3]<br>
E           frame <span class="hashtag-raw">#20:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#21:</span> PyEval_EvalFrameEx + 0x263e (0x4be51e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#22:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#23:</span> /usr/bin/python2() [0x4d57a3]<br>
E           frame <span class="hashtag-raw">#24:</span> /usr/bin/python2() [0x4eef5e]<br>
E           frame <span class="hashtag-raw">#25:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#26:</span> /usr/bin/python2() [0x548fc3]<br>
E           frame <span class="hashtag-raw">#27:</span> PyEval_EvalFrameEx + 0x578d (0x4c166d in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#28:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#29:</span> PyEval_EvalFrameEx + 0x6076 (0x4c1f56 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#30:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#31:</span> /usr/bin/python2() [0x4d5669]<br>
E           frame <span class="hashtag-raw">#32:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#33:</span> PyEval_EvalFrameEx + 0x263e (0x4be51e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#34:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#35:</span> PyEval_EvalFrameEx + 0x58e6 (0x4c17c6 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#36:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#37:</span> PyEval_EvalFrameEx + 0x6076 (0x4c1f56 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#38:</span> PyEval_EvalFrameEx + 0x553f (0x4c141f in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#39:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#40:</span> /usr/bin/python2() [0x4d57a3]<br>
E           frame <span class="hashtag-raw">#41:</span> /usr/bin/python2() [0x4eef5e]<br>
E           frame <span class="hashtag-raw">#42:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#43:</span> /usr/bin/python2() [0x548fc3]<br>
E           frame <span class="hashtag-raw">#44:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#45:</span> PyEval_EvalFrameEx + 0x263e (0x4be51e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#46:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#47:</span> PyEval_EvalFrameEx + 0x6076 (0x4c1f56 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#48:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#49:</span> PyEval_EvalFrameEx + 0x58e6 (0x4c17c6 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#50:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#51:</span> /usr/bin/python2() [0x4d57a3]<br>
E           frame <span class="hashtag-raw">#52:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#53:</span> PyEval_EvalFrameEx + 0x263e (0x4be51e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#54:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#55:</span> PyEval_EvalFrameEx + 0x58e6 (0x4c17c6 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#56:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#57:</span> PyEval_EvalFrameEx + 0x58e6 (0x4c17c6 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#58:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#59:</span> /usr/bin/python2() [0x4d5669]<br>
E           frame <span class="hashtag-raw">#60:</span> PyObject_Call + 0x3e (0x4a587e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#61:</span> PyEval_EvalFrameEx + 0x263e (0x4be51e in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#62:</span> PyEval_EvalCodeEx + 0x306 (0x4b9b66 in /usr/bin/python2)<br>
E           frame <span class="hashtag-raw">#63:</span> PyEval_EvalFrameEx + 0x58e6 (0x4c17c6 in /usr/bin/python2)</p>
<h2><a name="rootlocallibpython27site-packagescaffe2pythonworkspacepy211-runtimeerror-captured-stderr-call-warningcaffe2pythonworkspaceoriginal-python-traceback-for-operator-2-in-network-test_numa-in-exception-above-most-recent-call-last-warningcaffe2pythonworkspace-file-usrlibpython27runpypy-line-174-in-_run_module_as_main-warningcaffe2pythonworkspace-file-usrlibpython27runpypy-line-72-in-_run_code-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespytestpy-line-101-in-module-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestconfig__init__py-line-79-in-main-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-242-in-pytest_cmdline_main-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-205-in-wrap_session-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-249-in-_main-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-270-in-pytest_runtestloop-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-77-in-pytest_runtest_protocol-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-92-in-runtestprotocol-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-172-in-call_and_report-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-call_runtest_hook-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-225-in-from_call-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-122-in-pytest_runtest_call-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestunittestpy-line-221-in-runtest-warningcaffe2pythonworkspace-file-usrlibpython27unittestcasepy-line-393-in-__call__-warningcaffe2pythonworkspace-file-usrlibpython27unittestcasepy-line-329-in-run-warningcaffe2pythonworkspace-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-44-in-test_numa-warningcaffe2pythonworkspace-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-32-in-build_test_net-captured-log-call-warning-caffe2pythonworkspaceworkspacepy218-original-python-traceback-for-operator-2-in-network-test_numa-in-exception-above-most-recent-call-last-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27runpypy-line-174-in-_run_module_as_main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27runpypy-line-72-in-_run_code-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespytestpy-line-101-in-module-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestconfig__init__py-line-79-in-main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-242-in-pytest_cmdline_main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-205-in-wrap_session-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-249-in-_main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-270-in-pytest_runtestloop-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-77-in-pytest_runtest_protocol-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-92-in-runtestprotocol-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-172-in-call_and_report-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-call_runtest_hook-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-225-in-from_call-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-122-in-pytest_runtest_call-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestunittestpy-line-221-in-runtest-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27unittestcasepy-line-393-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27unittestcasepy-line-329-in-run-warning-caffe2pythonworkspaceworkspacepy223-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-44-in-test_numa-warning-caffe2pythonworkspaceworkspacepy223-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-32-in-build_test_net-generated-xml-file-homecaffe2caffe2_testspythonresultxml-onnx-coverage-operators-passedloadedtotal-00155-2" class="anchor" href="#rootlocallibpython27site-packagescaffe2pythonworkspacepy211-runtimeerror-captured-stderr-call-warningcaffe2pythonworkspaceoriginal-python-traceback-for-operator-2-in-network-test_numa-in-exception-above-most-recent-call-last-warningcaffe2pythonworkspace-file-usrlibpython27runpypy-line-174-in-_run_module_as_main-warningcaffe2pythonworkspace-file-usrlibpython27runpypy-line-72-in-_run_code-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespytestpy-line-101-in-module-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestconfig__init__py-line-79-in-main-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-242-in-pytest_cmdline_main-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-205-in-wrap_session-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-249-in-_main-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestmainpy-line-270-in-pytest_runtestloop-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-77-in-pytest_runtest_protocol-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-92-in-runtestprotocol-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-172-in-call_and_report-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-call_runtest_hook-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-225-in-from_call-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-122-in-pytest_runtest_call-warningcaffe2pythonworkspace-file-usrlocallibpython27dist-packages_pytestunittestpy-line-221-in-runtest-warningcaffe2pythonworkspace-file-usrlibpython27unittestcasepy-line-393-in-__call__-warningcaffe2pythonworkspace-file-usrlibpython27unittestcasepy-line-329-in-run-warningcaffe2pythonworkspace-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-44-in-test_numa-warningcaffe2pythonworkspace-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-32-in-build_test_net-captured-log-call-warning-caffe2pythonworkspaceworkspacepy218-original-python-traceback-for-operator-2-in-network-test_numa-in-exception-above-most-recent-call-last-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27runpypy-line-174-in-_run_module_as_main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27runpypy-line-72-in-_run_code-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespytestpy-line-101-in-module-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestconfig__init__py-line-79-in-main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-242-in-pytest_cmdline_main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-205-in-wrap_session-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-249-in-_main-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestmainpy-line-270-in-pytest_runtestloop-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-77-in-pytest_runtest_protocol-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-92-in-runtestprotocol-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-172-in-call_and_report-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-call_runtest_hook-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-225-in-from_call-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-197-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggyhookspy-line-289-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-68-in-_hookexec-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggymanagerpy-line-62-in-lambda-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packagespluggycallerspy-line-187-in-_multicall-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestrunnerpy-line-122-in-pytest_runtest_call-warning-caffe2pythonworkspaceworkspacepy223-file-usrlocallibpython27dist-packages_pytestunittestpy-line-221-in-runtest-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27unittestcasepy-line-393-in-__call__-warning-caffe2pythonworkspaceworkspacepy223-file-usrlibpython27unittestcasepy-line-329-in-run-warning-caffe2pythonworkspaceworkspacepy223-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-44-in-test_numa-warning-caffe2pythonworkspaceworkspacepy223-file-rootlocallibpython27site-packagescaffe2pythonnuma_testpy-line-32-in-build_test_net-generated-xml-file-homecaffe2caffe2_testspythonresultxml-onnx-coverage-operators-passedloadedtotal-00155-2"></a>/root/.local/lib/python2.7/site-packages/caffe2/python/workspace.py:211: RuntimeError<br>
----------------------------- Captured stderr call -----------------------------<br>
WARNING:caffe2.python.workspace:Original python traceback for operator <code>2</code> in network <code>test_numa</code> in exception above (most recent call last):<br>
WARNING:caffe2.python.workspace:  File “/usr/lib/python2.7/runpy.py”, line 174, in _run_module_as_main<br>
WARNING:caffe2.python.workspace:  File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pytest.py”, line 101, in <br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/config/<strong>init</strong>.py”, line 79, in main<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 242, in pytest_cmdline_main<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 205, in wrap_session<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 249, in _main<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 270, in pytest_runtestloop<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 77, in pytest_runtest_protocol<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 92, in runtestprotocol<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 172, in call_and_report<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 197, in call_runtest_hook<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 225, in from_call<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 197, in <br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 122, in pytest_runtest_call<br>
WARNING:caffe2.python.workspace:  File “/usr/local/lib/python2.7/dist-packages/_pytest/unittest.py”, line 221, in runtest<br>
WARNING:caffe2.python.workspace:  File “/usr/lib/python2.7/unittest/case.py”, line 393, in <strong>call</strong><br>
WARNING:caffe2.python.workspace:  File “/usr/lib/python2.7/unittest/case.py”, line 329, in run<br>
WARNING:caffe2.python.workspace:  File “/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py”, line 44, in test_numa<br>
WARNING:caffe2.python.workspace:  File “/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py”, line 32, in build_test_net<br>
------------------------------ Captured log call -------------------------------<br>
WARNING  caffe2.python.workspace:workspace.py:218 Original python traceback for operator <code>2</code> in network <code>test_numa</code> in exception above (most recent call last):<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/lib/python2.7/runpy.py”, line 174, in _run_module_as_main<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pytest.py”, line 101, in <br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/config/<strong>init</strong>.py”, line 79, in main<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 242, in pytest_cmdline_main<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 205, in wrap_session<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 249, in _main<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/main.py”, line 270, in pytest_runtestloop<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 77, in pytest_runtest_protocol<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 92, in runtestprotocol<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 172, in call_and_report<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 197, in call_runtest_hook<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 225, in from_call<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 197, in <br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py”, line 289, in <strong>call</strong><br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 68, in _hookexec<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/manager.py”, line 62, in <br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/pluggy/callers.py”, line 187, in _multicall<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/runner.py”, line 122, in pytest_runtest_call<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/local/lib/python2.7/dist-packages/_pytest/unittest.py”, line 221, in runtest<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/lib/python2.7/unittest/case.py”, line 393, in <strong>call</strong><br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/usr/lib/python2.7/unittest/case.py”, line 329, in run<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py”, line 44, in test_numa<br>
WARNING  caffe2.python.workspace:workspace.py:223   File “/root/.local/lib/python2.7/site-packages/caffe2/python/numa_test.py”, line 32, in build_test_net<br>
------- generated xml file: /home/caffe2/caffe2_tests/python/result.xml --------<br>
---------- onnx coverage: ----------<br>
Operators (passed/loaded/total): 0/0/155</h2>
<p>Operator    Attributes<br>
(name: <span class="hashtag-raw">#values</span>)======= 1 failed, 392 passed, 7 skipped, 251 warnings in 492.91 seconds ========</p>