Repository: pytorch
Issue: testConvnetBenchmarks intermittently segfaults
Labels: triaged,module: flaky-tests,better-engineering

Sample: https://ci.pytorch.org/jenkins/job/caffe2-builds/job/py2-clang7-rocmdeb-ubuntu16.04-test/10928/console

```
23:22:36 ============================= test session starts ==============================
23:22:36 platform linux2 -- Python 2.7.12, pytest-4.2.0, py-1.7.0, pluggy-0.8.1 -- /usr/bin/python2
23:22:36 cachedir: .pytest_cache
23:22:36 rootdir: /var/lib/jenkins, inifile:
23:22:36 plugins: sugar-0.9.2, hypothesis-3.59.0
23:22:47 collecting ... INFO:caffe2.python.net_drawer:Cannot import pydot, which is required for drawing a network. This can usually be installed in python with "pip install pydot". Also, pydot requires graphviz to convert dot files to pdf: in ubuntu, this can usually be installed with "sudo apt-get install graphviz".
23:22:53 net_drawer will not run correctly. Please install the correct dependencies.
23:22:53 collected 2395 items / 1 skipped / 2394 selected
23:22:53 
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/allcompare_test.py::TestAllCompare::test_allcompare PASSED [  0%]
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_arg_scope PASSED [  0%]
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_arg_scope_nested PASSED [  0%]
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_arg_scope_single PASSED [  0%]
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_cnn_model_helper_deprecated PASSED [  0%]
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_cond PASSED [  0%]
23:22:54 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_double_register PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_dropout PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_fc PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_get_params PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_has_helper PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_loop PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_model_helper PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_param_consistence PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_relu PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_tanh PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewTest::test_validate PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewGPUTest::test_relu PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/brew_test.py::BrewGPUTest::test_tanh PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/caffe_translator_test.py::TestNumericalEquivalence::testBlobs SKIPPED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/checkpoint_test.py::TestCheckpoint::test_ckpt_name_and_load_model_from_ckpts PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/checkpoint_test.py::TestCheckpoint::test_ckpt_save_failure PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/checkpoint_test.py::TestCheckpoint::test_download_group_simple PASSED [  0%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/checkpoint_test.py::TestCheckpoint::test_reuse_checkpoint_manager PASSED [  1%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/checkpoint_test.py::TestCheckpoint::test_single_checkpoint PASSED [  1%]
23:22:55 ../.local/lib/python2.7/site-packages/caffe2/python/checkpoint_test.py::TestCheckpoint::test_upload_checkpoint PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/context_test.py::TestContext::testDecorator PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/context_test.py::TestContext::testMultiThreaded PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_ops_grad_test.py::TestControl::test_disambiguate_grad_if_op_output PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testBoolNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testCombineConditions PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testDoUntilLoopWithNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testDoUntilLoopWithStep PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testDoWhileLoopWithNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testDoWhileLoopWithStep PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testForLoopWithNets PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testForLoopWithStep PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfCondFalseOnBlob PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfCondFalseOnNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfCondTrueOnBlob PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfCondTrueOnNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfElseCondFalseOnBlob PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfElseCondFalseOnNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfElseCondTrueOnBlob PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfElseCondTrueOnNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotCondFalseOnBlob PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotCondFalseOnNet PASSED [  1%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotCondTrueOnBlob PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotCondTrueOnNet PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotElseCondFalseOnBlob PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotElseCondFalseOnNet PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotElseCondTrueOnBlob PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testIfNotElseCondTrueOnNet PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testMergeConditionNets PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testSwitch PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testSwitchNot PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testUntilLoopWithNet PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testUntilLoopWithStep PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testWhileLoopWithNet PASSED [  2%]
23:22:56 ../.local/lib/python2.7/site-packages/caffe2/python/control_test.py::TestControl::testWhileLoopWithStep PASSED [  2%]
23:24:56 ../.local/lib/python2.7/site-packages/caffe2/python/convnet_benchmarks_test.py::TestConvnetBenchmarks::testConvnetBenchmarks ./.jenkins/caffe2/test.sh: line 118: 14948 Aborted                 (core dumped) "$PYTHON" -m pytest -x -v --disable-warnings --junit-xml="$pytest_reports_dir/result.xml" --ignore "$caffe2_pypath/python/test/executor_test.py" --ignore "$caffe2_pypath/python/operator_test/matmul_op_test.py" --ignore "$caffe2_pypath/python/operator_test/pack_ops_test.py" --ignore 
```

cc @bddppq @iotamudelta