Repository: pytorch/TensorRT

Issue:
🐛 [Bug] Memory leak in ConversionCtx::SerializeEngine()
##  Bug Description
Using `trtorch::CompileGraph` many times will increase memory usage.

## To Reproduce
[Source code](https://github.com/SakodaShintaro/trtorch-test/blob/7e9f7a457b3570b40e5aa938ac400aaf5e0028ba/main.cpp)

```cpp
#include <torch/script.h>
#include <trtorch/trtorch.h>
using namespace std;

void compile() {
  constexpr int64_t INPUT_CHANNEL_NUM = 256;
  constexpr int64_t WIDTH = 32;
  torch::jit::Module module = torch::jit::load("model.ts");
  module.to(torch::kCUDA);
  module.eval();

  std::vector<int64_t> in_sizes = {1, INPUT_CHANNEL_NUM, WIDTH, WIDTH};
  trtorch::CompileSpec::InputRange range(in_sizes);
  trtorch::CompileSpec info({range});
  module = trtorch::CompileGraph(module, info);
}

int main() {
  for (int64_t i = 0; i < 10000; i++) {
    cout << i << endl;
    compile();
  }
}
```

[Dockerfile](https://github.com/SakodaShintaro/trtorch-test/blob/7e9f7a457b3570b40e5aa938ac400aaf5e0028ba/docker/Dockerfile)

## What seems to be the cause
When I verified it with Valgrind, it seemed that the cause was ConversionCtx::SerializeEngine. Looking at the code, it seemed that the `serialized_engine` isn't released. After making the following modifications, the memory leak no longer occurs.

```
diff --git a/core/conversion/conversionctx/ConversionCtx.cpp b/core/conversion/conversionctx/ConversionCtx.cpp
index ff23692..c241f9c 100644
--- a/core/conversion/conversionctx/ConversionCtx.cpp
+++ b/core/conversion/conversionctx/ConversionCtx.cpp
@@ -147,7 +147,9 @@ std::string ConversionCtx::SerializeEngine() {
   auto engine = builder->buildEngineWithConfig(*net, *cfg);
   auto serialized_engine = engine->serialize();
   engine->destroy();
-  return std::string((const char*)serialized_engine->data(), serialized_engine->size());
+  std::string result = std::string((const char*)serialized_engine->data(), serialized_engine->size());
+  serialized_engine->destroy();
+  return result;
 }
 ```

I would appreciate it if you could respond appropriately.


Discussion:

Resolving patch:
diff --git a/py/trtorch/_compiler.py b/py/trtorch/_compiler.py
--- a/py/trtorch/_compiler.py
+++ b/py/trtorch/_compiler.py
@@ -157,5 +157,6 @@ def get_build_info() -> str:
     build_info = "TRTorch Version: " + str(__version__) + '\n' + build_info
     return build_info
 
+
 def set_device(gpu_id):
     trtorch._C.set_device(gpu_id)


Test patch:
diff --git a/tests/py/test_multi_gpu.py b/tests/py/test_multi_gpu.py
--- a/tests/py/test_multi_gpu.py
+++ b/tests/py/test_multi_gpu.py
@@ -5,7 +5,9 @@
 
 from model_test_case import ModelTestCase
 
+
 class TestMultiGpuSwitching(ModelTestCase):
+
     def setUp(self):
         if torch.cuda.device_count() < 2:
             self.fail("Test is not relevant for this platform since number of available CUDA devices is less than 2")
@@ -55,12 +57,14 @@ def test_compile_script(self):
         trtorch.set_device(0)
         self.assertTrue(same < 2e-3)
 
+
 def test_suite():
     suite = unittest.TestSuite()
     suite.addTest(TestMultiGpuSwitching.parametrize(TestMultiGpuSwitching, model=models.resnet18(pretrained=True)))
 
     return suite
 
+
 suite = test_suite()
 
 runner = unittest.TextTestRunner()