Repository: pytorch
Issue: Memory leak issue with pytorch_java_only 1.9.0 and libtorch 1.9.0+cpu
Labels: module: memory usage,triaged,module: android,oncall: mobile,oncall: java,mobile_perf

## 🐛 Bug

I have a Java application which uses libtorch 1.9.0 to load a pre-built Pytorch module and returns a result of Module.forward to an another application. The application has a constant load about 3 requests per second. And every request just calls Module.forward on a preloaded model.

Every 3 days my application ruins due to out of memory in a docker container running the app (it has a limit of 4Gb of RAM to be used). The JVM running the app has Xmx option set to 1Gb, and a short analysis shows that the JVM doesn't use it fully. The app fails due to overflow of offheap memory of its JVM process.

I use libtorch downloaded from [the official site](https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-1.9.0%2Bcpu.zip) and pytorch_java_only 1.9.0 from Maven Central Repository.

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

I tried to isolate the issue and wrote this simple code, which just runs Module.forward in a loop from several threads. This code roughly emulates what my real application is doing:

```java
Module module = Module.load("./model.pt");
var pool = Executors.newFixedThreadPool(8);

for (int i = 0; i < 8; i++) {
    pool.submit(() -> {
        var bufsize = 256;
        var buffer = ByteBuffer.allocateDirect(bufsize * Long.BYTES)
            .order(ByteOrder.nativeOrder())
            .asLongBuffer();
        for (var j = 0; j < buffer.capacity(); j++) {
            buffer.put(j, j);
        }
        var data = Tensor.fromBlob(buffer, new long[]{1, 256});
        while (true) {
            module.forward(IValue.from(data));
        }
    });
}
```

While running this code I was watching for PSS memory increase of the process by calling the following bash command: 

```bash
time while true; do date +%R; cat /proc/19864/smaps | grep -i 'Pss:' | awk '{Total+=$2} END {print "  PSS: " Total}'; sleep 20; done
```
After 70 minutes minutes of waiting the command output showed that PSS was constantly increasing from 248071Kb to 546843Kb. Then I stopped observation, but seems like the PSS will continue increasing.

```bash
12:18
  PSS: 248071
12:18
  PSS: 346831
12:19
  PSS: 482363
12:19
  PSS: 486850
12:19
  PSS: 486439
...
12:48
  PSS: 534447
12:49
  PSS: 534491
12:49
  PSS: 534491
12:49
  PSS: 534515
...
13:28
  PSS: 546763
13:28
  PSS: 546819
13:28
  PSS: 546843

^C

real	70m48.965s
user	0m1.866s
sys	0m6.374s
```

For JVM I used -Xmx128m, and observation showed that used heap size wasn't higher than 85Mb.

Also I tried to create a separate Module instance for each thread, but the result is the same.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Expected behavior

I expect that there shouldn't be any memory leak issue in such simple case.

## Environment
 - Libtorch 1.9.0+cpu | pytorch_java_only (1.9.0)
 - OS: Ubuntu 21.04
 - Libtorch 1.9.0 was downloaded from [pytorch.org](https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-1.9.0%2Bcpu.zip), pytorch_java_only 1.9.0 - from [the Maven Central](https://mvnrepository.com/artifact/org.pytorch/pytorch_java_only/1.9.0).