Predictor in Streamlit Docker eating all memory and OOM
Author: Aklmenrah
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): DistilBert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Training on my datatset and creation of a model on Colab 
2. Saving the said model locally
3. Loading the predictor in my app, with Streamlit library
4. Deployment of said app in a Docker container

This is my app, which use a predictor with Streamlit. 

```
import pprint
import re

import ktrain
import numpy as np
import pandas as pd
import streamlit as st
import trafilatura
from googlesearch import search
from ktrain import text, predictor

st.title(':crystal_ball: schemaPredictor :crystal_ball:')

add_selectbox = st.selectbox(
    'How would you like to predict?',
    ('Text', 'Url'))

predictor = ktrain.load_predictor('./tmp/schema_mapping')


def get_prob(p):
    i = 0
    for x in p:
        if x > i:
            i = x
    return i


if add_selectbox == "Text":
    body = st.text_area('Insert your text here, as clean as possible.')
    if st.button("Predict"):
        st.success(":crystal_ball: " + predictor.predict(body) + " :crystal_ball:")
        st.success("With a probability of " + "{:.1%}".format(get_prob(predictor.predict_proba(body))))
        


elif add_selectbox == "Url":
    body = st.text_input('Insert your url here')
    if st.button("Predict"):
        page = body
        downloaded = trafilatura.fetch_url(page)
        result = trafilatura.extract(downloaded, include_tables=False, include_formatting=False,
                                     include_comments=False)
        st.success(":crystal_ball: " + predictor.predict(result) + " :crystal_ball:")
        st.success("With a probability of " + "{:.1%}".format(get_prob(predictor.predict_proba(result))))
```

## Expected behavior

If I run this app locally, without a Docker container, but in a conda env it works differently, it still takes memory at each iteration, but when it gets to around 10/11gb it frees memory to use it again. That on my 12gb ram laptop.
So I expected this to happen in my container too, but what happens in a container is that at each 'predict' it takes some memory, but it goes on untill it runs OOM. I tried with a Docker cointainer with 4 CPU, 12 gb of RAM and 1 gb SWAP

## Environment info
 Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
2020-07-09 11:56:21.400037: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-07-09 11:56:21.404641: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3800005000 Hz
2020-07-09 11:56:21.404943: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55dce5a7cca0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-09 11:56:21.405087: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-07-09 11:56:21.406403: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-07-09 11:56:21.406441: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
2020-07-09 11:56:21.406509: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (f8630e8d0e49): /proc/driver/nvidia/version does not exist

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 2.11.0
- Platform: Linux-4.19.76-linuxkit-x86_64-with-debian-10.4
- Python version: 3.7.7
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): 2.1.0 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

#

    

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.