Repository: tensorflow
Issue: -D_GLIBCXX_USE_CXX11_ABI=1 increases a lot RAM usage
Labels: stat:awaiting tensorflower,type:build/install,subtype: ubuntu/linux,type:performance,TF 2.9

**System information**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
- TensorFlow installed from: pip
- TensorFlow version (use command below): 1.13.1
- Python version: python3.7
- CUDA/cuDNN version: 10.1
- GPU model and memory: GTX 1080 ti


**Describe the current behavior**

Since g++7 is now the default version on Ubuntu 18 and most distributions, most builds will use  `_GLIBCXX_USE_CXX11_ABI=1`. It seems also that when tensorflow is built with  `_GLIBCXX_USE_CXX11_ABI=0`, it implies recompiling all other libraries of the project with this flag which can be unconvenient. 

We noticed that building with `_GLIBCXX_USE_CXX11_ABI=1` increases the RAM by a lot. 

**Describe the expected behavior**

Both packages should consume the same amount of RAM.

**Code to reproduce the issue**

You can install tensorflow using `python3.7 -m pip install tensorflow==1.13.1` (related to https://github.com/tensorflow/tensorflow/issues/27078) and makes sure 
`python3.7 -c "import tensorflow; print(tensorflow.sysconfig.get_compile_flags())"` prints `-D_GLIBCXX_USE_CXX11_ABI=1`. 
Then you can install it in python3.6 `python3.6 -m pip install tensorflow==1.13.1` and make sure ` python3.6 -c "import tensorflow; print(tensorflow.sysconfig.get_compile_flags())"` prints `D_GLIBCXX_USE_CXX11_ABI=0`.

Now run this script with python3.6 and python3.7 and you will see that the second one consume a lot more (x3 on the model I use). Any `saved_model.pb` should work.

```python
import io
import os
import sys
try:
    from urllib import urlopen
except ImportError:
    from urllib.request import urlopen

import numpy
import psutil
from PIL import Image

import tensorflow as tf
from tensorflow.core.protobuf import saved_model_pb2
from tensorflow.python.platform import gfile
from tensorflow.python.util import compat

process = psutil.Process(os.getpid())

def print_ram(prefix=''):
    print("RAM", prefix, process.memory_info().rss / 1024. / 1024.)


if __name__ == '__main__':

    if len(sys.argv) == 1:
        model_filename = 'saved_model.pb'
    else:
        model_filename = sys.argv[1]

    with gfile.FastGFile(model_filename, 'rb') as f:
        data = compat.as_bytes(f.read())
        sm = saved_model_pb2.SavedModel()
        sm.ParseFromString(data)
        if 1 != len(sm.meta_graphs):
            print('More than one graph found. Not sure which to write')
            sys.exit(1)

    img_url = 'https://i.dailymail.co.uk/1s/2019/11/23/09/21370544-7717313-image-a-1_1574501083030.jpg'
    image_data = urlopen(img_url).read()
    decoded_data = numpy.array(Image.open(io.BytesIO(image_data)))
    decoded_data = numpy.expand_dims(decoded_data, axis=0)

    print_ram('before graph import')
    graph = tf.import_graph_def(sm.meta_graphs[0].graph_def)

    print_ram('before device')
    with tf.device("/device:GPU:0"):
        with tf.Session(graph=graph, config=None) as sess:
            print_ram('after session')
            output = sess.graph.get_tensor_by_name('import/predictions:0')
            print_ram('before run')
            for i in range(10000):
                results = sess.run(output, feed_dict={"import/image_tensor:0": decoded_data})
                print_ram('after run')
```

**Other info / logs**

python3.6:
```
RAM before graph import 527.55078125
RAM before device 856.59375
2019-12-16 17:51:27.400388: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-16 17:51:27.425979: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3399580000 Hz
2019-12-16 17:51:27.426637: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x229fee0 executing computations on platform Host. Devices:
2019-12-16 17:51:27.426655: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
RAM after session 860.25
RAM before run 860.25
RAM after run 948.12109375
RAM after run 998.0
RAM after run 1022.2265625
RAM after run 1038.984375
RAM after run 1038.984375
RAM after run 1056.0
RAM after run 1092.1953125
RAM after run 1097.09375
RAM after run 1097.09375
RAM after run 1097.09375
RAM after run 1097.09375
RAM after run 1097.09375
RAM after run 1116.42578125
RAM after run 1116.42578125
RAM after run 1116.42578125
RAM after run 1116.42578125
RAM after run 1116.42578125
```

python3.7:
```
RAM before graph import 510.33984375
RAM before device 752.87890625
2019-12-16 17:26:00.118658: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-16 17:26:00.141979: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3399580000 Hz
2019-12-16 17:26:00.142737: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x265aff0 executing computations on platform Host. Devices:
2019-12-16 17:26:00.142775: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
RAM after session 756.5625
RAM before run 756.5625
RAM after run 2977.8515625
RAM after run 3030.1640625
RAM after run 3047.8046875
RAM after run 3090.08203125
RAM after run 3121.296875
RAM after run 3122.0703125
RAM after run 3123.1015625
RAM after run 3123.1015625
RAM after run 3135.9921875
```