Repository: keras-team/keras

Issue:
Keras documentation: multi_gpu_model examples do not show properly on the home page keras.io
Some examples of usage of `multi_gpu_model` appear on the documentation of the function in the [source code](https://github.com/keras-team/keras/blob/master/keras/utils/multi_gpu_utils.py).  However they do not display correctly on the [Keras home page](https://keras.io/utils/):

```Example 1 - Training models with weights merge on CPU

$Example_2_-_Training_models_with_weights_merge_on_CPU_using_cpu_relocation$0

Example 2 - Training models with weights merge on CPU using cpu_relocation

$Example_2_-_Training_models_with_weights_merge_on_CPU_using_cpu_relocation$1

Example 3 - Training models with weights merge on GPU (recommended for NV-link)

$Example_2_-_Training_models_with_weights_merge_on_CPU_using_cpu_relocation$2```
Keras documentation of multi_gpu_model: example 2 can be misleading 
In the Keras documentation for `multi_gpu_model`, it is stated:

> To save the multi-gpu model, use .save(fname) or .save_weights(fname) with the template model (the argument you passed to multi_gpu_model), rather than the model returned by multi_gpu_model.

However in example 2 the template model is overwritten by the multi-gpu model:

```python
         ..
         # Not needed to change the device scope for model definition:
         model = Xception(weights=None, ..)
         try:
             model = multi_gpu_model(model, cpu_relocation=True)
             print("Training using multiple GPUs..")
         except:
             print("Training using single GPU or CPU..")
         model.compile(..)
         ..
```

This means that in this example it would not be possible to save the weights of the template model. I suggest rewritting to something like:


```python
         ..
         # Not needed to change the device scope for model definition:
         model = Xception(weights=None, ..)
         try:
             parallel_model = multi_gpu_model(model, cpu_relocation=True)
             print("Training using multiple GPUs..")
         except ValueError:
             parallel_model = model
             print("Training using single GPU or CPU..")
         parallel_model.compile(..)
         ..
```

(I take this opportunity to except only a specific error)



Discussion:
Thanks for reporting it!

You are totally right. Thanks!

Resolving patch:
diff --git a/keras/utils/multi_gpu_utils.py b/keras/utils/multi_gpu_utils.py
--- a/keras/utils/multi_gpu_utils.py
+++ b/keras/utils/multi_gpu_utils.py
@@ -59,7 +59,9 @@ def multi_gpu_model(model, gpus=None, cpu_merge=True, cpu_relocation=False):
         A Keras `Model` instance which can be used just like the initial
         `model` argument, but which distributes its workload on multiple GPUs.
 
-    # Example 1 - Training models with weights merge on CPU
+    # Examples
+
+    Example 1 - Training models with weights merge on CPU
 
     ```python
         import tensorflow as tf
@@ -100,7 +102,7 @@ def multi_gpu_model(model, gpus=None, cpu_merge=True, cpu_relocation=False):
         model.save('my_model.h5')
     ```
 
-    # Example 2 - Training models with weights merge on CPU using cpu_relocation
+    Example 2 - Training models with weights merge on CPU using cpu_relocation
 
     ```python
          ..
@@ -108,16 +110,16 @@ def multi_gpu_model(model, gpus=None, cpu_merge=True, cpu_relocation=False):
          model = Xception(weights=None, ..)
 
          try:
-             model = multi_gpu_model(model, cpu_relocation=True)
+             parallel_model = multi_gpu_model(model, cpu_relocation=True)
              print("Training using multiple GPUs..")
-         except:
+         except ValueError:
+             parallel_model = model
              print("Training using single GPU or CPU..")
-
-         model.compile(..)
+         parallel_model.compile(..)
          ..
     ```
 
-    # Example 3 - Training models with weights merge on GPU (recommended for NV-link)
+    Example 3 - Training models with weights merge on GPU (recommended for NV-link)
 
     ```python
          ..
@@ -125,12 +127,13 @@ def multi_gpu_model(model, gpus=None, cpu_merge=True, cpu_relocation=False):
          model = Xception(weights=None, ..)
 
          try:
-             model = multi_gpu_model(model, cpu_merge=False)
+             parallel_model = multi_gpu_model(model, cpu_merge=False)
              print("Training using multiple GPUs..")
          except:
+             parallel_model = model
              print("Training using single GPU or CPU..")
 
-         model.compile(..)
+         parallel_model.compile(..)
          ..
     ```
 


Test patch: