Repository: keras-team/keras

Issue:
mnist_net2net.py  run error
When I run mnist_net2net.py, it is  "AssertionError: weight and bias from same layer should have compatible shapes" 




Incorrect weights' shape and questions about get_layer in example/mnist_net2net.py
In Keras 1: the shape of weight in Conv2D layer is of (filters,channels,height,weight), but in Keras 2: the shape is (kw,kh,num_channel,filters). I have fixed this bug, please ref `mnist_net2net_fixed.py` in gist:

https://gist.github.com/luzai/3336bd582650ff2c8c4ff97b1186afcd

Meanwhile, I find that `model.get_layer(name)` returns a `None` even if `model.layers` contain this layer. Please ref to `get_layer_bug.py` in gist.
Temporarily, I fix this bug by some tricks shown below, but may I ask why  `model.get_layer(name)` returns a `None`? 
```
if model.get_layer(name='conv2-deeper') is None:
	names2ind = {layer.name: ind for ind, layer in enumerate(model.layers)}
	ind = names2ind['conv2-deeper']
	model.layers[ind].set_weights(new_weights)
else:
	model.get_layer(name='conv2-deeper').set_weights(new_weights)
```



Discussion:
@qiyuqing I had the sane issue utilizing theano backend. Any solutions so far? 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity  occurs, but feel free to re-open a closed issue if needed.



Resolving patch:
diff --git a/examples/mnist_net2net.py b/examples/mnist_net2net.py
--- a/examples/mnist_net2net.py
+++ b/examples/mnist_net2net.py
@@ -29,12 +29,12 @@
 - Net2WiderNet experiment:
   + Student model has a wider Conv2D layer and a wider FC layer.
   + Comparison of 'random-padding' vs 'net2wider' weight initialization.
-  + With both methods, student model should immediately perform as well as
+  + With both methods, after 1 epoch, student model should perform as well as
     teacher model, but 'net2wider' is slightly better.
 - Net2DeeperNet experiment:
   + Student model has an extra Conv2D layer and an extra FC layer.
   + Comparison of 'random-init' vs 'net2deeper' weight initialization.
-  + Starting performance of 'net2deeper' is better than 'random-init'.
+  + After 1 epoch, performance of 'net2deeper' is better than 'random-init'.
 - Hyper-parameters:
   + SGD with momentum=0.9 is used for training teacher and student models.
   + Learning rate adjustment: it's suggested to reduce learning rate
@@ -44,14 +44,19 @@
     when a Dropout layer is used.
 
 Results
-- Tested with 'Theano' backend and 'channels_first' image_data_format.
-- Running on GPU GeForce GTX 980M
+- Tested with TF backend and 'channels_last' image_data_format.
+- Running on GPU GeForce GTX Titan X Maxwell
 - Performance Comparisons - validation loss values during first 3 epochs:
-(1) teacher_model:             0.075    0.041    0.041
-(2) wider_random_pad:          0.036    0.034    0.032
-(3) wider_net2wider:           0.032    0.030    0.030
-(4) deeper_random_init:        0.061    0.043    0.041
-(5) deeper_net2deeper:         0.032    0.031    0.029
+
+Experiment of Net2WiderNet ...
+(1) teacher_model:             0.0537   0.0354   0.0356
+(2) wider_random_pad:          0.0320   0.0317   0.0289
+(3) wider_net2wider:           0.0271   0.0274   0.0270
+
+Experiment of Net2DeeperNet ...
+(4) teacher_model:             0.0522   0.0386   0.0358
+(5) deeper_random_init:        0.0682   0.0506   0.0468
+(4) deeper_net2deeper:         0.0292   0.0294   0.0286
 '''
 
 from __future__ import print_function
@@ -104,35 +109,35 @@ def wider2net_conv2d(teacher_w1, teacher_b1, teacher_w2, new_width, init):
     '''
     assert teacher_w1.shape[0] == teacher_w2.shape[1], (
         'successive layers from teacher model should have compatible shapes')
-    assert teacher_w1.shape[0] == teacher_b1.shape[0], (
+    assert teacher_w1.shape[3] == teacher_b1.shape[0], (
         'weight and bias from same layer should have compatible shapes')
-    assert new_width > teacher_w1.shape[0], (
+    assert new_width > teacher_w1.shape[3], (
         'new width (filters) should be bigger than the existing one')
 
-    n = new_width - teacher_w1.shape[0]
+    n = new_width - teacher_w1.shape[3]
     if init == 'random-pad':
-        new_w1 = np.random.normal(0, 0.1, size=(n, ) + teacher_w1.shape[1:])
+        new_w1 = np.random.normal(0, 0.1, size=teacher_w1.shape[:3] + (n,))
         new_b1 = np.ones(n) * 0.1
-        new_w2 = np.random.normal(0, 0.1, size=(
-            teacher_w2.shape[0], n) + teacher_w2.shape[2:])
+        new_w2 = np.random.normal(0, 0.1,
+                                  size=teacher_w2.shape[:2] + (n, teacher_w2.shape[3]))
     elif init == 'net2wider':
-        index = np.random.randint(teacher_w1.shape[0], size=n)
+        index = np.random.randint(teacher_w1.shape[3], size=n)
         factors = np.bincount(index)[index] + 1.
-        new_w1 = teacher_w1[index, :, :, :]
+        new_w1 = teacher_w1[:, :, :, index]
         new_b1 = teacher_b1[index]
-        new_w2 = teacher_w2[:, index, :, :] / factors.reshape((1, -1, 1, 1))
+        new_w2 = teacher_w2[:, :, index, :] / factors.reshape((1, 1, -1, 1))
     else:
         raise ValueError('Unsupported weight initializer: %s' % init)
 
-    student_w1 = np.concatenate((teacher_w1, new_w1), axis=0)
+    student_w1 = np.concatenate((teacher_w1, new_w1), axis=3)
     if init == 'random-pad':
-        student_w2 = np.concatenate((teacher_w2, new_w2), axis=1)
+        student_w2 = np.concatenate((teacher_w2, new_w2), axis=2)
     elif init == 'net2wider':
         # add small noise to break symmetry, so that student model will have
         # full capacity later
         noise = np.random.normal(0, 5e-2 * new_w2.std(), size=new_w2.shape)
-        student_w2 = np.concatenate((teacher_w2, new_w2 + noise), axis=1)
-        student_w2[:, index, :, :] = new_w2
+        student_w2 = np.concatenate((teacher_w2, new_w2 + noise), axis=2)
+        student_w2[:, :, index, :] = new_w2
     student_b1 = np.concatenate((teacher_b1, new_b1), axis=0)
 
     return student_w1, student_b1, student_w2
@@ -193,12 +198,12 @@ def deeper2net_conv2d(teacher_w):
 
     # Arguments
         teacher_w: `weight` of previous conv2d layer,
-          of shape (filters, num_channel, kh, kw)
+          of shape (kh, kw, num_channel, filters)
     '''
-    filters, num_channel, kh, kw = teacher_w.shape
-    student_w = np.zeros((filters, filters, kh, kw))
+    kh, kw, num_channel, filters = teacher_w.shape
+    student_w = np.zeros_like(teacher_w)
     for i in range(filters):
-        student_w[i, i, (kh - 1) / 2, (kw - 1) / 2] = 1.
+        student_w[(kh - 1) / 2, (kw - 1) / 2, i, i] = 1.
     student_b = np.zeros(filters)
     return student_w, student_b
 


Test patch: