Repository: chainer/chainer

Issue:
Imagenet example failed to evaluate the model
I couldn't run imagenet example. It failed to evaluate the model.

```
Traceback (most recent call last):
  File "examples/imagenet/train_imagenet.py", line 166, in <module>
    main()
  File "examples/imagenet/train_imagenet.py", line 162, in main
    trainer.run()
  File "/home/unno/git/chainer/chainer/training/trainer.py", line 269, in run
    entry.extension(self)
  File "/home/unno/git/chainer/chainer/training/extensions/evaluator.py", line 134, in __call__
    result = self.evaluate()
  File "/home/unno/git/chainer/chainer/training/extensions/evaluator.py", line 169, in evaluate
    eval_func(*in_vars)
  File "/home/unno/git/chainer/examples/imagenet/nin.py", line 29, in __call__
    h = F.max_pooling_2d(F.relu(self.mlpconv1(x)), 3, stride=2)
  File "/home/unno/git/chainer/chainer/links/connection/mlp_convolution_2d.py", line 73, in __call__
    x = f(l(x))
  File "/home/unno/git/chainer/chainer/links/connection/convolution_2d.py", line 101, in __call__
    x, self.W, self.b, self.stride, self.pad, self.use_cudnn)
AttributeError: 'Convolution2D' object has no attribute 'W'
```



Discussion:
I think it is caused by `Link.copy` not handling the uninitialized parameters. The image net example currently copies the model before running the training to run evaluations in test mode. In order to fix it, we have to make `Link.copy` also copy the information of uninitialized parameters (`_uninitialized_params`) and catch up with the actual initialization of these variables. One solution is to add a reference that points to the original link as an attribute of the copied link, and use this reference to check in `add_params` whether the original parameter is already initialized, and if so use a shallow copy of the initialized parameter instead of initializing a new one.

This problem seems to be essentially same as #1615.

How about to abstract parameters to have the states of initialized and uninitialized?

```
original_link - W attribute -----> parameter (initialized or uninitialized)
                              |
copied_link - W attribute ----+
```

Alghtough this change may have effect to the internal implementation of links that already support uninitialized parameters, we should be able to keep external APIs of `Link` and such links.

To hide the extra abstraction layer when accessing link attributes, e.g. `link.W` and `link.W = some_array`, we may need to implement getter and setter methods to `Link`.

Ah-, I find that shared is only Variable's data and parameter Variables are independent, it is more complicated than what I wrote, where Variables would need (shape-)uninitialized state.


Resolving patch:
diff --git a/examples/imagenet/train_imagenet.py b/examples/imagenet/train_imagenet.py
--- a/examples/imagenet/train_imagenet.py
+++ b/examples/imagenet/train_imagenet.py
@@ -65,6 +65,16 @@ def get_example(self, i):
         return image, label
 
 
+class TestModeEvaluator(extensions.Evaluator):
+
+    def evaluate(self):
+        model = self.get_target('main')
+        model.train = False
+        ret = super(TestModeEvaluator, self).evaluate()
+        model.train = True
+        return ret
+
+
 def main():
     archs = {
         'alex': alex.Alex,
@@ -136,11 +146,7 @@ def main():
     val_interval = (10 if args.test else 100000), 'iteration'
     log_interval = (10 if args.test else 1000), 'iteration'
 
-    # Copy the chain with shared parameters to flip 'train' flag only in test
-    eval_model = model.copy()
-    eval_model.train = False
-
-    trainer.extend(extensions.Evaluator(val_iter, eval_model, device=args.gpu),
+    trainer.extend(TestModeEvaluator(val_iter, model, device=args.gpu),
                    trigger=val_interval)
     trainer.extend(extensions.dump_graph('main/loss'))
     trainer.extend(extensions.snapshot(), trigger=val_interval)


Test patch: