Repository: pytorch/vision

Issue:
Make initialization of GoogleNet / Inception faster
There have been several reports from users that GoogleNet and Inception are very slow to construct, see https://github.com/pytorch/vision/issues/1797 , https://github.com/pytorch/vision/issues/1977 and https://github.com/pytorch/vision/issues/2145 for example.

The underlying issue is that these models use `scipy.truncnorm`, for which the implementation was recently updated and became 100x slower than it was before, see https://github.com/scipy/scipy/issues/11299 for reference. This slowdown has been fixed in scipy and will be present in the 1.5.0 release, but in the meantime, users of torchvision still obtain very long startup times.

I think the simplest alternative is to make `init_weights` default to `False`, and use a weight initialization from PyTorch instead. This is BC-breaking for users who want to train Inception from scratch, but I'm not sure how much it will affect users in general.


Discussion:
Thanks for openning an seperate issue for this. As you said i was looking into the Scipy issue and trying to understand the behaviour of https://github.com/pytorch/vision/blob/f9ef235c402f48a335293c626e17bd8504d3af87/torchvision/models/inception.py#L103-L116.
I think weight initialization with PyTorch `nn.init` module can significantly improve the scenario. I have tested with 
```python
modules=[
        nn.Conv2d(3,512,2),
        nn.BatchNorm2d(512),
        nn.Linear(512,3) ]  X 17 times
```
for weight initialization, the current implementation takes around `1.21` sec, where the `nn.init.uniform_` API takes `0.0048` sec, I think it is far better until next scipy release.
```python
model=nn.Sequential(*modules)
def weight_init(m):
  if isinstance(m, nn.Conv2d) or isinstance(m, nn.Linear):
    nn.init.uniform_(m.weight,-2,2)
  elif isinstance(m, nn.BatchNorm2d):
    nn.init.constant_(m.weight, 1)
    nn.init.constant_(m.bias, 0)
model.apply(weight_init)
```
Would you mind if I work on it?
Hi,

Sure, please it would be great if you could work on it.

I think that we should still keep the old behavior if the user wants, and raise a warning to make users aware of it.

One option would be to change the default value of `init_weights` to be `None`, and raise a warning if it's None (forcing the users to be aware of it until they explicitly set the value to either True or False).
Hii,
> One option would be to change the default value of init_weights to be None, and raise a warning if it's None (forcing the users to be aware of it until they explicitly set the value to either True or False).

Sure. That would be better.
Thanks. 

Resolving patch:
diff --git a/torchvision/models/googlenet.py b/torchvision/models/googlenet.py
--- a/torchvision/models/googlenet.py
+++ b/torchvision/models/googlenet.py
@@ -62,11 +62,16 @@ def googlenet(pretrained=False, progress=True, **kwargs):
 class GoogLeNet(nn.Module):
     __constants__ = ['aux_logits', 'transform_input']
 
-    def __init__(self, num_classes=1000, aux_logits=True, transform_input=False, init_weights=True,
+    def __init__(self, num_classes=1000, aux_logits=True, transform_input=False, init_weights=None,
                  blocks=None):
         super(GoogLeNet, self).__init__()
         if blocks is None:
             blocks = [BasicConv2d, Inception, InceptionAux]
+        if init_weights is None:
+            warnings.warn('The default weight initialization of GoogleNet will be changed in future releases of '
+                          'torchvision. If you wish to keep the old behavior (which leads to long initialization times'
+                          ' due to scipy/scipy#11299), please set init_weights=True.', FutureWarning)
+            init_weights = True
         assert len(blocks) == 3
         conv_block = blocks[0]
         inception_block = blocks[1]
diff --git a/torchvision/models/inception.py b/torchvision/models/inception.py
--- a/torchvision/models/inception.py
+++ b/torchvision/models/inception.py
@@ -63,13 +63,18 @@ def inception_v3(pretrained=False, progress=True, **kwargs):
 class Inception3(nn.Module):
 
     def __init__(self, num_classes=1000, aux_logits=True, transform_input=False,
-                 inception_blocks=None, init_weights=True):
+                 inception_blocks=None, init_weights=None):
         super(Inception3, self).__init__()
         if inception_blocks is None:
             inception_blocks = [
                 BasicConv2d, InceptionA, InceptionB, InceptionC,
                 InceptionD, InceptionE, InceptionAux
             ]
+        if init_weights is None:
+            warnings.warn('The default weight initialization of inception_v3 will be changed in future releases of '
+                          'torchvision. If you wish to keep the old behavior (which leads to long initialization times'
+                          ' due to scipy/scipy#11299), please set init_weights=True.', FutureWarning)
+            init_weights = True
         assert len(inception_blocks) == 7
         conv_block = inception_blocks[0]
         inception_a = inception_blocks[1]


Test patch: