Repository: pytorch
Issue: Segmentation fault (core dumped) when running optimize_for_mobile
Labels: oncall: mobile

## 🐛 Bug: Segmentation fault (core dumped) when running optimize_for_mobile on a toy model

Seeing core dumps when running optimize_for_mobile on a quantized model which as nn.Sequentials, such as this one.  Happens on master.

```
class Child(nn.Module):                                 
    def __init__(self):                                 
        super(Child, self).__init__()                   
        self.conv1 = nn.Conv2d(1, 1, 1)                 
                                                        
    def forward(self, x):                               
        x = self.conv1(x)                               
        return x                                        
                                                        
class Parent(nn.Module):                                
    def __init__(self):                                 
        super(Parent, self).__init__()                  
        self.quant = torch.quantization.QuantStub()     
        self.conv1 = nn.Conv2d(1, 1, 1)                 
        self.child = Child()                            
        self.child2 = nn.Sequential(                    
            Child(),                                    
            Child(),                                    
        )                                               
        self.dequant = torch.quantization.DeQuantStub() 
                                                        
    def forward(self, x):                               
        x = self.quant(x)                               
        x = self.conv1(x)                               
        x = self.child(x)                               
        x = self.child2(x)                              
        x = self.dequant(x)                             
        return x                                        
```

## To Reproduce

Steps to reproduce the behavior:

1. run this script: https://gist.github.com/vkuzo/9589c61b03e662bd8f82874006fe2a5c

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

output of script: Segmentation fault (core dumped)

## Expected behavior

should be able to run data through model after the optimize_for_mobile pass

## Environment

```
Collecting environment information...                                              
PyTorch version: 1.7.0a0+42a0b51                                                   
Is debug build: No                                                                 
CUDA used to build PyTorch: Could not collect                                      
                                                                                   
OS: CentOS Linux 7 (Core)                                                          
GCC version: (GCC) 7.3.1 20180303 (Red Hat 7.3.1-5)                                
CMake version: version 3.14.0                                                      
                                                                                   
Python version: 3.8                                                                
Is CUDA available: No                                                              
CUDA runtime version: Could not collect                                            
GPU models and configuration:                                                      
GPU 0: Tesla P100-SXM2-16GB                                                        
GPU 1: Tesla P100-SXM2-16GB                                                        
GPU 2: Tesla P100-SXM2-16GB                                                        
GPU 3: Tesla P100-SXM2-16GB                                                        
GPU 4: Tesla P100-SXM2-16GB                                                        
GPU 5: Tesla P100-SXM2-16GB                                                        
GPU 6: Tesla P100-SXM2-16GB                                                        
GPU 7: Tesla P100-SXM2-16GB                                                        
                                                                                   
Nvidia driver version: 396.69                                                      
cuDNN version: Could not collect                                                   
                                                                                   
Versions of relevant libraries:                                                    
[pip] numpy==1.18.1                                                                
[pip] torch==1.7.0a0+42a0b51                                                       
[pip] torchvision==0.8.0a0+4433a5b                                                 
[conda] blas                      1.0                         mkl                  
[conda] magma-cuda90              2.5.0                         1    pytorch       
[conda] magma-cuda92              2.5.2                         1    pytorch       
[conda] mkl                       2020.0                      166                  
[conda] mkl-include               2020.0                      166                  
[conda] mkl-service               2.3.0            py38he904b0f_0                  
[conda] mkl_fft                   1.0.15           py38ha843d7b_0                  
[conda] mkl_random                1.1.0            py38h962f231_0                  
[conda] numpy                     1.18.1           py38h4f9e942_0                  
[conda] numpy-base                1.18.1           py38hde5b4d6_1                  
[conda] torch                     1.7.0a0+42a0b51           dev_0    <develop>     
[conda] torchvision               0.8.0a0+4433a5b           dev_0    <develop>     
```

## Additional context

<!-- Add any other context about the problem here. -->