Problem in understanding densenet model

Shraban_Chatterjee: <p>I have a densenet121 model from pytorch<br>
DenseNet(<br>
(features): Sequential(<br>
(conv0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)<br>
(norm0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu0): ReLU(inplace=True)<br>
(pool0): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)<br>
(denseblock1): _DenseBlock(<br>
(denselayer1): _DenseLayer(<br>
(norm1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer2): _DenseLayer(<br>
(norm1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(96, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer3): _DenseLayer(<br>
(norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer4): _DenseLayer(<br>
(norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer5): _DenseLayer(<br>
(norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer6): _DenseLayer(<br>
(norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
)<br>
(transition1): _Transition(<br>
(norm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)<br>
)<br>
(denseblock2): _DenseBlock(<br>
(denselayer1): _DenseLayer(<br>
(norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer2): _DenseLayer(<br>
(norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer3): _DenseLayer(<br>
(norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer4): _DenseLayer(<br>
(norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer5): _DenseLayer(<br>
(norm1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer6): _DenseLayer(<br>
(norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(288, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer7): _DenseLayer(<br>
(norm1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(320, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer8): _DenseLayer(<br>
(norm1): BatchNorm2d(352, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(352, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer9): _DenseLayer(<br>
(norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer10): _DenseLayer(<br>
(norm1): BatchNorm2d(416, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer11): _DenseLayer(<br>
(norm1): BatchNorm2d(448, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(448, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer12): _DenseLayer(<br>
(norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(480, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
)<br>
(transition2): _Transition(<br>
(norm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)<br>
)<br>
(denseblock3): _DenseBlock(<br>
(denselayer1): _DenseLayer(<br>
(norm1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer2): _DenseLayer(<br>
(norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(288, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer3): _DenseLayer(<br>
(norm1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(320, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer4): _DenseLayer(<br>
(norm1): BatchNorm2d(352, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(352, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer5): _DenseLayer(<br>
(norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer6): _DenseLayer(<br>
(norm1): BatchNorm2d(416, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer7): _DenseLayer(<br>
(norm1): BatchNorm2d(448, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(448, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer8): _DenseLayer(<br>
(norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(480, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer9): _DenseLayer(<br>
(norm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer10): _DenseLayer(<br>
(norm1): BatchNorm2d(544, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(544, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer11): _DenseLayer(<br>
(norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(576, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer12): _DenseLayer(<br>
(norm1): BatchNorm2d(608, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(608, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer13): _DenseLayer(<br>
(norm1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(640, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer14): _DenseLayer(<br>
(norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(672, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer15): _DenseLayer(<br>
(norm1): BatchNorm2d(704, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(704, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer16): _DenseLayer(<br>
(norm1): BatchNorm2d(736, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(736, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer17): _DenseLayer(<br>
(norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer18): _DenseLayer(<br>
(norm1): BatchNorm2d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(800, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer19): _DenseLayer(<br>
(norm1): BatchNorm2d(832, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer20): _DenseLayer(<br>
(norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(864, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer21): _DenseLayer(<br>
(norm1): BatchNorm2d(896, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(896, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer22): _DenseLayer(<br>
(norm1): BatchNorm2d(928, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(928, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer23): _DenseLayer(<br>
(norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(960, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer24): _DenseLayer(<br>
(norm1): BatchNorm2d(992, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(992, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
)<br>
(transition3): _Transition(<br>
(norm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu): ReLU(inplace=True)<br>
(conv): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)<br>
)<br>
(denseblock4): _DenseBlock(<br>
(denselayer1): _DenseLayer(<br>
(norm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer2): _DenseLayer(<br>
(norm1): BatchNorm2d(544, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(544, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer3): _DenseLayer(<br>
(norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(576, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer4): _DenseLayer(<br>
(norm1): BatchNorm2d(608, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(608, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer5): _DenseLayer(<br>
(norm1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(640, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer6): _DenseLayer(<br>
(norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(672, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer7): _DenseLayer(<br>
(norm1): BatchNorm2d(704, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(704, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer8): _DenseLayer(<br>
(norm1): BatchNorm2d(736, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(736, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer9): _DenseLayer(<br>
(norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer10): _DenseLayer(<br>
(norm1): BatchNorm2d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(800, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer11): _DenseLayer(<br>
(norm1): BatchNorm2d(832, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer12): _DenseLayer(<br>
(norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(864, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer13): _DenseLayer(<br>
(norm1): BatchNorm2d(896, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(896, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer14): _DenseLayer(<br>
(norm1): BatchNorm2d(928, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(928, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer15): _DenseLayer(<br>
(norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(960, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
(denselayer16): _DenseLayer(<br>
(norm1): BatchNorm2d(992, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu1): ReLU(inplace=True)<br>
(conv1): Conv2d(992, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)<br>
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
(relu2): ReLU(inplace=True)<br>
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)<br>
)<br>
)<br>
(norm5): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)<br>
)<br>
(classifier): Linear(in_features=1024, out_features=6, bias=True)<br>
)<br>
When I run the model, it works fine.</p>
<p>Case1: <strong>What works is:</strong><br>
model.classifier = Linear(in_features=1024, out_features=1024, bias=True)<br>
out = model(input)</p>
<p>Case2: <strong>What does not work is:</strong><br>
f = model.features(input) # f.shape = torch.Size([1, 1024, 7, 7])<br>
linear = torch.nn.Linear(in_features=1024, out_features=1024, bias=True)<br>
output = linear(f)</p>
<p>input is a single image of shape 224*224 with three channels</p>
<p>My question is that when I extract the features from ‘features’ (extracted feature has shape 1, 1024, 7, 7) and pass it to a self-created Linear layer, it throws an error( ERROR: mat1 and mat2 shapes cannot be multiplied (14336x7 and 1024x7)). But, when I replace the model ‘classifier’ with the same linear layer, I get an output vector of size [1,1024].</p>
<p>To me, both of the operations are the same, but I keep getting the error in the second case.<br>
Please let me know where I am going wrong.</p>

ptrblck: <aside class="quote no-group" data-username="Shraban_Chatterjee" data-post="1" data-topic="184837">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/shraban_chatterjee/48/55061_2.png" class="avatar"> Shraban_Chatterjee:</div>
<blockquote>
<p>To me, both of the operations are the same</p>
</blockquote>
</aside>
<p>They are not the same since you are missing <a href="https://github.com/pytorch/vision/blob/29418e34a94e2c43f861a321265f7f21035e7b19/torchvision/models/densenet.py#L215-L216">these functional API calls</a>.</p>

J_Johnson: <p>Your model print suggests:</p>
<p>inputs → features → transition 1 → transition 2 → transition 3 → norm5 → classifier</p>
<p>But sounds like you tried:</p>
<p>inputs → features → classifier</p>
<p>Granted, your print statement says nothing about the actual flow of data in the model. We would need to see the forward method for that.</p>

Shraban_Chatterjee: <p>Thanks a lot. I should have looked into the forward method myself.</p>

Shraban_Chatterjee: <p>Thanks a lot. The forward method contains three more steps before performing classification.</p>