Comparison time complexity between Convolution and Fully connected models

Am_colab: <p>hi<br>
I designed the following model, but the convolution structure (2) has a much smaller number of parameters<br>
But in terms of time complexity, it has more latency than the fully connected Foley model<br>
What is the reason for this ??<br>
If the model has fewer flops, the delay is less ??</p>
<p>How can I show that the convolution model has a much smaller number of flops, so the latency or time complexity is less?</p>
<p><strong>both model has equal latency time for inference . is reasonable ???</strong></p>
<p>model 1 : (Fully connected model)<br>
class modelss(nn.Module):</p>
<pre><code>def __init__(self):

    super(modelss, self).__init__()

    self.channel_num_in = 256

    self.encoder = nn.Sequential(

        nn.Linear(self.channel_num_in, 512),

        nn.Linear(512, 512),

        nn.Linear(512, 256),

        nn.Linear(256, 128),

        nn.Linear(128, 64),

        nn.Linear(64, 32),

        nn.Linear(32, 16),

    )

    self.fc = nn.Linear(16,7)

def forward(self, x):

    x = self.layer(x)

    x = x.view(x.size(0), -1)

    x=self.fc(x)

    return  x 
</code></pre>
<p>model = modelss()</p>
<hr>
<pre><code>    Layer (type)               Output Shape          Params           FLOPs           Madds
</code></pre>
<h1>
<a name="linear-1-2-512-131584-131072-261632-linear-2-2-512-262656-262144-523776-linear-3-2-256-131328-131072-261888-linear-4-2-128-32896-32768-65408-linear-5-2-64-8256-8192-16320-linear-6-2-32-2080-2048-4064-linear-7-2-16-528-512-1008-linear-8-2-7-119-112-217-1" class="anchor" href="#linear-1-2-512-131584-131072-261632-linear-2-2-512-262656-262144-523776-linear-3-2-256-131328-131072-261888-linear-4-2-128-32896-32768-65408-linear-5-2-64-8256-8192-16320-linear-6-2-32-2080-2048-4064-linear-7-2-16-528-512-1008-linear-8-2-7-119-112-217-1"></a>=========================================<br>
Linear-1                   [2, 512]         131,584         131,072         261,632<br>
Linear-2                   [2, 512]         262,656         262,144         523,776<br>
Linear-3                   [2, 256]         131,328         131,072         261,888<br>
Linear-4                   [2, 128]          32,896          32,768          65,408<br>
Linear-5                    [2, 64]           8,256           8,192          16,320<br>
Linear-6                    [2, 32]           2,080           2,048           4,064<br>
Linear-7                    [2, 16]             528             512           1,008<br>
Linear-8                     [2, 7]             119             112             217</h1>
<h2>
<a name="total-params-569447-trainable-params-569447-non-trainable-params-0-total-flops-567920-total-madds-1134313-2" class="anchor" href="#total-params-569447-trainable-params-569447-non-trainable-params-0-total-flops-567920-total-madds-1134313-2"></a>Total params: 569,447<br>
Trainable params: 569,447<br>
Non-trainable params: 0<br>
Total FLOPs: 567,920<br>
Total Madds: 1,134,313</h2>
<p>Input size (MB): 0.03<br>
Forward/backward pass size (MB): 0.01<br>
Params size (MB): 0.54<br>
Estimated Total Size (MB): 0.58<br>
FLOPs size (GB): 0.00<br>
Madds size (GB): 0.00</p>
<p>=========================================</p>
<p>model 2 : (Conv Model)</p>
<p>class ConvModel(nn.Module):</p>
<pre><code>def __init__(self):

    super(ConvModel, self).__init__()

    self.channel_num_in = 1

    self.layer = nn.Sequential(

        nn.Conv2d(self.channel_num_in, 16,2,2),

        nn.Conv2d(16, 8,2,2),

        nn.Conv2d(8, 4,2,2),

    )

    self.fc = nn.Linear(16,7)

def forward(self, x):

    x = self.layer(x)

    x = x.view(x.size(0), -1)

    x=self.fc(x)

    return  x 
</code></pre>
<p>model = ConvModel()</p>
<hr>
<pre><code>    Layer (type)               Output Shape          Params           FLOPs           Madds
</code></pre>
<h1>
<a name="conv2d-1-2-16-8-8-80-5120-8192-conv2d-2-2-8-4-4-520-8320-16384-conv2d-3-2-4-2-2-132-528-1024-linear-4-2-7-119-112-217-3" class="anchor" href="#conv2d-1-2-16-8-8-80-5120-8192-conv2d-2-2-8-4-4-520-8320-16384-conv2d-3-2-4-2-2-132-528-1024-linear-4-2-7-119-112-217-3"></a>=====================================================<br>
Conv2d-1              [2, 16, 8, 8]              80           5,120           8,192<br>
Conv2d-2               [2, 8, 4, 4]             520           8,320          16,384<br>
Conv2d-3               [2, 4, 2, 2]             132             528           1,024<br>
Linear-4                     [2, 7]             119             112             217</h1>
<h2>
<a name="total-params-851-trainable-params-851-non-trainable-params-0-total-flops-14080-total-madds-25817-4" class="anchor" href="#total-params-851-trainable-params-851-non-trainable-params-0-total-flops-14080-total-madds-25817-4"></a>Total params: 851<br>
Trainable params: 851<br>
Non-trainable params: 0<br>
Total FLOPs: 14,080<br>
Total Madds: 25,817</h2>
<p>Input size (MB): 0.03<br>
Forward/backward pass size (MB): 0.00<br>
Params size (MB): 0.00<br>
Estimated Total Size (MB): 0.04<br>
FLOPs size (GB): 0.00<br>
Madds size (GB): 0.00</p>
<p>==================================</p>
<h1>
<a name="this-code-used-for-calculate-inference-time-in-models-5" class="anchor" href="#this-code-used-for-calculate-inference-time-in-models-5"></a>this code used for calculate inference time in models…</h1>
<p><span class="hashtag">#model</span> = EfficientNet.from_pretrained(‘efficientnet-b0’)</p>
<p>device = torch.device(“cuda”)</p>
<p>model.to(device)</p>
<p>dummy_input = torch.randn(128, 1, 16,16,dtype=torch.float).to(device)</p>
<p>starter, ender = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)</p>
<p>repetitions = 300</p>
<p>timings=np.zeros((repetitions,1))</p>
<p><span class="hashtag">#GPU-WARM-UP</span></p>
<p>for _ in range(10):</p>
<p>_ = model(dummy_input)</p>
<h1>
<a name="measure-performance-6" class="anchor" href="#measure-performance-6"></a>MEASURE PERFORMANCE</h1>
<p>with torch.no_grad():</p>
<p>for rep in range(repetitions):</p>
<pre><code> starter.record()

 _ = model(dummy_input)

 ender.record()

 # WAIT FOR GPU SYNC

 torch.cuda.synchronize()

 curr_time = starter.elapsed_time(ender)

 timings[rep] = curr_time
</code></pre>
<p>mean_syn = np.sum(timings) / repetitions</p>
<p>std_syn = np.std(timings)</p>
<p>print(mean_syn)</p>