  Deeper and wider CNNs are known to provide improved performance for deep
learning tasks. However, most such networks have poor performance gain per
parameter increase. In this paper, we investigate whether the gain observed in
deeper models is purely due to the addition of more optimization parameters or
whether the physical size of the network as well plays a role. Further, we
present a novel rescaling strategy for CNNs based on learnable repetition of
its parameters. Based on this strategy, we rescale CNNs without changing their
parameter count, and show that learnable sharing of weights itself can provide
significant boost in the performance of any given model without changing its
parameter count. We show that small base networks when rescaled, can provide
performance comparable to deeper networks with as low as 6% of optimization
parameters of the deeper one.
  The relevance of weight sharing is further highlighted through the example of
group-equivariant CNNs. We show that the significant improvements obtained with
group-equivariant CNNs over the regular CNNs on classification problems are
only partly due to the added equivariance property, and part of it comes from
the learnable repetition of network weights. For rot-MNIST dataset, we show
that up to 40% of the relative gain reported by state-of-the-art methods for
rotation equivariance could actually be due to just the learnt repetition of
weights.
