Maybe that could work, to get the right shape, but that would mean that we introduce a copy of the input data even when no padding is necessary.  I don't think this is such a good idea from a performance point of view.