Do not force input_sizes to be set if it is not necessary

- this is just a quick hack

Signed-off-by: Stefan Marr <git@stefan-marr.de>