fall back without padding if no colors

Signed-off-by: Alfredo Deza <alfredo.deza@inktank.com>