Fix overzealous assert for all-SCALAR case