added check for not null tokens