new implementation of the function tokens and add new functionality