updated token encoding and tests