added diagonal and up/down directions to tokenize