torch.longTensor recast
Hello,
I was looking through the model card and in the quick use code example I noticed in the generate_text function something that caught my curiosity.
After the tokenization of the input prompt intended to be going to the model in GPU("tokens = tokenizer.encode(instruction)"), the tokens are recast as longTensors (64-bit signed interreges tensors recast at "tokens = torch.LongTensor(tokens).unsqueeze(0)").
I've not seen a lot of others doing this with what I've seen so far in my Ai journey. I was curious as to what the reasoning why. I can speculate several reasons why. I figure it wouldn't hurt if I ask directly. I'll also be looking online. As well as checking with migtissera who posted the non 16k version of the model. (I've noticed the recast was done in that models card as well.)
If you can, could you point me at any material I can look at that further supports why it's smart to recast a tensor before sending off to the GPU, that would be appreciated. If you can't or do not want to, that is understandable.
Thank you for your time.