Token size

#60
by developerZoyal - opened

How to increase the token size and what is the name of the parameter for the token size?

@developerZoyal did you solved the error?, I am getting same one, and I am unable to figure it out so.

@Harsh6519 I have used the concept of loop. broke into chunk of 1024 tokens and then passed into the model

what tokenizer can be used for this model.

Sign up or log in to comment