Maximum Chunk Size for RAG
#27
by
mox
- opened
What would be the maximum Chunk Size that I can use with this embedding model, if I want to split up my documents into chunks for RAG?
It would be 512 tokens.
Hi, I have a follow up question. What is the expected behaviour when the passed text is longer than 512 tokens? I assume it gets cut off at 512.