Chunk sizes and overlaps
#2
by
Marcoindahouse
- opened
Hey, did anybody find an optimal size for this model?
any news?
Do you mean an optimal size in terms of sequence length & how to truncate? I'd go with truncating after 300 tokens