`max_position_embeddings=32768` and `precompute_freqs_cis` with `end=128_000`
#6
by
Nadav-Timor
- opened
Hello!
First off, I'd like to express my appreciation for the great work shared here. I've been examining the implementation details, specifically concerning the positional encoding. There's an ongoing discussion on this topic, and I thought someone here might have insights or clarifications about it.
Thank you in advance!
Hi @Nadav-Timor The reply is here https://github.com/awslabs/extending-the-context-length-of-open-source-llms/issues/11 Thank you!
yinsong1986
changed discussion status to
closed