What is the context size for Gemma? I get error when asking for it in the config file e.g., AttributeError("'GemmaConfig' object has no attribute 'context_length'")

#32

by brando - opened Mar 7

Mar 7

What is the context size for Gemma? I get error when asking for it in the config file e.g., AttributeError("'GemmaConfig' object has no attribute 'context_length'")

ybelkada

Mar 13

•

edited Mar 13

Hi ! I think it should be 8192: https://huggingface.co/google/gemma-2b/blob/main/config.json#L14

ybelkada

Mar 13

This has been confirmed by the author here: https://huggingface.co/google/gemma-7b-it/discussions/73#65e9678c0cda621164a95bad

lkv

Google org about 8 hours ago

Hi @brando , The 2B model actually uses MQA (just 1 KV head), whereas the 7B uses MHA. Both models have the same sequence/context length of 8192, as specified in the technical report. please find this report for reference. Thank you.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment