Undi95

/

Meta-Llama-3-70B-Instruct-hf

like 15

https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct/blob/main/config.json
It's still not in the OG repo so for now I will not modify it as I want to be 1:1, but thank you!

For me it shows up:

For me it shows up:

Ah yeah for me too, I looked at the wrong file, I do that then

@Undi95
You could try the ortho script on llama-2-13b-chat with a dataset of 32 and 512 and then see if there's a difference.

@Undi95
You could try the ortho script on llama-2-13b-chat with a dataset of 32 and 512 and then see if there's a difference.

Yeah, into small model I go back I guess, I made a post explaining the situation lmao