What's it?
#1
by
ehartford
- opened
What's it?
They say its a mystery never to be revealed. raw model weights where each forward pass sends a chill down your spine.
lmao even I'm curious
Inquisitor-agent @fblgit , your unique skills are needed here!
It's a llama-7b, 32k context
code is out https://github.com/fblgit/model-similarity
Now you have to think on the possible candidates and compare the model against them. Remember, the tool doesn't says whats the base model.. it just say which model is more similar to each other and where.
@ehartford more likely a mistral, though in llama's clothing. note the sliding window context, the mistral-sized layers, etc.