Wouldn't it be interesting to try training weights?
This model might not work right away, but for what is essentially a complete joke, it remarkably possesses all the necessary prerequisites to function as an LLM. Naturally, additional pre-training would be required—but it would certainly be fun to see whether it proves more efficient than a model with random initialization!
would be a nice experiment if it could even run on anything at all
Moreover, if this were a complete fake, most people would likely either turn a blind eye or lose interest, resulting in very little engagement. This suggests that these model weights might contain something inconvenient for those currently resisting them so fiercely. In my view, this "imperfection" simply means these are model weights that were discarded because they were incomplete Pre-training though, depending on how they were trained, they might have had the potential to approach the performance of Claude (setting aside the issue of cost). The real question is at what stage they were abandoned. Unless someone performs a small-scale training run using the released dataset and examines the loss, I suspect that claims—seen elsewhere—that these are merely "random weights" are likely coming from suspicious bots. By the way, are there any Anthropic employees present here on Hugging Face?
Moreover, if this were a complete fake, most people would likely either turn a blind eye or lose interest, resulting in very little engagement. This suggests that these model weights might contain something inconvenient for those currently resisting them so fiercely. In my view, this "imperfection" simply means these are model weights that were discarded because they were incomplete Pre-training though, depending on how they were trained, they might have had the potential to approach the performance of Claude (setting aside the issue of cost). The real question is at what stage they were abandoned. Unless someone performs a small-scale training run using the released dataset and examines the loss, I suspect that claims—seen elsewhere—that these are merely "random weights" are likely coming from suspicious bots. By the way, are there any Anthropic employees present here on Hugging Face?
maybe, i mean i dont think anthropic would go as far as getting a bot swarm to discredit the leak (if its real) but your explanation seems pretty plausible, but it is a bit convenient that everyone coming across this model cant run it at all, thats like the single only defense preventing this from being properly verified
Maybe all according to keikaku.
Everyone is making way too much of a fuss—as if they’re dealing with some nuisance they want to get rid of as quickly as possible.