metadata
extra_gated_heading: Access Llama 2 on Hugging Face
extra_gated_description: >-
This is a form to enable access to Llama 2 on Hugging Face after you have been
granted access from Meta. Please visit the [Meta
website](https://ai.meta.com/resources/models-and-libraries/llama-downloads)
and accept our license terms and acceptable use policy before submitting this
form. Requests will be processed in 1-2 days.
extra_gated_prompt: >-
**Your Hugging Face account email address MUST match the email you provide on
the Meta website, or your request will not be approved.**
extra_gated_button_content: Submit
extra_gated_fields:
I agree to share my name, email address and username with Meta and confirm that I have already been granted download access on the Meta website: checkbox
language:
- en
pipeline_tag: text-generation
inference: false
tags:
- facebook
- meta
- pytorch
- llama
- llama-2
This is 82M parameters llama model of random weights. This model can be use for proof of concept.
Tokenizer is copy of meta-llama/Llama-2-7b
# Use a pipeline as a high-level helper
from transformers import LlamaConfig, LlamaForCausalLM, LlamaTokenizer
import numpy as np
config = LlamaConfig(vocab_size=32000, hidden_size=768, intermediate_size=768*4, num_hidden_layers=4, num_attention_heads=8)
tokenizer = LlamaTokenizer.from_pretrained("meta-llama/Llama-2-7b")
model = LlamaForCausalLM(config).half()
model_parameters = filter(lambda p: p.requires_grad, model.parameters())
params = sum([np.prod(p.size()) for p in model_parameters])
print(params / 1024 / 1024) # 82.881591796875
hub_id = "heegyu/llama-small-randomweights"
tokenizer.push_to_hub(hub_id)
model.push_to_hub(hub_id)