winglian
/

llama-3-1m-context-gradient-lora

Model card Files Files and versions Community

winglian commited on Apr 30, 2024

Commit

52b97c1

·

1 Parent(s): 32745ae

Create README.md

Files changed (1) hide show

README.md +3 -0

README.md ADDED Viewed

	@@ -0,0 +1,3 @@

+LoRA extraction from Gradient AI's https://huggingface.co/gradientai/Llama-3-8B-Instruct-Gradient-1048k model.
+LoRA extraction only targeted from the self_attn modules.
+Rank: 1024