RichardErkhov commited on
Commit
d8f59a0
1 Parent(s): 4832ee7

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +78 -0
README.md ADDED
@@ -0,0 +1,78 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ Llama-2-7b-evolcodealpaca - bnb 4bits
11
+ - Model creator: https://huggingface.co/neuralmagic/
12
+ - Original model: https://huggingface.co/neuralmagic/Llama-2-7b-evolcodealpaca/
13
+
14
+
15
+
16
+
17
+ Original model description:
18
+ ---
19
+ base_model: meta-llama/Llama-2-7b-hf
20
+ inference: true
21
+ model_type: llama
22
+ pipeline_tag: text-generation
23
+ datasets:
24
+ - theblackcat102/evol-codealpaca-v1
25
+ tags:
26
+ - code
27
+ ---
28
+
29
+ # Llama-2-7b-evolcodealpaca
30
+
31
+ This repo contains a [Llama 2 7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) finetuned for code generation tasks using the [Evolved CodeAlpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1) dataset.
32
+
33
+ Official model weights from [Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment](https://arxiv.org/abs/2405.03594).
34
+
35
+ **Authors**: Neural Magic, Cerebras
36
+
37
+ ## Usage
38
+
39
+ Below we share some code snippets on how to get quickly started with running the model.
40
+
41
+ ### Sparse Transfer
42
+
43
+ By leveraging a pre-sparsified model's structure, you can efficiently fine-tune on new data, leading to reduced hyperparameter tuning, training times, and computational costs. Learn about this process [here](https://neuralmagic.github.io/docs-v2/get-started/transfer).
44
+
45
+ ### Running the model
46
+
47
+ This model may be run with the transformers library. For accelerated inference with sparsity, deploy with [nm-vllm](https://github.com/neuralmagic/nm-vllm) or [deepsparse](https://github.com/neuralmagic/deepsparse).
48
+
49
+ ```python
50
+ # pip install transformers accelerate
51
+ from transformers import AutoTokenizer, AutoModelForCausalLM
52
+
53
+ tokenizer = AutoTokenizer.from_pretrained("neuralmagic/Llama-2-7b-evolcodealpaca")
54
+ model = AutoModelForCausalLM.from_pretrained("neuralmagic/Llama-2-7b-evolcodealpaca", device_map="auto")
55
+
56
+ input_text = "def fibonacci(n):\n"
57
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
58
+
59
+ outputs = model.generate(**input_ids)
60
+ print(tokenizer.decode(outputs[0]))
61
+ ```
62
+
63
+ ## Evaluation Benchmark Results
64
+
65
+ Model evaluation metrics and results.
66
+
67
+ | Benchmark | Metric | Llama-2-7b-evolcodealpaca |
68
+ |------------------------------------------------|---------------|-------------|
69
+ | [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 | 32.03 |
70
+
71
+ ## Model Training Details
72
+
73
+ Coming soon.
74
+
75
+ ## Help
76
+
77
+ For further support, and discussions on these models and AI in general, join [Neural Magic's Slack Community](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ)
78
+