afrideva commited on
Commit
90f371a
1 Parent(s): 6e27f45

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -1
README.md CHANGED
@@ -33,7 +33,45 @@ Quantized GGUF model files for [airoboros-3b-3p0](https://huggingface.co/jondurb
33
  | [airoboros-3b-3p0.q6_k.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q6_k.gguf) | q6_k | 2.30 GB |
34
  | [airoboros-3b-3p0.q8_0.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q8_0.gguf) | q8_0 | 2.97 GB |
35
 
36
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
 
38
  ## Original Model Card:
39
  ### Overview
 
33
  | [airoboros-3b-3p0.q6_k.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q6_k.gguf) | q6_k | 2.30 GB |
34
  | [airoboros-3b-3p0.q8_0.gguf](https://huggingface.co/afrideva/airoboros-3b-3p0-GGUF/resolve/main/airoboros-3b-3p0.q8_0.gguf) | q8_0 | 2.97 GB |
35
 
36
+ # Test Run
37
+ `./llama.cpp/main -m airoboros-3b-3p0.q8_0.gguf -p "[INST]Please give me a color representing Namibia's beauty.[/INST]"
38
+ ```
39
+ llm_load_print_meta: model type = 3B
40
+ llm_load_print_meta: model ftype = mostly Q8_0
41
+ llm_load_print_meta: model params = 2.80 B
42
+ llm_load_print_meta: model size = 2.77 GiB (8.50 BPW)
43
+ llm_load_print_meta: general.name = airoboros-3b-3p0
44
+ llm_load_print_meta: BOS token = 0 '<|endoftext|>'
45
+ llm_load_print_meta: EOS token = 0 '<|endoftext|>'
46
+ llm_load_print_meta: UNK token = 0 '<|endoftext|>'
47
+ llm_load_print_meta: LF token = 128 'Ä'
48
+ llm_load_tensors: ggml ctx size = 0.13 MB
49
+ llm_load_tensors: mem required = 2833.63 MB
50
+ .............................................................................................
51
+ llama_new_context_with_model: n_ctx = 512
52
+ llama_new_context_with_model: freq_base = 10000.0
53
+ llama_new_context_with_model: freq_scale = 1
54
+ llama_new_context_with_model: kv self size = 160.00 MB
55
+ llama_build_graph: non-view tensors processed: 1125/1125
56
+ llama_new_context_with_model: compute buffer total size = 109.81 MB
57
+
58
+ system_info: n_threads = 2 / 2 | AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
59
+ sampling:
60
+ repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
61
+ top_k = 40, tfs_z = 1.000, top_p = 0.950, min_p = 0.050, typical_p = 1.000, temp = 0.800
62
+ mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
63
+ generate: n_ctx = 512, n_batch = 512, n_predict = 128, n_keep = 0
64
+
65
+
66
+ [INST]Please give me a color representing Namibia's beauty.[/INST]
67
+ The colors of the ocean, sand dunes, and deserts in Namibia are truly breathtaking. The vibrant blues of the sea contrast sharply with the warm greys of the desert and the rich browns of the dunes. The mixture of these colors creates a stunning landscape that is unique to this region. [end of text]
68
+
69
+ llama_print_timings: load time = 19034.49 ms
70
+ llama_print_timings: sample time = 61.34 ms / 65 runs ( 0.94 ms per token, 1059.72 tokens per second)
71
+ llama_print_timings: prompt eval time = 7435.06 ms / 19 tokens ( 391.32 ms per token, 2.56 tokens per second)
72
+ llama_print_timings: eval time = 31744.48 ms / 64 runs ( 496.01 ms per token, 2.02 tokens per second)
73
+ llama_print_timings: total time = 39302.86 ms
74
+ ```
75
 
76
  ## Original Model Card:
77
  ### Overview