sabafallah commited on
Commit
06f3c76
1 Parent(s): f331ad3

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +80 -0
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: sentence-transformers/all-MiniLM-L12-v2
3
+ datasets:
4
+ - s2orc
5
+ - flax-sentence-embeddings/stackexchange_xml
6
+ - ms_marco
7
+ - gooaq
8
+ - yahoo_answers_topics
9
+ - code_search_net
10
+ - search_qa
11
+ - eli5
12
+ - snli
13
+ - multi_nli
14
+ - wikihow
15
+ - natural_questions
16
+ - trivia_qa
17
+ - embedding-data/sentence-compression
18
+ - embedding-data/flickr30k-captions
19
+ - embedding-data/altlex
20
+ - embedding-data/simple-wiki
21
+ - embedding-data/QQP
22
+ - embedding-data/SPECTER
23
+ - embedding-data/PAQ_pairs
24
+ - embedding-data/WikiAnswers
25
+ language: en
26
+ library_name: sentence-transformers
27
+ license: apache-2.0
28
+ pipeline_tag: sentence-similarity
29
+ tags:
30
+ - sentence-transformers
31
+ - feature-extraction
32
+ - sentence-similarity
33
+ - transformers
34
+ - llama-cpp
35
+ - gguf-my-repo
36
+ ---
37
+
38
+ # sabafallah/all-MiniLM-L12-v2-Q8_0-GGUF
39
+ This model was converted to GGUF format from [`sentence-transformers/all-MiniLM-L12-v2`](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
40
+ Refer to the [original model card](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) for more details on the model.
41
+
42
+ ## Use with llama.cpp
43
+ Install llama.cpp through brew (works on Mac and Linux)
44
+
45
+ ```bash
46
+ brew install llama.cpp
47
+
48
+ ```
49
+ Invoke the llama.cpp server or the CLI.
50
+
51
+ ### CLI:
52
+ ```bash
53
+ llama-cli --hf-repo sabafallah/all-MiniLM-L12-v2-Q8_0-GGUF --hf-file all-minilm-l12-v2-q8_0.gguf -p "The meaning to life and the universe is"
54
+ ```
55
+
56
+ ### Server:
57
+ ```bash
58
+ llama-server --hf-repo sabafallah/all-MiniLM-L12-v2-Q8_0-GGUF --hf-file all-minilm-l12-v2-q8_0.gguf -c 2048
59
+ ```
60
+
61
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
62
+
63
+ Step 1: Clone llama.cpp from GitHub.
64
+ ```
65
+ git clone https://github.com/ggerganov/llama.cpp
66
+ ```
67
+
68
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
69
+ ```
70
+ cd llama.cpp && LLAMA_CURL=1 make
71
+ ```
72
+
73
+ Step 3: Run inference through the main binary.
74
+ ```
75
+ ./llama-cli --hf-repo sabafallah/all-MiniLM-L12-v2-Q8_0-GGUF --hf-file all-minilm-l12-v2-q8_0.gguf -p "The meaning to life and the universe is"
76
+ ```
77
+ or
78
+ ```
79
+ ./llama-server --hf-repo sabafallah/all-MiniLM-L12-v2-Q8_0-GGUF --hf-file all-minilm-l12-v2-q8_0.gguf -c 2048
80
+ ```