Upload 3 files
Browse filesadded q6_k auantized for gguf server and 1 shot command
- .gitattributes +2 -0
- README.md +14 -0
- mmproj-obsidian-f16.gguf +3 -0
- obsidian-q6.gguf +3 -0
.gitattributes
CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
mmproj-obsidian-f16.gguf filter=lfs diff=lfs merge=lfs -text
|
37 |
+
obsidian-q6.gguf filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
@@ -1,3 +1,17 @@
|
|
1 |
---
|
2 |
license: mit
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: mit
|
3 |
---
|
4 |
+
|
5 |
+
# NousResearch - Obsidian-3B-V0.5 gguf q6 version by Nisten
|
6 |
+
|
7 |
+
## How to Run the server and download the models all in one command, you need ~2.9gb of space
|
8 |
+
|
9 |
+
To run this on mac or linux, make a new folder, go in it and run this whole in one shot:
|
10 |
+
|
11 |
+
```bash
|
12 |
+
git clone -b stablelm-support https://github.com/Galunid/llama.cpp.git && \
|
13 |
+
cd llama.cpp && \
|
14 |
+
make && \
|
15 |
+
wget https://huggingface.co/nisten/obsidian-3b-multimodal-q6-gguf/blob/main/obsidian-q6.gguf && \
|
16 |
+
wget https://huggingface.co/nisten/obsidian-3b-multimodal-q6-gguf/blob/main/mmproj-obsidian-f16.gguf && \
|
17 |
+
./server -m obsidian-q6.gguf --mmproj mmproj-obsidian-f16.gguf -ngl 42
|
mmproj-obsidian-f16.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:2b62df2eef4461a92c9edb9f312584f872611ffcc0ce213eb6d59792629b6cab
|
3 |
+
size 600829088
|
obsidian-q6.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ca1ae854c5ef58494b9435fb5466897b5bf511dfcdfdd9bcbb6ae0e050d4bee0
|
3 |
+
size 2295984640
|