Upload README.md with huggingface_hub

e08380a verified 2 months ago

No virus

4.39 kB

	---
	language:
	- en
	license: apache-2.0
	library_name: transformers
	tags:
	- llama-cpp
	- gguf-my-repo
	base_model: mistralai/Mistral-7B-Instruct-v0.2
	datasets:
	- NeuralNovel/Neural-Story-v1
	inference: false
	model-index:
	- name: Mistral-7B-Instruct-v0.2-Neural-Story
	results:
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: AI2 Reasoning Challenge (25-Shot)
	type: ai2_arc
	config: ARC-Challenge
	split: test
	args:
	num_few_shot: 25
	metrics:
	- type: acc_norm
	value: 64.08
	name: normalized accuracy
	source:
	url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: HellaSwag (10-Shot)
	type: hellaswag
	split: validation
	args:
	num_few_shot: 10
	metrics:
	- type: acc_norm
	value: 83.97
	name: normalized accuracy
	source:
	url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: MMLU (5-Shot)
	type: cais/mmlu
	config: all
	split: test
	args:
	num_few_shot: 5
	metrics:
	- type: acc
	value: 60.67
	name: accuracy
	source:
	url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: TruthfulQA (0-shot)
	type: truthful_qa
	config: multiple_choice
	split: validation
	args:
	num_few_shot: 0
	metrics:
	- type: mc2
	value: 66.89
	source:
	url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: Winogrande (5-shot)
	type: winogrande
	config: winogrande_xl
	split: validation
	args:
	num_few_shot: 5
	metrics:
	- type: acc
	value: 75.85
	name: accuracy
	source:
	url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
	name: Open LLM Leaderboard
	- task:
	type: text-generation
	name: Text Generation
	dataset:
	name: GSM8k (5-shot)
	type: gsm8k
	config: main
	split: test
	args:
	num_few_shot: 5
	metrics:
	- type: acc
	value: 38.29
	name: accuracy
	source:
	url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story
	name: Open LLM Leaderboard
	---

	# DavidAU/Mistral-7B-Instruct-v0.2-Neural-Story-Q6_K-GGUF
	This model was converted to GGUF format from [`NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story`](https://huggingface.co/NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
	Refer to the [original model card](https://huggingface.co/NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story) for more details on the model.
	## Use with llama.cpp

	Install llama.cpp through brew.

	```bash
	brew install ggerganov/ggerganov/llama.cpp
	```
	Invoke the llama.cpp server or the CLI.

	CLI:

	```bash
	llama-cli --hf-repo DavidAU/Mistral-7B-Instruct-v0.2-Neural-Story-Q6_K-GGUF --model mistral-7b-instruct-v0.2-neural-story.Q6_K.gguf -p "The meaning to life and the universe is"
	```

	Server:

	```bash
	llama-server --hf-repo DavidAU/Mistral-7B-Instruct-v0.2-Neural-Story-Q6_K-GGUF --model mistral-7b-instruct-v0.2-neural-story.Q6_K.gguf -c 2048
	```

	Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.

	```
	git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m mistral-7b-instruct-v0.2-neural-story.Q6_K.gguf -n 128
	```