miqu-1-120b-GGUF / README.md

Update README.md

6d89d59 verified 4 months ago

No virus

3.93 kB

	---
	base_model:
	- 152334H/miqu-1-70b-sf
	language:
	- en
	- de
	- fr
	- es
	- it
	library_name: transformers
	tags:
	- mergekit
	- merge
	---
	# miqu-1-120b

	![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6303ca537373aacccd85d8a7/LxO9j7OykuabKLYQHIodG.jpeg)

	* EXL2: [2.4bpw](https://huggingface.co/LoneStriker/wolfram_miqu-1-120b-2.4bpw-h6-exl2) \| [2.65bpw](https://huggingface.co/LoneStriker/wolfram_miqu-1-120b-2.65bpw-h6-exl2) \| [3.0bpw](https://huggingface.co/LoneStriker/wolfram_miqu-1-120b-3.0bpw-h6-exl2) \| [4.0bpw](https://huggingface.co/LoneStriker/wolfram_miqu-1-120b-4.0bpw-h6-exl2) \| [5.0bpw](https://huggingface.co/LoneStriker/wolfram_miqu-1-120b-5.0bpw-h6-exl2)
	* More GGUF: [Q2_K-Q5_K_M](https://huggingface.co/LoneStriker/wolfram_miqu-1-120b-GGUF/)
	* HF FP16: [wolfram/miqu-1-120b](https://huggingface.co/wolfram/miqu-1-120b)

	This is a 120b frankenmerge of [miqu-1-70b](https://huggingface.co/miqudev/miqu-1-70b) created by interleaving layers of [miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf) with itself using [mergekit](https://github.com/cg123/mergekit).

	Inspired by [Venus-120b-v1.2](https://huggingface.co/nsfwthrowitaway69/Venus-120b-v1.2), [MegaDolphin-120b](https://huggingface.co/cognitivecomputations/MegaDolphin-120b), and [goliath-120b](https://huggingface.co/alpindale/goliath-120b).

	Thanks for the support, [CopilotKit](https://github.com/CopilotKit/CopilotKit) - the open-source platform for building in-app AI Copilots into any product, with any LLM model. Check out their GitHub.

	Thanks for the EXL2 and GGUF quants, [Lone Striker](https://huggingface.co/LoneStriker)!

	## Prompt template: Mistral

	```
	<s>[INST] {prompt} [/INST]
	```

	See also: [🐺🐦‍⬛ LLM Prompt Format Comparison/Test: Mixtral 8x7B Instruct with 17 different instruct templates : LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/comments/18ljvxb/llm_prompt_format_comparisontest_mixtral_8x7b/)

	## Model Details

	* Max Context: 32764 tokens (kept the weird number from the original/base model)
	* Layers: 140

	## Merge Details

	### Merge Method

	This model was merged using the passthrough merge method.

	### Models Merged

	The following models were included in the merge:
	* [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf)

	### Configuration

	The following YAML configuration was used to produce this model:

	```yaml
	dtype: float16
	merge_method: passthrough
	slices:
	- sources:
	- layer_range: [0, 20]
	model: 152334H/miqu-1-70b-sf
	- sources:
	- layer_range: [10, 30]
	model: 152334H/miqu-1-70b-sf
	- sources:
	- layer_range: [20, 40]
	model: 152334H/miqu-1-70b-sf
	- sources:
	- layer_range: [30, 50]
	model: 152334H/miqu-1-70b-sf
	- sources:
	- layer_range: [40, 60]
	model: 152334H/miqu-1-70b-sf
	- sources:
	- layer_range: [50, 70]
	model: 152334H/miqu-1-70b-sf
	- sources:
	- layer_range: [60, 80]
	model: 152334H/miqu-1-70b-sf
	```

	## Credits & Special Thanks

	* original (unreleased) model: [mistralai (Mistral AI_)](https://huggingface.co/mistralai)
	* leaked model: [miqudev/miqu-1-70b](https://huggingface.co/miqudev/miqu-1-70b)
	* f16 model: [152334H/miqu-1-70b-sf](https://huggingface.co/152334H/miqu-1-70b-sf)
	* mergekit: [arcee-ai/mergekit: Tools for merging pretrained large language models.](https://github.com/arcee-ai/mergekit)
	* mergekit_config.yml: [nsfwthrowitaway69/Venus-120b-v1.2](https://huggingface.co/nsfwthrowitaway69/Venus-120b-v1.2)

	### Support

	* [My Ko-fi page](https://ko-fi.com/wolframravenwolf) if you'd like to tip me to say thanks or request specific models to be tested or merged with priority. Also consider supporting your favorite model creators, quantizers, or frontend/backend devs if you can afford to do so. They deserve it!

	#### DISCLAIMER: THIS IS [BASED ON A LEAKED ASSET](https://huggingface.co/miqudev/miqu-1-70b/discussions/10) AND HAS NO LICENSE ASSOCIATED WITH IT. USE AT YOUR OWN RISK.