Update README.md

37536d8 verified 5 months ago

No virus

4.57 kB

	---
	language:
	- en
	- es
	- ru
	- zh
	- de
	- fr
	- th
	- ca
	- it
	- ja
	- pl
	- eo
	- eu
	- vi
	- fi
	- hu
	- ar
	- nl
	- da
	- tr
	- ko
	- he
	- id
	- cs
	- bn
	- sv
	base_model:
	- NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
	- NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1
	- NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1
	tags:
	- mergekit
	- merge
	widget:
	- text: "<\|im_start\|>system\nYou are a helpful AI assistant.<\|im_end\|>\n<\|im_start\|>user\npodrias escribir un codigo de ejemplo en Python<\|im_end\|>\n<\|im_start\|>assistant\n"
	license: apache-2.0
	---
	# merged

	This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

	## Merge Details
	### Merge Method

	This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1](https://huggingface.co/NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1) as a base.

	### Models Merged

	The following models were included in the merge:
	* [NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1](https://huggingface.co/NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1)
	* [NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1](https://huggingface.co/NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1)

	### Configuration

	The following YAML configuration was used to produce this model:

	```yaml
	base_model:
	model:
	path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
	dtype: bfloat16
	merge_method: dare_ties
	slices:
	- sources:
	- layer_range: [0, 22]
	model:
	model:
	path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
	- layer_range: [0, 22]
	model:
	model:
	path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
	parameters:
	density: 0.55
	weight: 0.55
	- layer_range: [0, 22]
	model:
	model:
	path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1
	parameters:
	density: 0.55
	weight: 0.56
	- layer_range: [0, 22]
	model:
	model:
	path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1
	parameters:
	density: 0.55
	weight: 0.56
	```



	```Python
	from transformers import (
	AutoModelForCausalLM,
	AutoTokenizer,
	BitsAndBytesConfig,
	HfArgumentParser,
	TrainingArguments,
	pipeline,
	logging,
	GenerationConfig,
	TextIteratorStreamer,
	)
	import torch

	new_model= "NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_all_Cluster_merge_v1"
	model = AutoModelForCausalLM.from_pretrained(#f'NickyNicky/{new_model}',
	new_model,
	device_map="auto",
	trust_remote_code=True,
	torch_dtype=torch.bfloat16,

	low_cpu_mem_usage= True,
	# use_flash_attention_2=False,

	)


	tokenizer = AutoTokenizer.from_pretrained(new_model,
	max_length=2048,
	trust_remote_code=True,
	use_fast = True,
	)

	tokenizer.pad_token = tokenizer.eos_token
	# tokenizer.padding_side = 'left'
	tokenizer.padding_side = 'right'


	prompt= """<\|im_start\|>system
	You are a helpful AI assistant.<\|im_end\|>
	<\|im_start\|>user
	escribe una historia de amor.<\|im_end\|>
	<\|im_start\|>assistant
	"""

	inputs = tokenizer.encode(prompt,
	return_tensors="pt",
	add_special_tokens=False).cuda()#.to("cuda") # False # True


	generation_config = GenerationConfig(
	max_new_tokens=700,
	temperature=0.5,
	top_p=0.9,
	top_k=40,
	repetition_penalty=1.1, #1.1, # 1.0 means no penalty, > 1.0 means penalty, 1.2 from CTRL paper
	do_sample=True,
	pad_token_id=tokenizer.eos_token_id,
	eos_token_id=tokenizer.eos_token_id,
	)
	outputs = model.generate(
	generation_config=generation_config,
	input_ids=inputs,)
	# tokenizer.decode(outputs[0], skip_special_tokens=False) #True
	print(tokenizer.decode(outputs[0], skip_special_tokens=False))
	```