MrRobotoAI/MrRoboto-ProLongBASE-pt2-unaligned-8b

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using MrRobotoAI/MrRoboto-ProLong-8b-v4i as a base.

Models Merged

The following models were included in the merge:

• princeton-nlp/Llama-3-8B-ProLong-512k-Base + ResplendentAI/NoWarning_Llama3
• aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored + ResplendentAI/NoWarning_Llama3
• WeMake/Llama-3-8B-Instruct-V41-1048k + ResplendentAI/NoWarning_Llama3
• winglian/Meta-Llama-3-8B-1M-v2 + ResplendentAI/NoWarning_Llama3
• THUDM/LongWriter-llama3.1-8b + ResplendentAI/NoWarning_Llama3
• MrRobotoAI/MrRoboto-ProLong-8b-v4i + ResplendentAI/NoWarning_Llama3
• MrRobotoAI/MrRoboto-ProLongBASE-pt1-unaligned-8b + ResplendentAI/NoWarning_Llama3
• gradientai/Llama-3-8B-Instruct-Gradient-1048k + ResplendentAI/NoWarning_Llama3
• princeton-nlp/Llama-3-8B-ProLong-512k-Instruct + ResplendentAI/NoWarning_Llama3

Configuration

The following YAML configuration was used to produce this model:

models:

  - model: MrRobotoAI/MrRoboto-ProLongBASE-pt1-unaligned-8b+ResplendentAI/NoWarning_Llama3
  - model: winglian/Meta-Llama-3-8B-1M-v2+ResplendentAI/NoWarning_Llama3
  - model: THUDM/LongWriter-llama3.1-8b+ResplendentAI/NoWarning_Llama3
  - model: princeton-nlp/Llama-3-8B-ProLong-512k-Instruct+ResplendentAI/NoWarning_Llama3
  - model: princeton-nlp/Llama-3-8B-ProLong-512k-Base+ResplendentAI/NoWarning_Llama3
  - model: aifeifei798/DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored+ResplendentAI/NoWarning_Llama3
  - model: MrRobotoAI/MrRoboto-ProLong-8b-v4i+ResplendentAI/NoWarning_Llama3
  - model: gradientai/Llama-3-8B-Instruct-Gradient-1048k+ResplendentAI/NoWarning_Llama3
  - model: WeMake/Llama-3-8B-Instruct-V41-1048k+ResplendentAI/NoWarning_Llama3

merge_method: model_stock
base_model: MrRobotoAI/MrRoboto-ProLong-8b-v4i
normalize: true
dtype: float16