File size: 1,415 Bytes
130b895 5a2348c 50e0fab 130b895 efbfa4f fca6229 130b895 a608653 50e0fab |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
---
base_model:
- LeroyDyer/Mixtral_AI_Cyber_Orca
- LeroyDyer/Mixtral_AI_Cyber_4.0
- LeroyDyer/Mixtral_AI_Cyber_4.0_m1
- LeroyDyer/Mixtral_AI_Cyber_Dolphin
- LeroyDyer/Mixtral_AI_Cyber_4_m1_SFT
- LeroyDyer/Mixtral_AI_Cyber_3.m2
library_name: transformers
tags:
- mergekit
- merge
license: apache-2.0
language:
- en
datasets:
- cognitivecomputations/dolphin
- Open-Orca/OpenOrca
---
GOOD ONE!
This summary describes the latest language model (LLM), which is a merge of pre-trained language models using MergeKit.
Merging these models is crucial for consolidating the internal predictive nature of the network. Each model undergoes different fine-tuning and adjustment to its weights, maintaining consistent size across models is essential. Despite using the Mistral transformer network as the base,
it's worth noting that the merged models (Commercial Orca, Dolphin, Nous, Starling, etc.) may exhibit contamination,
leading to some questions being already present in the dataset and potential biases towards the creator's personal psychometric understanding of the world.
Fine-tuning aims to adapt the LLM to new types of questions or tasks, but misalignment during this process can result in erroneous text outputs.
Future tuning will be tailored to specific tasks, leveraging the merged common models as a base. Observations on stability and performance of other models are welcomed for further refinement. |