Lumitess This model was a attempt to create a good llama 3 uncencored model with long context. At it worked like a charm Did a merge with breadcrumbs_ties method. Instruct gradient, Lumimaid and Tess. Uses llama 3 context Sampler wise it has a very wide optimal so works with lots of different settings. base_model: [] library_name: transformers tags: - mergekit - merge --- # model This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the breadcrumbs_ties merge method using I:\Llama-3-70B-Instruct-Gradient-262k as a base. ### Models Merged The following models were included in the merge: * E:\Llama-3-Lumimaid-70B-v0.1-OAS * I:\Tess-2.0-Llama-3-70B-v0.2 ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: I:\Llama-3-70B-Instruct-Gradient-262k parameters: weight: 0.20 density: 0.90 gamma: 0.01 - model: I:\Tess-2.0-Llama-3-70B-v0.2 parameters: weight: 0.20 density: 0.90 gamma: 0.01 - model: E:\Llama-3-Lumimaid-70B-v0.1-OAS parameters: weight: 0.60 density: 0.90 gamma: 0.01 merge_method: breadcrumbs_ties base_model: I:\Llama-3-70B-Instruct-Gradient-262k dtype: bfloat16 ```