grimjim commited on
Commit
b37ab2f
1 Parent(s): 456664c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +44 -44
README.md CHANGED
@@ -1,44 +1,44 @@
1
- ---
2
- base_model:
3
- - meta-llama/Meta-Llama-3.1-8B-Instruct
4
- - grimjim/Llama-3-Instruct-abliteration-LoRA-8B
5
- library_name: transformers
6
- tags:
7
- - mergekit
8
- - merge
9
- license: llama3.1
10
- pipeline_tag: text-generation
11
- ---
12
- # Llama-3.1-8B-Instruct-abliterated_via_adapter
13
-
14
- This model is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
-
16
- A LoRA was applied to "abliterate" refusals in [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct). The result appears to work despite the LoRA having been derived from Llama 3 instead of Llama 3.1, which implies that there is significant feature commonality between the 3 and 3.1 models.
17
-
18
- The LoRA was extracted from [failspy/Meta-Llama-3-8B-Instruct-abliterated-v3](https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3) and uses [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) as a base.
19
-
20
- Built with Llama.
21
-
22
- ## Merge Details
23
- ### Merge Method
24
-
25
- This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B) as a base.
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- base_model: meta-llama/Meta-Llama-3.1-8B-Instruct+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
33
- dtype: bfloat16
34
- merge_method: task_arithmetic
35
- parameters:
36
- normalize: false
37
- slices:
38
- - sources:
39
- - layer_range: [0, 32]
40
- model: meta-llama/Meta-Llama-3.1-8B-Instruct+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
41
- parameters:
42
- weight: 1.0
43
-
44
- ```
 
1
+ ---
2
+ base_model:
3
+ - meta-llama/Meta-Llama-3.1-8B-Instruct
4
+ - grimjim/Llama-3-Instruct-abliteration-LoRA-8B
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+ license: llama3.1
10
+ pipeline_tag: text-generation
11
+ ---
12
+ # Llama-3.1-8B-Instruct-abliterated_via_adapter
13
+
14
+ This model is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
+
16
+ A LoRA was applied to "abliterate" refusals in [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct). The result appears to work despite the LoRA having been derived from Llama 3 instead of Llama 3.1, which implies that there is significant feature commonality between the 3 and 3.1 models.
17
+
18
+ The LoRA was extracted from [failspy/Meta-Llama-3-8B-Instruct-abliterated-v3](https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3) using [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) as a base.
19
+
20
+ Built with Llama.
21
+
22
+ ## Merge Details
23
+ ### Merge Method
24
+
25
+ This model was merged using the [task arithmetic](https://arxiv.org/abs/2212.04089) merge method using [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B) as a base.
26
+
27
+ ### Configuration
28
+
29
+ The following YAML configuration was used to produce this model:
30
+
31
+ ```yaml
32
+ base_model: meta-llama/Meta-Llama-3.1-8B-Instruct+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
33
+ dtype: bfloat16
34
+ merge_method: task_arithmetic
35
+ parameters:
36
+ normalize: false
37
+ slices:
38
+ - sources:
39
+ - layer_range: [0, 32]
40
+ model: meta-llama/Meta-Llama-3.1-8B-Instruct+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
41
+ parameters:
42
+ weight: 1.0
43
+
44
+ ```