umisetokikaze commited on
Commit
0bf01f9
β€’
1 Parent(s): ea51642

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +43 -47
README.md CHANGED
@@ -1,50 +1,46 @@
1
  ---
2
- base_model: []
3
- library_name: transformers
 
 
4
  tags:
5
- - mergekit
6
- - merge
7
-
8
  ---
9
- # dump
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [linear](https://arxiv.org/abs/2203.05482) merge method.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * /home/ubuntu/work/Exveria/text-generation-webui/models/Ninja-v1
22
- * /home/ubuntu/work/Umise/TGenwebui/models/VT4o3
23
- * /home/ubuntu/work/Umise/TGenwebui/models/VT3
24
- * /home/ubuntu/work/Umise/TGenwebui/models/VT4
25
- * /home/ubuntu/work/Exveria/text-generation-webui/models/Ninja-32k-NSFW
26
-
27
- ### Configuration
28
-
29
- The following YAML configuration was used to produce this model:
30
-
31
- ```yaml
32
- models:
33
- - model: /home/ubuntu/work/Exveria/text-generation-webui/models/Ninja-32k-NSFW
34
- parameters:
35
- weight: 0.6
36
- - model: /home/ubuntu/work/Umise/TGenwebui/models/VT4o3
37
- parameters:
38
- weight: 1
39
- - model: /home/ubuntu/work/Exveria/text-generation-webui/models/Ninja-v1
40
- parameters:
41
- weight: 0.6
42
- - model: /home/ubuntu/work/Umise/TGenwebui/models/VT3
43
- parameters:
44
- weight: 1
45
- - model: /home/ubuntu/work/Umise/TGenwebui/models/VT4
46
- parameters:
47
- weight: 0.6
48
- merge_method: linear
49
- dtype: bfloat16
50
- ```
 
1
  ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ - ja
6
  tags:
7
+ - finetuned
8
+ library_name: transformers
9
+ pipeline_tag: text-generation
10
  ---
11
+
12
+ # Model Card for VecTeus-v1.0
13
+
14
+ The Mistral-7B--based Large Language Model (LLM) is an noveldataset fine-tuned version of the Mistral-7B-v0.1
15
+
16
+ VecTeus has the following changes compared to Mistral-7B-v0.1.
17
+ - 128k context window (8k context in v0.1)
18
+ - Achieving both high quality Japanese and English generation
19
+ - Can be generated NSFW
20
+ - Memory ability that does not forget even after long-context generation
21
+
22
+ This model was created with the help of GPUs from the first LocalAI hackathon.
23
+
24
+ We would like to take this opportunity to thank
25
+
26
+ ## List of Creation Methods
27
+
28
+ - Chatvector for multiple models
29
+ - Simple linear merging of result models
30
+ - Domain and Sentence Enhancement with LORA
31
+ - Context expansion
32
+
33
+ ## Instruction format
34
+
35
+ Freed from templates. Congratulations
36
+
37
+ ## Example prompts to improve (Japanese)
38
+
39
+ - BAD: あγͺγŸγ―β—‹β—‹γ¨γ—γ¦ζŒ―γ‚‹θˆžγ„γΎγ™
40
+ - GOOD: あγͺγŸγ―β—‹β—‹γ§γ™
41
+
42
+ - BAD: あγͺγŸγ―β—‹β—‹γŒγ§γγΎγ™
43
+ - GOOD: あγͺγŸγ―β—‹β—‹γ‚’γ—γΎγ™
44
+
45
+ # Other points to keep in mind
46
+ If possible, we recommend inferring with llamacpp rather than Transformers.