Text Generation
Transformers
PyTorch
English
llama
Merge
slerp
text-generation-inference
baebee commited on
Commit
8276bfe
1 Parent(s): 6a9c804

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +60 -0
README.md CHANGED
@@ -1,3 +1,63 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ datasets:
4
+ - Open-Orca/OpenOrca
5
+ - conceptofmind/cot_submix_original
6
+ - conceptofmind/t0_submix_original
7
+ - conceptofmind/niv2_submix_original
8
+ - conceptofmind/flan2021_submix_original
9
+ - ehartford/dolphin
10
+ language:
11
+ - en
12
+ tags:
13
+ - merge
14
+ - slerp
15
+ inference: false
16
+ metrics:
17
+ - accuracy
18
+ - bleu
19
  ---
20
+ <h1 style="text-align: center">Orfini</h1>
21
+ <h2 style="text-align: center">An experimental model</h2>
22
+ <hr>
23
+
24
+
25
+ ## Model Details
26
+ Orfini is an experimental merged model created from the following three foundation models:
27
+
28
+ - stabilityai/StableBeluga-7B
29
+ - pankajmathur/orca_mini_v3_7b
30
+ - AIDC-ai-business/Marcoroni-7B
31
+
32
+ Orfini was created by merging the weights and architectures of these three models using a custom merging technique. No further fine-tuning was performed after the merge.
33
+
34
+ Once the model obtains it's evaluation scores, then we'll know if it works or not.
35
+
36
+ ## Intended Use
37
+ As an experimental model, Orfini is intended for testing and research purposes only. It should not be used for production systems or to generate content for public use.
38
+
39
+ ## Training Data
40
+ Orfini inherits training data from its three foundation models:
41
+
42
+ - StableBeluga-7B: COT, Niv2, t0, & FLAN2021
43
+ - dolphin-llama2-7b: Dolphin
44
+ - Marcoroni-7B: OpenOrca
45
+
46
+ ## Limitations
47
+ As an untested merged model, Orfini has unknown capabilities and limitations. Potential issues include:
48
+
49
+ - Instability due to merged architectures
50
+ - Compounded bias and issues from all three foundation models
51
+ - Decreased performance on some tasks compared to the foundation models
52
+
53
+ Extensive testing is required to characterize Orfini's capabilities and limitations.
54
+
55
+ ## Ethical Considerations
56
+ - Orfini may exhibit harmful biases inherited from its training data
57
+ - Output may be unreliable or manipulated due to instability
58
+ - Experimental nature increases potential for misuse
59
+
60
+ Use this model ethically and do not deploy it for sensitive applications.
61
+
62
+ ## Contact Information
63
+ Please report issues or concerns with this model to the creator for further investigation.