Commit
•
a9d71cf
1
Parent(s):
68a35af
Upload folder using huggingface_hub
Browse files- README.md +75 -0
- config.json +27 -0
- generation_config.json +6 -0
- huggingface-metadata.txt +55 -0
- model.safetensors.index.json +0 -0
- output-00001-of-00009.safetensors +3 -0
- output-00002-of-00009.safetensors +3 -0
- output-00003-of-00009.safetensors +3 -0
- output-00004-of-00009.safetensors +3 -0
- output-00005-of-00009.safetensors +3 -0
- output-00006-of-00009.safetensors +3 -0
- output-00007-of-00009.safetensors +3 -0
- output-00008-of-00009.safetensors +3 -0
- output-00009-of-00009.safetensors +3 -0
- special_tokens_map.json +28 -0
- tokenizer.json +0 -0
- tokenizer.model +3 -0
- tokenizer_config.json +43 -0
README.md
ADDED
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: llama2
|
3 |
+
---
|
4 |
+
|
5 |
+
# Tess
|
6 |
+
|
7 |
+
![Tess](https://huggingface.co/migtissera/Tess-XL-v1.0/resolve/main/Tess.png)
|
8 |
+
|
9 |
+
Tess, short for Tessoro/Tessoso, is a general purpose Large Language Model series. Tess-XL is trained on the Goliath-120B as the base.
|
10 |
+
|
11 |
+
While there are task specific Tess models (Creative or STEM), this model is general purpose.
|
12 |
+
|
13 |
+
# Prompt format:
|
14 |
+
|
15 |
+
```
|
16 |
+
SYSTEM: Elaborate on the topic using a Tree of Thoughts and backtrack when necessary to construct a clear, cohesive Chain of Thought reasoning. Always answer without hesitation.
|
17 |
+
USER: What is the relationship between Earth's atmosphere, magnetic field and gravity?
|
18 |
+
ASSISTANT:
|
19 |
+
```
|
20 |
+
|
21 |
+
|
22 |
+
# Quantized Versions
|
23 |
+
|
24 |
+
Quantized versions are available in the following repos:
|
25 |
+
|
26 |
+
- GPTQ: https://huggingface.co/TheBloke/Tess-XL-v1.0-GPTQ
|
27 |
+
- AWQ: https://huggingface.co/TheBloke/Tess-XL-v1.0-AWQ
|
28 |
+
- GGUF: https://huggingface.co/TheBloke/Tess-XL-v1.0-GGUF
|
29 |
+
|
30 |
+
|
31 |
+
# Sample Conversation
|
32 |
+
|
33 |
+
```
|
34 |
+
You: What is the relationship between an electrical field and a magnetic field?
|
35 |
+
The relationship between an electric field and a magnetic field can be summarized in two important ways:
|
36 |
+
|
37 |
+
1. Cause and effect: A changing electric field can induce a magnetic field, and a changing magnetic field can in turn induce an electric field. This is the principle of electromagnetic induction.
|
38 |
+
|
39 |
+
2. Duality: Electric and magnetic fields are often described as components of a single electromagnetic field. This duality is most clearly seen in Maxwell's equations, which describe how electric and magnetic fields are related.
|
40 |
+
|
41 |
+
Chain of Thought:
|
42 |
+
- Start with the question: What is the relationship between an electric field and a magnetic field?
|
43 |
+
- Think about the basic principles of electromagnetism: cause and effect, as well as the duality of electric and magnetic fields.
|
44 |
+
- Explain the relationship by referring to electromagnetic induction and Maxwell's equations.
|
45 |
+
|
46 |
+
You: In some instances that are associated with UAP, an increase in electrical field causes a decrease in the measured magnetic field. This is contradictory to known physics. Can we theorize what could cause this?
|
47 |
+
The relationship you describe between an increase in the electric field and a decrease in the measured magnetic field is indeed unusual and does not align with the conventional understanding of electromagnetic fields. However, we can consider some possible explanations:
|
48 |
+
|
49 |
+
1. Unknown phenomena: It's possible that there are physical phenomena or interactions that we have not yet discovered or fully understood that could cause this effect. These could include new particles or forces that we have not yet detected, or unusual properties of known particles or fields.
|
50 |
+
|
51 |
+
2. Measurement errors: Another possibility is that the observed relationship is due to measurement errors or inaccuracies in the instruments used to measure the electric and magnetic fields. This could include calibration issues, interference from other sources, or limitations in the sensitivity of the equipment.
|
52 |
+
|
53 |
+
3. Environmental effects: Changes in the environment or the nature of the object emitting the fields may also affect the relationship between the electric and magnetic fields. For example, if the object is moving or undergoing changes in composition or structure, this could affect the way that the fields are generated and detected.
|
54 |
+
|
55 |
+
In summary, while the observed relationship between an increase in the electric field and a decrease in the measured magnetic field is not consistent with our current understanding of electromagnetism, there could be several possible explanations, including unknown phenomena, measurement errors, or environmental effects. Further research and investigation would be needed to determine the cause of this phenomenon.
|
56 |
+
|
57 |
+
You: What is the most advanced theory of electro-magnetic fields right now?
|
58 |
+
The most advanced theory of electromagnetic fields is quantum electrodynamics (QED). QED is a part of the larger framework of quantum field theory (QFT), which aims to describe the interactions of particles and fields in the context of quantum mechanics.
|
59 |
+
|
60 |
+
QED specifically describes the interactions between photons (which are the quanta of the electromagnetic field) and charged particles, such as electrons and positrons. It provides a theoretical framework for understanding how photons can be created and absorbed by charged particles, and how they can interact with each other.
|
61 |
+
|
62 |
+
The predictions of QED have been experimentally verified to an extremely high degree of accuracy, making it one of the most successful theories in physics. Despite its success, QED is not considered a complete theory, as it does not include gravity and is inconsistent with the principles of general relativity. The search for a unified theory that can reconcile QED with gravity is an area of active research in theoretical physics.
|
63 |
+
|
64 |
+
You: Give me a detailed introduction to QED.
|
65 |
+
Quantum electrodynamics (QED) is a quantum field theory that describes the interactions between electrically charged particles (such as electrons and positrons) and the photons that make up the electromagnetic field. It is a part of the larger framework of quantum field theory (QFT), which aims to describe the interactions of particles and fields in the context of quantum mechanics.
|
66 |
+
|
67 |
+
In QED, electric and magnetic fields are described as being made up of individual packets of energy called photons. Photons are the quanta of the electromagnetic field, and they can be created and absorbed by charged particles.
|
68 |
+
|
69 |
+
QED provides a theoretical framework for understanding how photons can interact with each other and with charged particles. The theory predicts that photons can be created or annihilated in pairs, and that charged particles can emit and absorb photons. These interactions are mediated by the exchange of virtual particles, which are other photons that exist only fleetingly and cannot be directly observed.
|
70 |
+
|
71 |
+
The predictions of QED have been experimentally verified to an extremely high degree of accuracy, making it one of the most successful theories in physics. Despite its success, QED is not considered a complete theory, as it does not include gravity and is inconsistent with the principles of general relativity. The search for a unified theory that can reconcile QED with gravity is an area of active research in theoretical physics.
|
72 |
+
|
73 |
+
In summary, QED is a highly successful theory that describes the interactions between charged particles and the electromagnetic field in terms of photons. While it is not a complete theory, it provides a fundamental understanding of the electromagnetic force and has led to many important discoveries in particle physics.
|
74 |
+
```
|
75 |
+
|
config.json
ADDED
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"_name_or_path": "/home/migel/goliath-120b",
|
3 |
+
"architectures": [
|
4 |
+
"LlamaForCausalLM"
|
5 |
+
],
|
6 |
+
"attention_bias": false,
|
7 |
+
"bos_token_id": 1,
|
8 |
+
"eos_token_id": 2,
|
9 |
+
"hidden_act": "silu",
|
10 |
+
"hidden_size": 8192,
|
11 |
+
"initializer_range": 0.02,
|
12 |
+
"intermediate_size": 28672,
|
13 |
+
"max_position_embeddings": 4096,
|
14 |
+
"model_type": "llama",
|
15 |
+
"num_attention_heads": 64,
|
16 |
+
"num_hidden_layers": 137,
|
17 |
+
"num_key_value_heads": 8,
|
18 |
+
"pretraining_tp": 1,
|
19 |
+
"rms_norm_eps": 1e-05,
|
20 |
+
"rope_scaling": null,
|
21 |
+
"rope_theta": 10000.0,
|
22 |
+
"tie_word_embeddings": false,
|
23 |
+
"torch_dtype": "float16",
|
24 |
+
"transformers_version": "4.35.0",
|
25 |
+
"use_cache": true,
|
26 |
+
"vocab_size": 32000
|
27 |
+
}
|
generation_config.json
ADDED
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"_from_model_config": true,
|
3 |
+
"bos_token_id": 1,
|
4 |
+
"eos_token_id": 2,
|
5 |
+
"transformers_version": "4.35.0"
|
6 |
+
}
|
huggingface-metadata.txt
ADDED
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
url: https://huggingface.co/migtissera/Tess-XL-v1.0
|
2 |
+
branch: main
|
3 |
+
download date: 2023-11-19 13:45:46
|
4 |
+
sha256sum:
|
5 |
+
e84db398d888de5752d0dfaf2a5ce59b5753f44777ca1a98ac096440b58dc27d model-00001-of-00050.safetensors
|
6 |
+
a909ba86aa4e4d53bcdae5018749c3eb9958913de3c4c85f6a341cb4ad836fc6 model-00002-of-00050.safetensors
|
7 |
+
4b732dd48aa8d1ed47393c96ac0396a72c56e64e951db063127a38d3ee58951b model-00003-of-00050.safetensors
|
8 |
+
cd328ab294a324487101cc54c1b4e2249b4f675c4f4886438e293acc8c305b29 model-00004-of-00050.safetensors
|
9 |
+
6efc4746a482c689c230b6d784e64a395bb0cded1ae04d991a55a21dadab74d3 model-00005-of-00050.safetensors
|
10 |
+
2184a9aff076bcbb182c700fa5d5e6b21d3f2cb8a4f75c9e22298e9b1c00bf03 model-00006-of-00050.safetensors
|
11 |
+
1297cc5406f18d73a6f9b294117b49ad900a36b144bf7f9a7f2f0fb6f587b085 model-00007-of-00050.safetensors
|
12 |
+
4de660eb03c67991ad6b3f2302c06763e59ba51072cff2ee83155129d8b1d03d model-00008-of-00050.safetensors
|
13 |
+
437856374bdec3cf15d0d4a5a892052e62221b6ed0dc2734d601a0cd524e90f7 model-00009-of-00050.safetensors
|
14 |
+
c21bf0ede169702e4876b1ec713dcc62a2df318d5d543e00b6af82df4718986a model-00010-of-00050.safetensors
|
15 |
+
ec337241862d0799c78c45b5e3306d87bc766fba2ec3245702bf51deddcdd758 model-00011-of-00050.safetensors
|
16 |
+
5ecbaa6474654d8b025fd636e7cfba08bb813c4b6393635fd458da213a96df93 model-00012-of-00050.safetensors
|
17 |
+
5955199dcc85fbf369484d74440a88bafcab6f66d676b72f3b4d2ab3f3ec4ad9 model-00013-of-00050.safetensors
|
18 |
+
ddc29f8b4691cb62d9fa3c19f637436b473e953205bfaf66c6d993dc57498307 model-00014-of-00050.safetensors
|
19 |
+
aa522f620803cbde2b2459bbb57aee13de7a3f22d745fe33558fa788a42a7654 model-00015-of-00050.safetensors
|
20 |
+
25ee73c336659f05af4bc4b7155ee3f938f41bbffafb31f5209c89a9679bc532 model-00016-of-00050.safetensors
|
21 |
+
b2a74b674221e6ca85c2111380e58081094b41dbe3a50ecc998474b8da830011 model-00017-of-00050.safetensors
|
22 |
+
7fffa126ebded001a11e609da3af16c9e8cf62b7f6c9d70b8206d93a0eacdf9a model-00018-of-00050.safetensors
|
23 |
+
3b702744b785dd7e2fac560c2e3597178ea2efb211d2ba507356666d729b88c5 model-00019-of-00050.safetensors
|
24 |
+
5f0a0bc37c4a39c8fedc1853c0abecf8eda3b25cfd7176ce90151499bffc46d1 model-00020-of-00050.safetensors
|
25 |
+
eac0586048acc71ff64062744368a155de41c22490ab6e2e30ba140457bc1f86 model-00021-of-00050.safetensors
|
26 |
+
fcc222e8491c117a4d305df2e8e04866375fa5eda7a1dabd9918e65c448f7399 model-00022-of-00050.safetensors
|
27 |
+
cd30c6be94678773b15fe244206e413dd46c25ae03a9fc3b13bceee09b6bc135 model-00023-of-00050.safetensors
|
28 |
+
81bd75af6a3b109d3fbb65ac3d1600798a8b77f1d265e17795e8323c63e9c7ad model-00024-of-00050.safetensors
|
29 |
+
607245f306eac346fa09283a79bb35939c5813a32cd6738bc26f5441234a444d model-00025-of-00050.safetensors
|
30 |
+
f32de21e6eac901e751da0f0ccfc512843d55463144a3ca5d68f4ac232fe2cea model-00026-of-00050.safetensors
|
31 |
+
7790da2f6b3215163357f151f968df2c6dcd2b79ded3bc09728f18ca5f54b85d model-00027-of-00050.safetensors
|
32 |
+
05814a7d8d9f56e2792dc578391c330780148c9c26db8088c329360a15f4a6bc model-00028-of-00050.safetensors
|
33 |
+
c30dc42b96f3c077de7eb46ede3396273e17fced5a3605d46326498337f92e43 model-00029-of-00050.safetensors
|
34 |
+
56f8d7704ca766aeb22033457ead8fd6607d2df6f727b5f42d02a3b1c12490a7 model-00030-of-00050.safetensors
|
35 |
+
afd61addd461bb2e8ce2486bc333207b3bb3005d3a20f1b2c867d49067d4d794 model-00031-of-00050.safetensors
|
36 |
+
6bc1eb8dd39277359a0631e4266f0745006d662993543f104f9d699b7ae63d04 model-00032-of-00050.safetensors
|
37 |
+
fbf1247955b7059bbfb63895629d17cbd6ccc1259eba347ae0e48b4d8873cda6 model-00033-of-00050.safetensors
|
38 |
+
10107bf13fdd2c102abbafbc6fb51f7a0523c08ef3031632be0bba2d81398f41 model-00034-of-00050.safetensors
|
39 |
+
62c614d77ca60667ae0735873deb77c86f5e07035b03ee73b37178ff7b4de52c model-00035-of-00050.safetensors
|
40 |
+
15a27b712bcd57b83b49caa0c3232ebcfbdf5ad64ba62aacc1020acf26fc166e model-00036-of-00050.safetensors
|
41 |
+
a05f17910d666abc459d8d1d9a57910e8272fd3a0deef0116320e29b2a58a918 model-00037-of-00050.safetensors
|
42 |
+
ae2a209ac113abf2f81a53ac26586c3ccc9b989cd6a1b0f1e84e8f7d81652ddc model-00038-of-00050.safetensors
|
43 |
+
e063bf4869da03c4b56124693dd196d08eda0631a04b7d8994c58d8cd36777d1 model-00039-of-00050.safetensors
|
44 |
+
7fec77ab33d2cf0a87f81fddab6d858d0be0adf9b8c82c4a6a761f7c56c8c481 model-00040-of-00050.safetensors
|
45 |
+
861aff0cedf845494452249e078c3c90505b842f474c1541ddeca714d3ad70a3 model-00041-of-00050.safetensors
|
46 |
+
e2cef40c1a04f49b8588a57051e3869f65c44eb76fcd0f152d4b3a6c5f1903fd model-00042-of-00050.safetensors
|
47 |
+
1451df32a996c063c67857e660c883a1dd9ce1b22cf2eb85f7ff063d45f72a0e model-00043-of-00050.safetensors
|
48 |
+
b34765a9955a85acc3ecc24add3cef81237a175607bce058af49d392f9dcec42 model-00044-of-00050.safetensors
|
49 |
+
10b06dfa35374fda739482d5c09f0c627e9bfc52c2510a7fd36e4f274696754d model-00045-of-00050.safetensors
|
50 |
+
37d6c7a1a8b54b1961a8622f773e79c10100b7693a3870af4fa8e2267b6f76cb model-00046-of-00050.safetensors
|
51 |
+
77c0520eda2c286849cc4bf00f8453d1d3d87aff2a72954bd03535784b308cbe model-00047-of-00050.safetensors
|
52 |
+
2089c1308443ec3d24ce8699e0ae65087b817c8c45b12d3d304b6257aef7b13e model-00048-of-00050.safetensors
|
53 |
+
216f1ad616d7955ad2ae010aa10a97baf9b1ae6e68e20122be99995640f3bb9a model-00049-of-00050.safetensors
|
54 |
+
a9fe65daa03c1fd29b72f4c7ae72b3ff9ee57f78e469dbc7002cb81e9c2b8973 model-00050-of-00050.safetensors
|
55 |
+
9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347 tokenizer.model
|
model.safetensors.index.json
ADDED
The diff for this file is too large to render.
See raw diff
|
|
output-00001-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:0847459d5670c2f923ba61dff8545528f6a3c9ce0bec10f509ceeba843789b8e
|
3 |
+
size 8537005000
|
output-00002-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:62e123e1dd10dbea2118e93b679e1cd1c191ff143a6064c5f00af9ff54dd3208
|
3 |
+
size 8587619288
|
output-00003-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:db397f754bbdc9a1090659389a69bb39323b57645f979190d2cbec37bcadec94
|
3 |
+
size 8582610584
|
output-00004-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:e97223ba97d989733604b01128abcdbbada8a771f4c30c95ee63bf27d8d5abf1
|
3 |
+
size 8503445912
|
output-00005-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ab68f54aa854a7a4dcd794996bb5a56fb3803cdfba5075fcfc0603a4c3e4c431
|
3 |
+
size 8517072248
|
output-00006-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:559ac6204ec36684beaf75c03d354fff48b413000a776ea5babad63a124ea04f
|
3 |
+
size 8509539704
|
output-00007-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:39255303ea8494084c07eee5b2a41747971ad71345c9ca1afcdfc3b2ce1b14a4
|
3 |
+
size 8569957784
|
output-00008-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:d2dd906c346b7be52b529a57d2dfe140afb131dd8e458cbaf91926e5acc742b5
|
3 |
+
size 8458230120
|
output-00009-of-00009.safetensors
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:043281df2819e0bdf3f5585db176a2ecb5810e65487003e9cafc0a49267fe61b
|
3 |
+
size 3584033232
|
special_tokens_map.json
ADDED
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"additional_special_tokens": [
|
3 |
+
"<unk>",
|
4 |
+
"<s>",
|
5 |
+
"</s>"
|
6 |
+
],
|
7 |
+
"bos_token": {
|
8 |
+
"content": "<s>",
|
9 |
+
"lstrip": false,
|
10 |
+
"normalized": false,
|
11 |
+
"rstrip": false,
|
12 |
+
"single_word": false
|
13 |
+
},
|
14 |
+
"eos_token": {
|
15 |
+
"content": "</s>",
|
16 |
+
"lstrip": false,
|
17 |
+
"normalized": false,
|
18 |
+
"rstrip": false,
|
19 |
+
"single_word": false
|
20 |
+
},
|
21 |
+
"unk_token": {
|
22 |
+
"content": "<unk>",
|
23 |
+
"lstrip": false,
|
24 |
+
"normalized": false,
|
25 |
+
"rstrip": false,
|
26 |
+
"single_word": false
|
27 |
+
}
|
28 |
+
}
|
tokenizer.json
ADDED
The diff for this file is too large to render.
See raw diff
|
|
tokenizer.model
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
|
3 |
+
size 499723
|
tokenizer_config.json
ADDED
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"added_tokens_decoder": {
|
3 |
+
"0": {
|
4 |
+
"content": "<unk>",
|
5 |
+
"lstrip": false,
|
6 |
+
"normalized": false,
|
7 |
+
"rstrip": false,
|
8 |
+
"single_word": false,
|
9 |
+
"special": true
|
10 |
+
},
|
11 |
+
"1": {
|
12 |
+
"content": "<s>",
|
13 |
+
"lstrip": false,
|
14 |
+
"normalized": false,
|
15 |
+
"rstrip": false,
|
16 |
+
"single_word": false,
|
17 |
+
"special": true
|
18 |
+
},
|
19 |
+
"2": {
|
20 |
+
"content": "</s>",
|
21 |
+
"lstrip": false,
|
22 |
+
"normalized": false,
|
23 |
+
"rstrip": false,
|
24 |
+
"single_word": false,
|
25 |
+
"special": true
|
26 |
+
}
|
27 |
+
},
|
28 |
+
"additional_special_tokens": [
|
29 |
+
"<unk>",
|
30 |
+
"<s>",
|
31 |
+
"</s>"
|
32 |
+
],
|
33 |
+
"bos_token": "<s>",
|
34 |
+
"clean_up_tokenization_spaces": false,
|
35 |
+
"eos_token": "</s>",
|
36 |
+
"legacy": false,
|
37 |
+
"model_max_length": 1000000000000000019884624838656,
|
38 |
+
"pad_token": null,
|
39 |
+
"sp_model_kwargs": {},
|
40 |
+
"tokenizer_class": "LlamaTokenizer",
|
41 |
+
"unk_token": "<unk>",
|
42 |
+
"use_default_system_prompt": true
|
43 |
+
}
|