theblackcat102
commited on
Commit
•
f1bb80a
1
Parent(s):
ef72ee3
Update README.md
Browse files
README.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1 |
---
|
2 |
-
license:
|
3 |
---
|
4 |
|
5 |
-
A asymmetric version of bigscience's mt0 large model, which trim down like [Meena chatbot](https://arxiv.org/pdf/2001.09977.pdf) from Google. An interesting aspect of Meena is that they use a small encoder, big decoder architecture.
|
6 |
|
7 |
> Meena has a single Evolved Transformer encoder block and 13 Evolved Transformer decoder blocks, as illustrated below. The encoder is responsible for processing the conversation context to help Meena understand what has already been said in the conversation. The decoder then uses that information to formulate an actual response. Through tuning the hyper-parameters, we discovered that a more powerful decoder was the key to higher conversational quality.
|
8 |
|
@@ -57,4 +57,4 @@ input :こんにちは!お元気</s>
|
|
57 |
trimmed output : !,
|
58 |
```
|
59 |
|
60 |
-
note that it's impossible to have the performance of the orignal model since roughly 30% of the weights were trimmed away.
|
|
|
1 |
---
|
2 |
+
license: mit
|
3 |
---
|
4 |
|
5 |
+
A asymmetric version of [bigscience's mt0 large model](https://huggingface.co/bigscience/mt0-large), which trim down like [Meena chatbot](https://arxiv.org/pdf/2001.09977.pdf) from Google. An interesting aspect of Meena is that they use a small encoder, big decoder architecture.
|
6 |
|
7 |
> Meena has a single Evolved Transformer encoder block and 13 Evolved Transformer decoder blocks, as illustrated below. The encoder is responsible for processing the conversation context to help Meena understand what has already been said in the conversation. The decoder then uses that information to formulate an actual response. Through tuning the hyper-parameters, we discovered that a more powerful decoder was the key to higher conversational quality.
|
8 |
|
|
|
57 |
trimmed output : !,
|
58 |
```
|
59 |
|
60 |
+
note that it's impossible to have the performance of the orignal model since roughly 30% of the weights were trimmed away.
|