|
--- |
|
language: ja |
|
tags: |
|
- t5 |
|
- text2text-generation |
|
- pilota |
|
license: apache-2.0 |
|
--- |
|
|
|
# Pilota model for scud2query |
|
|
|
A model for [Pilota](https://github.com/megagonlabs/pilota) trained with <https://github.com/megagonlabs/scud2query>. |
|
|
|
- ``scud`` |
|
- Fine tuned model of [t5-base-japanese-web (with Byte-fallback, 8K)](https://huggingface.co/megagonlabs/t5-base-japanese-web-8k) |
|
- The original model is distributed in [the Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
- ``scorer`` |
|
- Fine tuned model of [LINE DistilBERT Japanese](https://huggingface.co/line-corporation/line-distilbert-base-japanese) |
|
- The original model is distributed in [the Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
|
|
## Usage |
|
|
|
1. Install [Pilota](https://github.com/megagonlabs/pilota) |
|
2. Prepare inputs |
|
- Command |
|
|
|
```bash |
|
echo -e 'ι¨ε±γ«ε·θ΅εΊ«γγγγ¨θ―γγγ¬γ³γΏγ«γΌγ΅γΌγγΉγγγγγγ«γγcustomerγγεΈζγγγ' | python -m pilota.convert.plain2request | tee input.jsonl |
|
``` |
|
|
|
- Output |
|
|
|
```json |
|
{"context":null,"utterance":"ι¨ε±γ«ε·θ΅εΊ«γγγγ¨θ―γγγ¬γ³γΏγ«γΌγ΅γΌγγΉγγγγγγ«γγcustomerγγεΈζγγγ","sentences":null,"meta":{}} |
|
``` |
|
|
|
3. Feed it to Pilota |
|
- Command |
|
|
|
```console |
|
pilota -m megagonlabs/pilota_scud2query --batch_size 1 --outlen 60 --nbest 1 --beam 5 < input.jsonl |
|
``` |
|
|
|
- Output (Formatted by ``jq .``) |
|
|
|
```json |
|
[ |
|
{ |
|
"scuds_nbest": [ |
|
[ |
|
"ι¨ε±γ«ε·θ΅εΊ«γγγγ" |
|
] |
|
], |
|
"original_ranks": [ |
|
0 |
|
], |
|
"scores": [ |
|
0.9769772589206696 |
|
], |
|
"scores_detail": [ |
|
{ |
|
"OK": 0.9232575297355652, |
|
"incorrect_none": 0.0034886503126472235, |
|
"lack": 0.023772092536091805, |
|
"limited": 0.013821585103869438, |
|
"untruth": 0.04332486167550087 |
|
} |
|
], |
|
"sentence": "ι¨ε±γ«ε·θ΅εΊ«γγγγ¨θ―γγ" |
|
}, |
|
{ |
|
"scuds_nbest": [ |
|
[ |
|
"γ¬γ³γΏγ«γΌγ΅γΌγγΉγγγγγγ«γ γ" |
|
] |
|
], |
|
"original_ranks": [ |
|
0 |
|
], |
|
"scores": [ |
|
0.9876023113727569 |
|
], |
|
"scores_detail": [ |
|
{ |
|
"OK": 0.9586743712425232, |
|
"incorrect_none": 0.004059707745909691, |
|
"lack": 0.0024317132774740458, |
|
"limited": 0.007630097679793835, |
|
"untruth": 0.04025880992412567 |
|
} |
|
], |
|
"sentence": "γ¬γ³γΏγ«γΌγ΅γΌγγΉγγγγγγ«γγcustomerγγεΈζγγγ" |
|
} |
|
] |
|
``` |
|
|
|
## License |
|
|
|
Apache License 2.0 |
|
|
