Commit
•
7f2de48
1
Parent(s):
0e15208
Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,83 @@
|
|
1 |
---
|
2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: mit
|
3 |
+
language: ja
|
4 |
+
tags:
|
5 |
+
- luke
|
6 |
+
- pytorch
|
7 |
+
- transformers
|
8 |
+
- commonsenseqa
|
9 |
+
- commonsense-qa
|
10 |
+
- CommonsenseQA
|
11 |
+
- commonsense_qa
|
12 |
+
|
13 |
---
|
14 |
+
|
15 |
+
# このモデルはluke-japanese-baseをファインチューニングして、JCommonsenseQA(選択式応答)に用いれるようにしたものです。
|
16 |
+
このモデルはluke-japanese-baseを
|
17 |
+
yahoo japan/JGLUEのJCommonsenseQA( https://github.com/yahoojapan/JGLUE )
|
18 |
+
を用いてファインチューニングしたものです。
|
19 |
+
|
20 |
+
選択式の質問応答タスクに用いることができます。
|
21 |
+
|
22 |
+
# This model is fine-tuned model for commonsenseqa which is based on luke-japanese-base
|
23 |
+
|
24 |
+
This model is fine-tuned by using yahoo japan JGLUE JCommonsenseQA dataset.
|
25 |
+
|
26 |
+
You could use this model for commonsenseqa tasks.
|
27 |
+
|
28 |
+
# モデルの精度 accuracy of model
|
29 |
+
0.8007149240393296
|
30 |
+
|
31 |
+
# How to use 使い方
|
32 |
+
以下のコードを実行することで、commonsenseqaタスクを解かせることができます。
|
33 |
+
please execute this code.
|
34 |
+
```python
|
35 |
+
from transformers import AutoTokenizer, AutoModelForMultipleChoice
|
36 |
+
import torch
|
37 |
+
import numpy as np
|
38 |
+
|
39 |
+
# modelのロード
|
40 |
+
tokenizer = AutoTokenizer.from_pretrained('Mizuiro-sakura/luke-japanese-base-commonsenseqa')
|
41 |
+
model = AutoModelForMultipleChoice.from_pretrained('Mizuiro-sakura/luke-japanese-base-commonsenseqa')
|
42 |
+
|
43 |
+
# 質問と選択肢の代入
|
44 |
+
question = '電子機器で使用される最も主要な電子回路基板の事をなんと言う?'
|
45 |
+
choice1 = '掲示板'
|
46 |
+
choice2 = 'パソコン'
|
47 |
+
choice3 = 'マザーボード'
|
48 |
+
choice4 = 'ハードディスク'
|
49 |
+
choice5 = 'まな板'
|
50 |
+
|
51 |
+
# トークン化(エンコーディング・形態素解析)する
|
52 |
+
token = tokenizer([question,question,question,question,question],[choice1,choice2,choice3,choice4,choice5],return_tensors='pt',padding=True)
|
53 |
+
leng=len(token['input_ids'][0])
|
54 |
+
|
55 |
+
# modelに入力するための下準備
|
56 |
+
X1 = np.empty(shape=(1, 5, leng))
|
57 |
+
X2 = np.empty(shape=(1, 5, leng))
|
58 |
+
X1[0, :, :] = token['input_ids']
|
59 |
+
X2[0, :, :] = token['attention_mask']
|
60 |
+
|
61 |
+
# modelにトークンを入力する
|
62 |
+
results = model(torch.tensor(X1).to(torch.int64),torch.tensor(X2).to(torch.int64))
|
63 |
+
|
64 |
+
# 最も高い値のインデックスを取得する
|
65 |
+
max_result=torch.argmax(results.logits)
|
66 |
+
print(max_result)
|
67 |
+
```
|
68 |
+
|
69 |
+
|
70 |
+
# what is Luke? Lukeとは?[1]
|
71 |
+
LUKE (Language Understanding with Knowledge-based Embeddings) is a new pre-trained contextualized representation of words and entities based on transformer. LUKE treats words and entities in a given text as independent tokens, and outputs contextualized representations of them. LUKE adopts an entity-aware self-attention mechanism that is an extension of the self-attention mechanism of the transformer, and considers the types of tokens (words or entities) when computing attention scores.
|
72 |
+
|
73 |
+
LUKE achieves state-of-the-art results on five popular NLP benchmarks including SQuAD v1.1 (extractive question answering), CoNLL-2003 (named entity recognition), ReCoRD (cloze-style question answering), TACRED (relation classification), and Open Entity (entity typing). luke-japaneseは、単語とエンティティの知識拡張型訓練済み Transformer モデルLUKEの日本語版です。LUKE は単語とエンティティを独立したトークンとして扱い、これらの文脈を考慮した表現を出力します。
|
74 |
+
|
75 |
+
# Acknowledgments 謝辞
|
76 |
+
Lukeの開発者である山田先生とStudio ousiaさんには感謝いたします。 I would like to thank Mr.Yamada @ikuyamada and Studio ousia @StudioOusia.
|
77 |
+
|
78 |
+
# Citation
|
79 |
+
[1]@inproceedings{yamada2020luke, title={LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention}, author={Ikuya Yamada and Akari Asai and Hiroyuki Shindo and Hideaki Takeda and Yuji Matsumoto}, booktitle={EMNLP}, year={2020} }
|
80 |
+
|
81 |
+
|
82 |
+
|
83 |
+
|