Edited README.md
Browse files
README.md
CHANGED
@@ -1,23 +1,79 @@
|
|
1 |
---
|
|
|
|
|
|
|
2 |
license: mit
|
3 |
---
|
4 |
-
|
|
|
5 |
|
6 |
This BertForSequenceClassification Classical Chinese model is intended to predict whether a Classical Chinese sentence is a letter title (书信标题) or not. This model is first inherited from the BERT base Chinese model (MLM), and finetuned using a large corpus of Classical Chinese language (3GB textual dataset), then concatenated with the BertForSequenceClassification architecture to perform a binary classification task.
|
7 |
|
8 |
-
|
|
|
|
|
9 |
|
10 |
-
|
|
|
11 |
|
12 |
-
The
|
13 |
|
14 |
-
|
15 |
|
16 |
-
|
17 |
|
18 |
-
**Intended uses & limitations**
|
19 |
Note that this model is primiarly aimed at predicting whether a Classical Chinese sentence is a letter title (书信标题) or not.
|
20 |
|
21 |
-
|
22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
23 |
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
language: en
|
3 |
+
tags:
|
4 |
+
- SequenceClassification
|
5 |
license: mit
|
6 |
---
|
7 |
+
|
8 |
+
# BertForSequenceClassification model (Classical Chinese)
|
9 |
|
10 |
This BertForSequenceClassification Classical Chinese model is intended to predict whether a Classical Chinese sentence is a letter title (书信标题) or not. This model is first inherited from the BERT base Chinese model (MLM), and finetuned using a large corpus of Classical Chinese language (3GB textual dataset), then concatenated with the BertForSequenceClassification architecture to perform a binary classification task.
|
11 |
|
12 |
+
### Labels: 0 = non-letter, 1 = letter
|
13 |
+
|
14 |
+
## Model description
|
15 |
|
16 |
+
The BertForSequenceClassification model architecture inherits the BERT base model and concatenates a fully-connected linear layer to perform a binary-class classification task.More precisely, it
|
17 |
+
was pretrained with two objectives:
|
18 |
|
19 |
+
- Masked language modeling (MLM): The masked language modeling architecture randomly masks 15% of the words in the inputs, and the model is trained to predict the masked words. The BERT base model uses this MLM architecture and is pre-trained on a large corpus of data. BERT is proven to produce robust word embedding and can capture rich contextual and semantic relationships. Our model inherits the publicly available pre-trained BERT Chinese model trained on modern Chinese data. To perform a Classical Chinese letter classification task, we first finetuned the model using a large corpus of Classical Chinese data (3GB textual data), and then connected it to the BertForSequenceClassification architecture for Classical Chinese letter classification.
|
20 |
|
21 |
+
- Sequence classification: the model concatenates a fully-connected linear layer to output the probability of each class. In our binary classification task, the final linear layer has two classes.
|
22 |
|
23 |
+
## Intended uses & limitations
|
24 |
|
|
|
25 |
Note that this model is primiarly aimed at predicting whether a Classical Chinese sentence is a letter title (书信标题) or not.
|
26 |
|
27 |
+
### How to use
|
28 |
+
|
29 |
+
Note that this model is primiarly aimed at predicting whether a Classical Chinese sentence is a letter title (书信标题) or not.
|
30 |
+
|
31 |
+
Here is how to use this model to get the features of a given text in PyTorch:
|
32 |
+
|
33 |
+
```python
|
34 |
+
from transformers import BertTokenizer
|
35 |
+
from transformers import BertForSequenceClassification
|
36 |
+
import torch
|
37 |
+
from numpy import exp
|
38 |
+
|
39 |
+
tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')
|
40 |
+
model_path = '/content/drive/MyDrive/CBDB/Letter_Classifier/model/letter_classifer_epoch2' # here
|
41 |
+
model = BertForSequenceClassification.from_pretrained(model_path,
|
42 |
+
output_attentions=False,
|
43 |
+
output_hidden_states=False)
|
44 |
+
|
45 |
+
def softmax(vector):
|
46 |
+
e = exp(vector)
|
47 |
+
return e / e.sum()
|
48 |
+
|
49 |
+
def predict_class(test_sen):
|
50 |
+
tokens_test = tokenizer.encode_plus(
|
51 |
+
test_sen,
|
52 |
+
add_special_tokens=True,
|
53 |
+
return_attention_mask=True,
|
54 |
+
padding=True,
|
55 |
+
max_length=max_seq_len,
|
56 |
+
return_tensors='pt',
|
57 |
+
truncation=True
|
58 |
+
)
|
59 |
+
|
60 |
+
test_seq = torch.tensor(tokens_test['input_ids'])
|
61 |
+
test_mask = torch.tensor(tokens_test['attention_mask'])
|
62 |
+
|
63 |
+
# get predictions for test data
|
64 |
+
with torch.no_grad():
|
65 |
+
outputs = model(test_seq, test_mask)
|
66 |
+
outputs = outputs.logits.detach().cpu().numpy()
|
67 |
+
|
68 |
+
softmax_score = softmax(outputs)
|
69 |
+
pred_class_dict = {k:v for k, v in zip(label2idx.keys(), softmax_score[0])}
|
70 |
+
return pred_class_dict
|
71 |
+
|
72 |
+
max_seq_len = 512
|
73 |
+
label2idx = {'not-letter': 0,'letter': 1}
|
74 |
+
idx2label = {v:k for k,v in label2idx.items()}
|
75 |
|
76 |
+
test_sen = '上丞相康思公書'
|
77 |
+
pred_class_dict = predict_class(test_sen)
|
78 |
+
print(pred_class_dict)
|
79 |
+
```
|