cbdb commited on
Commit
6edcc8c
1 Parent(s): 81de5d9

Edited README.md

Browse files
Files changed (1) hide show
  1. README.md +65 -9
README.md CHANGED
@@ -1,23 +1,79 @@
1
  ---
 
 
 
2
  license: mit
3
  ---
4
- **BertForSequenceClassification model (Classical Chinese)**
 
5
 
6
  This BertForSequenceClassification Classical Chinese model is intended to predict whether a Classical Chinese sentence is a letter title (书信标题) or not. This model is first inherited from the BERT base Chinese model (MLM), and finetuned using a large corpus of Classical Chinese language (3GB textual dataset), then concatenated with the BertForSequenceClassification architecture to perform a binary classification task.
7
 
8
- **Labels: 0 = non-letter, 1 = letter**
 
 
9
 
10
- **Model description**
 
11
 
12
- The BertForSequenceClassification model architecture inherits the BERT base model and concatenates a fully-connected linear layer to perform a binary-class classification task.
13
 
14
- **Masked language modeling (MLM):** The masked language modeling architecture randomly masks 15% of the words in the inputs, and the model is trained to predict the masked words. The BERT base model uses this MLM architecture and is pre-trained on a large corpus of data. BERT is proven to produce robust word embedding and can capture rich contextual and semantic relationships. Our model inherits the publicly available pre-trained BERT Chinese model trained on modern Chinese data. To perform a Classical Chinese letter classification task, we first finetuned the model using a large corpus of Classical Chinese data (3GB textual data), and then connected it to the BertForSequenceClassification architecture for Classical Chinese letter classification.
15
 
16
- **Sequence classification:** the model concatenates a fully-connected linear layer to output the probability of each class. In our binary classification task, the final linear layer has two classes.
17
 
18
- **Intended uses & limitations**
19
  Note that this model is primiarly aimed at predicting whether a Classical Chinese sentence is a letter title (书信标题) or not.
20
 
21
- **How to use**
22
- You can use this model directly with a pipeline for masked language modeling:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
23
 
 
 
 
 
 
1
  ---
2
+ language: en
3
+ tags:
4
+ - SequenceClassification
5
  license: mit
6
  ---
7
+
8
+ # BertForSequenceClassification model (Classical Chinese)
9
 
10
  This BertForSequenceClassification Classical Chinese model is intended to predict whether a Classical Chinese sentence is a letter title (书信标题) or not. This model is first inherited from the BERT base Chinese model (MLM), and finetuned using a large corpus of Classical Chinese language (3GB textual dataset), then concatenated with the BertForSequenceClassification architecture to perform a binary classification task.
11
 
12
+ ### Labels: 0 = non-letter, 1 = letter
13
+
14
+ ## Model description
15
 
16
+ The BertForSequenceClassification model architecture inherits the BERT base model and concatenates a fully-connected linear layer to perform a binary-class classification task.More precisely, it
17
+ was pretrained with two objectives:
18
 
19
+ - Masked language modeling (MLM): The masked language modeling architecture randomly masks 15% of the words in the inputs, and the model is trained to predict the masked words. The BERT base model uses this MLM architecture and is pre-trained on a large corpus of data. BERT is proven to produce robust word embedding and can capture rich contextual and semantic relationships. Our model inherits the publicly available pre-trained BERT Chinese model trained on modern Chinese data. To perform a Classical Chinese letter classification task, we first finetuned the model using a large corpus of Classical Chinese data (3GB textual data), and then connected it to the BertForSequenceClassification architecture for Classical Chinese letter classification.
20
 
21
+ - Sequence classification: the model concatenates a fully-connected linear layer to output the probability of each class. In our binary classification task, the final linear layer has two classes.
22
 
23
+ ## Intended uses & limitations
24
 
 
25
  Note that this model is primiarly aimed at predicting whether a Classical Chinese sentence is a letter title (书信标题) or not.
26
 
27
+ ### How to use
28
+
29
+ Note that this model is primiarly aimed at predicting whether a Classical Chinese sentence is a letter title (书信标题) or not.
30
+
31
+ Here is how to use this model to get the features of a given text in PyTorch:
32
+
33
+ ```python
34
+ from transformers import BertTokenizer
35
+ from transformers import BertForSequenceClassification
36
+ import torch
37
+ from numpy import exp
38
+
39
+ tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')
40
+ model_path = '/content/drive/MyDrive/CBDB/Letter_Classifier/model/letter_classifer_epoch2' # here
41
+ model = BertForSequenceClassification.from_pretrained(model_path,
42
+ output_attentions=False,
43
+ output_hidden_states=False)
44
+
45
+ def softmax(vector):
46
+ e = exp(vector)
47
+ return e / e.sum()
48
+
49
+ def predict_class(test_sen):
50
+ tokens_test = tokenizer.encode_plus(
51
+ test_sen,
52
+ add_special_tokens=True,
53
+ return_attention_mask=True,
54
+ padding=True,
55
+ max_length=max_seq_len,
56
+ return_tensors='pt',
57
+ truncation=True
58
+ )
59
+
60
+ test_seq = torch.tensor(tokens_test['input_ids'])
61
+ test_mask = torch.tensor(tokens_test['attention_mask'])
62
+
63
+ # get predictions for test data
64
+ with torch.no_grad():
65
+ outputs = model(test_seq, test_mask)
66
+ outputs = outputs.logits.detach().cpu().numpy()
67
+
68
+ softmax_score = softmax(outputs)
69
+ pred_class_dict = {k:v for k, v in zip(label2idx.keys(), softmax_score[0])}
70
+ return pred_class_dict
71
+
72
+ max_seq_len = 512
73
+ label2idx = {'not-letter': 0,'letter': 1}
74
+ idx2label = {v:k for k,v in label2idx.items()}
75
 
76
+ test_sen = '上丞相康思公書'
77
+ pred_class_dict = predict_class(test_sen)
78
+ print(pred_class_dict)
79
+ ```