init project

Signed-off-by: mymusise <mymusise1@gmail.com>

README.md

@@ -0,0 +1,51 @@
+---
+language: zh
+widget:
+- text: "今天是下雨天"
+- text: "走向森林"
+---
+
+<h1 align="center">
+CPM
+</h1>
+
+
+CPM(Chinese Pre-Trained Language Models), which has 2.6B parameters, made by the research team of Beijing Zhiyuan Institute of artificial intelligence and Tsinghua University @TsinghuaAI.
+
+[repo: CPM-Generate](https://github.com/TsinghuaAI/CPM-Generate)
+
+The One Thing You Need to Know is this model is not uploaded by official, the conver script is [here](https://github.com/mymusise/CPM-TF2Transformer/blob/main/transfor_CMP.ipynb)
+
+# Overview
+
+- **Language model**: CPM
+- **Model size**: 2.6B parameters
+- **Language**: Chinese
+
+# How to use
+
+How to use this model directly from the 🤗/transformers library:
+
+```python
+from transformers import XLNetTokenizer, TFGPT2LMHeadModel
+
+tokenizer = XLNetTokenizer.from_pretrained('mymusise/CPM-Third-Party')
+model = TFGPT2LMHeadModel.from_pretrained("mymusise/CPM-Third-Party")
+```
+
+How to generate text
+
+```python
+from transformers import TextGenerationPipeline
+text_generater = TextGenerationPipeline(model, tokenizer)
+
+display(text_generater('今天天气不错', max_length=20))
+display(text_generater('今天天气不错', max_length=20, do_sample=True, top_k=10, top_p=0.95))
+display(text_generater('今天天气不错', max_length=20, do_sample=True, top_k=10, top_p=0.95, repetition_penalty=2))
+
+display(text_generater('天下武功, 唯快不破', max_length=30))
+display(text_generater('天下武功, 唯快不破', max_length=30, do_sample=True, top_k=10))
+```
+
+![avatar](https://github.com/mymusise/CPM-TF2Transformer/raw/main/example-cpm.png)
+

config.json

@@ -0,0 +1,36 @@
+{
+  "activation_function": "gelu",
+  "architectures": [
+    "TFGPT2LMHeadModel"
+  ],
+  "attn_pdrop": 0.0,
+  "bos_token_id": 50256,
+  "embd_pdrop": 0.0,
+  "eos_token_id": 50256,
+  "gradient_checkpointing": false,
+  "initializer_range": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "model_type": "gpt2",
+  "n_ctx": 1024,
+  "n_embd": 2560,
+  "n_head": 32,
+  "n_inner": null,
+  "n_layer": 32,
+  "n_positions": 1024,
+  "pad_token_id": 0,
+  "resid_pdrop": 0.0,
+  "summary_activation": null,
+  "summary_first_dropout": 0.0,
+  "summary_proj_to_labels": true,
+  "summary_type": "cls_index",
+  "summary_use_proj": true,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 120
+    }
+  },
+  "tokenizer_class": "XLNetTokenizer",
+  "use_cache": true,
+  "vocab_size": 30000
+}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25c1d178d54901291c1735cd2ae0788be90df4de01fb445e8a8a998cab35ba43
+size 713229