initial publish

README.md

@@ -6,4 +6,80 @@ pipeline_tag: summarization
 tags:
 - mT5
 - summarization
----
+---
+
+# HeackMT5-ZhSum100k: A Summarization Model for Chinese Texts
+
+This model, `heack/HeackMT5-ZhSum100k`, is a fine-tuned mT5 model for Chinese text summarization tasks. It was trained on a diverse set of Chinese datasets and is able to generate coherent and concise summaries for a wide range of texts.
+
+## Model Details
+
+- Model: mT5
+- Language: Chinese
+- Training data: Mainly Chinese Financial News Sources, NO BBC or CNN source. Training data contains 100k lines.
+- Finetuning epochs: 10
+
+## Evaluation Results
+
+The model achieved the following results:
+
+- ROUGE-1: 56.46
+- ROUGE-2: 45.81
+- ROUGE-L: 52.98
+- ROUGE-Lsum: 20.22
+
+## Usage
+
+Here is how you can use this model for text summarization:
+
+```python
+from transformers import MT5ForConditionalGeneration, T5Tokenizer
+
+model = MT5ForConditionalGeneration.from_pretrained("heack/HeackMT5-ZhSum100k")
+tokenizer = T5Tokenizer.from_pretrained("heack/HeackMT5-ZhSum100k")
+
+chunk = """
+财联社5月22日讯，据平安包头微信公众号消息，近日，包头警方发布一起利用人工智能（AI）实施电信诈骗的典型案例，福州市某科技公司法人代表郭先生10分钟内被骗430万元。
+4月20日中午，郭先生的好友突然通过微信视频联系他，自己的朋友在外地竞标，需要430万保证金，且需要公对公账户过账，想要借郭先生公司的账户走账。
+基于对好友的信任，加上已经视频聊天核实了身份，郭先生没有核实钱款是否到账，就分两笔把430万转到了好友朋友的银行卡上。郭先生拨打好友电话，才知道被骗。骗子通过智能AI换脸和拟声技术，佯装好友对他实施了诈骗。
+值得注意的是，骗子并没有使用一个仿真的好友微信添加郭先生为好友，而是直接用好友微信发起视频聊天，这也是郭先生被骗的原因之一。骗子极有可能通过技术手段盗用了郭先生好友的微信。幸运的是，接到报警后，福州、包头两地警银迅速启动止付机制，成功止付拦截336.84万元，但仍有93.16万元被转移，目前正在全力追缴中。
+"""
+inputs = tokenizer.encode("summarize: " + chunk, return_tensors='pt', max_length=512, truncation=True)
+summary_ids = model.generate(inputs, max_length=150, num_beams=4, length_penalty=1.5, no_repeat_ngram_size=2)
+summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
+
+print(summary)
+
+包头警方发布一起利用AI实施电信诈骗典型案例:法人代表10分钟内被骗430万元
+```
+## Credits
+This model is trained and maintained by KongYang from Shanghai Jiao Tong University. For any questions, please reach out to me at my WeChat ID: kongyang.
+
+## License
+This model is released under the CC BY-NC-SA 4.0 license.
+
+## Other
+这个模型基于google/mt5-base进行迭代训练，epoch=10，在我的GTX 4070机器上跑了大概7天时间。训练集有10万行，主要是国内的中文新闻，不存在BBC，所以翻译的时候不会出现类似于“本文不代表bbc观点”之类的语言。并且我的数据集也经过深度过滤，基本不会出现意外的文字，可以非常好的用来进行缩略。我的缩略主要输入分为300字的chunk，然后再进行缩略，然后再把文字join就可以完成大段的缩略。
+后续会进一步在我的100万行数据源进行训练，并开源模型，敬请期待。
+另外，我已经把此缩略用在了我的本地知识库项目中，项目链接为：
+https://github.com/erickong/document.ai
+
+另外感谢此项目：
+https://github.com/csebuetnlp/xl-sum/tree/master/seq2seq
+
+This model is iteratively trained based on google/mt5-base with 10 epochs, taking approximately 7 days on my machine equipped with a GTX 4070 graphics card. The training set consists of 100,000 lines, mainly domestic Chinese news, and does not include sources such as BBC, hence translations such as "This article does not represent the views of BBC" will not appear. My dataset has also undergone deep filtering, virtually eliminating unexpected text, making it excellent for summarization. The primary input for my summarization is chunks of 300 words, which are then summarized, and the resulting text can be joined together to summarize larger sections.
+
+In the future, I plan to further train this model with my dataset of 1 million lines and release the model. Please stay tuned.
+
+Additionally, I have already incorporated this summarization into my local knowledge base project. The link to the project is: https://github.com/erickong/document.ai
+
+## Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@misc{kongyang2023heackmt5zhsum100k,
+    title={HeackMT5-ZhSum100k: A Large-Scale Multilingual Abstractive Summarization for Chinese Texts},
+    author={Kong Yang},
+    year={2023}
+}

config.json

@@ -0,0 +1,31 @@
+{
+  "_name_or_path": "heack/HeackMT5-ZhSum100k",
+  "architectures": [
+    "MT5ForConditionalGeneration"
+  ],
+  "d_ff": 2048,
+  "d_kv": 64,
+  "d_model": 768,
+  "decoder_start_token_id": 0,
+  "dropout_rate": 0.1,
+  "eos_token_id": 1,
+  "feed_forward_proj": "gated-gelu",
+  "initializer_factor": 1.0,
+  "is_encoder_decoder": true,
+  "layer_norm_epsilon": 1e-06,
+  "length_penalty": 0.6,
+  "max_length": 84,
+  "model_type": "mt5",
+  "num_beams": 4,
+  "num_decoder_layers": 12,
+  "num_heads": 12,
+  "num_layers": 12,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_num_buckets": 32,
+  "tie_word_embeddings": false,
+  "tokenizer_class": "T5Tokenizer",
+  "transformers_version": "4.10.0.dev0",
+  "use_cache": true,
+  "vocab_size": 250112
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18c8d23ba8ca99183a85c3dd7e7e6b87bb653b7296789a50288c94c8a616a1a9
+size 2329706751

special_tokens_map.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>"}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef78f86560d809067d12bac6c09f19a462cb3af3f54d2b8acbba26e1433125d6
+size 4309802

tokenizer_config.json

@@ -0,0 +1 @@
+{"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 0, "additional_special_tokens": null, "special_tokens_map_file": "/home/patrick/.cache/torch/transformers/685ac0ca8568ec593a48b61b0a3c272beee9bc194a3c7241d15dcadb5f875e53.f76030f3ec1b96a8199b2593390c610e76ca8028ef3d24680000619ffb646276", "tokenizer_file": null, "name_or_path": "google/mt5-base"}