Repository: IDEA-CCNL/GTS-Engine (language: python)

Issue:
Dev sunqianguo updatereadme
更新readme，不涉及代码修改



Discussion / hints:

Fix patch:
diff --git a/.gitignore b/.gitignore
index 3e94ffc..0f11a39 100644
--- a/.gitignore
+++ b/.gitignore
@@ -14,7 +14,6 @@ dist/
 downloads/
 eggs/
 .eggs/
-lib/
 lib64/
 parts/
 sdist/
diff --git a/README.md b/README.md
index 04564cf..cce3906 100644
--- a/README.md
+++ b/README.md
@@ -22,7 +22,7 @@ GTS引擎（GTS-Engine）是一款开箱即用且性能强大的自然语言理
 
 GTS引擎提出“用AI生产AI”的理念，它基于封神榜开源体系的基础模型，并在下游进行了有监督预训练，同时集成了多种小样本学习技术，搭建了一个模型自动生产的流水线。
 
-GTS-Engine计划开源两个系列的引擎，分别为**乾坤鼎**系列和**八卦炉**系列。
+GTS-Engine已开源两个系列的引擎，分别为**乾坤鼎**系列和**八卦炉**系列。
 - **乾坤鼎**系列是以1.3B参数的大模型为底座，通过大模型结合多种小样本学习技术进行训练和推理的引擎。
 - **八卦炉**系列是以110M参数的base模型为底座，融合大模型、数据增强、协同训练等方法进行训练和推理的引擎。
 
@@ -36,13 +36,16 @@ GTS-Engine计划开源两个系列的引擎，分别为**乾坤鼎**系列和**
 
 ## 更新日志
 
-- `2022/11/22`：分类任务增加高级模式：支持用户上传无标注数据进行Self Training，进一步提升效果；
-- `2022/11/18`：正式发布引擎Beta版本，FewCLUE榜单任务训练细节将陆续公开；
+- `2022/12/30`：正式发布八卦炉引擎Beta版本，支持以110M参数的base模型为底座的分类、信息抽取类型任务；
+- `2022/11/22`：乾坤鼎分类任务增加高级模式：支持用户上传无标注数据进行Self Training，进一步提升效果；
+- `2022/11/18`：正式发布乾坤鼎引擎Beta版本，FewCLUE榜单任务训练细节将陆续公开；
 
 ## 即将发布
 
 - 更好的使用体验，更清晰的接口，更完善的模块划分，代码快速迭代中；
-- 增加信息抽取任务，基于统一范式的信息抽取模型即将公开；
+- 八卦炉引擎分类任务的高级模式，使用大模型生成数据、数据打标进一步提升base模型效果；
+- 八卦炉引擎自然语言推理、句子匹配任务；
+
 
 ## 安装
 
@@ -53,6 +56,7 @@ GTS-Engine计划开源两个系列的引擎，分别为**乾坤鼎**系列和**
     - 其他依赖请参考`requirements.txt`
 - 硬件环境需求
     - 乾坤鼎引擎至少需要一张24G显存的RTX3090，使用V100和A100能够获得更好的性能体验；
+    - 八卦炉引擎至少需要一张8G显存的GPU，使用V100和A100能够获得更好的性能体验；
 
 更多环境需求和软件依赖请参考我们的[文档](https://gts-engine-doc.readthedocs.io/en/latest/docs/requirement.html)。
 
@@ -217,6 +221,10 @@ client.start_train(
   val_data="tnews_val.json",
   test_data="tnews_test.json",
   label_data="tnews_label.json",
+  train_mode="standard",
+  max_num_epoch=1,
+  min_num_epoch=1, 
+  seed=42, 
   gpuid=0)
 ```
 
diff --git a/examples/ner/run_inference_bagualu_ner.sh b/examples/ner/run_inference_bagualu_ner.sh
new file mode 100644
index 0000000..5861393
--- /dev/null
+++ b/examples/ner/run_inference_bagualu_ner.sh
@@ -0,0 +1,22 @@
+#!/bin/bash
+
+WORK_DIR=$(dirname $(dirname $(dirname $(readlink -f "$0"))))
+echo "working directory: $WORK_DIR"
+
+cd $WORK_DIR
+
+TASK_DIR=$WORK_DIR/tasks/text_ie_example
+
+if [ ! -d $TASK_DIR ]; then
+    echo "task dir $TASK_DIR not exists, please train first."
+    exit 1
+fi
+
+export CUDA_VISIBLE_DEVICES=5
+python gts_engine/gts_engine_inference.py \
+    --task_dir=$TASK_DIR \
+    --engine_type=bagualu \
+    --task_type=ie \
+    --input_path=examples/ner/weibo_test.json \
+    --output_path=$TASK_DIR/output.json \
+    --max_epochs=50
diff --git a/examples/ner/run_train_bagualu_ner.sh b/examples/ner/run_train_bagualu_ner.sh
new file mode 100644
index 0000000..2208eca
--- /dev/null
+++ b/examples/ner/run_train_bagualu_ner.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+
+set -x
+
+WORK_DIR=$(dirname $(dirname $(dirname $(readlink -f "$0"))))
+echo "working directory: $WORK_DIR"
+
+cd $WORK_DIR
+mkdir -p $WORK_DIR/tasks
+mkdir -p $WORK_DIR/pretrained
+
+PRETRAINED_DIR=$WORK_DIR/pretrained
+TASK_DIR=$WORK_DIR/tasks/text_ie_example
+mkdir -p $TASK_DIR
+
+export CUDA_VISIBLE_DEVICES=4
+python gts_engine/gts_engine_train.py \
+    --engine_type=bagualu \
+    --train_mode=standard \
+    --task_dir=$TASK_DIR \
+    --task_type=ie \
+    --train_data=weibo_train.json \
+    --valid_data=weibo_val.json \
+    --test_data=weibo_test.json \
+    --data_dir=$WORK_DIR/examples/ner \
+    --save_path=$TASK_DIR/outputs \
+    --pretrained_model_dir=$PRETRAINED_DIR \
+    --train_batchsize=2 \
+    --valid_batchsize=4 \
+    --max_len=512 \
+    --max_epochs=1 \
+    --min_epochs=1 \
+    --seed=123
diff --git a/examples/ner/weibo_train.json b/examples/ner/weibo_train.json
new file mode 100644
index 0000000..b1512fe
--- /dev/null
+++ b/examples/ner/weibo_train.json
@@ -0,0 +1,100 @@
+{"task": "实体识别", "text": "科技全方位资讯智能，快捷的汽车生活需要有三屏一云爱你", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "对，输给一个女人，的成绩。失望", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "今天下午起来看到外面的太阳。。。。我第一反应竟然是强烈的想回家泪想我们一起在嘉鱼个时候了。。。。有好多好多的话想对你说李巾凡想要瘦瘦瘦成李帆我是想切开云朵的心", "entity_list": [{"entity_text": "嘉鱼", "entity_type": "地点", "entity_index": [38, 40]}, {"entity_text": "李巾凡", "entity_type": "人名", "entity_index": [59, 62]}, {"entity_text": "李帆", "entity_type": "人名", "entity_index": [68, 70]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "今年拜年不短信，就在微博拜大年寻龙记", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "浑身酸疼，两腿无力，眼神呆滞，怎么了", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "明显紧张状态没出来，失误多。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "三十年前，老爹带我来这里开会，今天打了个颠倒。我在这里", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "好活动呀，给力的商家，必须支持，希望以后多多办活动跟我们互动哈巧慧铣", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "人生如戏，导演是自己蜡烛", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "听说小米开卖了，刚刚预约了台。我爱小米手机因为他是迄今为止最快的小米手机月日中午点，希望能顺利抢到台小米手机分享预约地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "最热时尚榜女人不坏，男人不爱，一个男女必看的微博花心", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我参与了南都深圳读本发起的投票涨薪最慢十大行业，有你吗？，我投给了不上榜同样杯具这个选项。你也快来表态吧：", "entity_list": [{"entity_text": "南都", "entity_type": "地理位置", "entity_index": [4, 6]}, {"entity_text": "深圳", "entity_type": "地理位置", "entity_index": [6, 8]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "好嗯，能赢取耶，赞耶建设完后，俺肯定要第一个去游玩墨水冬冬好嗯，能赢取耶，赞耶建设完后，俺肯定要第一个去游玩", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "易经中的个智慧第六十二感悟：小过无错大过难容正确对待过失人非圣贤，孰能无过？在我们的工作、生活中，过失是很难避免的事，关键要学会正确对待过失，开动脑筋思考过失，从过失中得到启迪。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "一切表明素质，道德决定一切央视新闻海豚走了海豚送医后，由于呛水严重于昨晚点死亡。", "entity_list": [{"entity_text": "央视新闻", "entity_type": "组织机构", "entity_index": [13, 17]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "元奖品，真给力希望我会有好的运气哦", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "如果你总是快要迟到了，才费力地起床，一整天无精打采、哈欠连连，你需要的是尽快设定起床的形式，这些形式将能帮助你不用调较闹钟，也能愉悦地睁开眼睛。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "曾若彤璐骑单车的少年要考驾照驰鸣天下苦逼青年杨素晗不屑与你装霄", "entity_list": [{"entity_text": "曾若彤", "entity_type": "人名", "entity_index": [0, 3]}, {"entity_text": "杨素晗", "entity_type": "人名", "entity_index": [22, 25]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "程仁田达晨创投汗戴湧哈哈", "entity_list": [{"entity_text": "程仁田", "entity_type": "人名", "entity_index": [0, 3]}, {"entity_text": "达晨创", "entity_type": "人名", "entity_index": [3, 6]}, {"entity_text": "汗戴湧", "entity_type": "人名", "entity_index": [7, 10]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "倩萍小姐胡小亭转闲置还有天就开奖啦小亭会专程去深圳为你们寄奖品大家加油转发哦", "entity_list": [{"entity_text": "倩萍", "entity_type": "人名", "entity_index": [0, 2]}, {"entity_text": "胡小亭", "entity_type": "人名", "entity_index": [4, 7]}, {"entity_text": "小亭", "entity_type": "人名", "entity_index": [17, 19]}, {"entity_text": "深圳", "entity_type": "地理位置", "entity_index": [23, 25]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "美得呀〜顾天池苦逼青年杨素晗旻日记肖立伟嘻嘻嘻嘻嘻嘻美啊", "entity_list": [{"entity_text": "顾天池", "entity_type": "人名", "entity_index": [4, 7]}, {"entity_text": "杨素晗", "entity_type": "人名", "entity_index": [11, 14]}, {"entity_text": "肖立伟", "entity_type": "人名", "entity_index": [17, 20]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "乘机撇下熊孩子们出去玩哈哈哈哈哈哈赞啊赞啊凡先生今天，和为了给哥哥们加油去了伦敦。和为准备了什么样的惊喜呢？请大家期待哦〜", "entity_list": [{"entity_text": "伦敦", "entity_type": "地理位置", "entity_index": [38, 40]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "左右拿什么拯救你我的右脑京大蠢左右还真是平衡啊米曹米羔九九号大展见卧槽郁川月百年夜行重置子宫野霜曜一帆开门我来送水了有妹子了不起啊冯基杂碎跪下左右右脑都快开发完了左脑还没开始实", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "有奖转发有奖转发抱抱等人品爆发转发哈哈哈转发转发地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "那什么拯救你，大宝冒小疯瑞舍去老师羊羊思绪跑了调咩咩咩噗。。。＝真的是妹子进汉子出。。。滴滴溜儿溜小连韩非子小海快看快看魅色妹子进，汉子归", "entity_list": [{"entity_text": "韩非子", "entity_type": "人名", "entity_index": [51, 54]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "欧美时尚热门榜心谢耳朵。。", "entity_list": [{"entity_text": "欧", "entity_type": "地理位置", "entity_index": [0, 1]}, {"entity_text": "美", "entity_type": "地理位置", "entity_index": [1, 2]}, {"entity_text": "谢耳朵", "entity_type": "人名", "entity_index": [8, 11]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "为什么我会基本上都看过。天哪我居然基本上都看过潮音乐：看过", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我叫赵新民流哈子了。。。", "entity_list": [{"entity_text": "赵新民", "entity_type": "人名", "entity_index": [2, 5]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "起点或终点泪因特古拉燃尽踌躇我爱我的妈妈。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "当看破一切的时候，才知道，原来失去比拥有更塌实。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "看完了，心真的疼斯罗布莱特泪伤不起人与动物以及动物之间的情真意切，悲伤于永久的分离天上人间。", "entity_list": [{"entity_text": "斯罗布莱特", "entity_type": "人名", "entity_index": [8, 13]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "：是你望着我的样子、是我眼里只有你、是感觉非常非常特别、是没人比我更爱你心", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "谢谢亲的关心亲亲我已经习惯了太开心我还好啦，只有有点担心到处乱爬的虫", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我就是激动雯子小菇凉至于嘛，不久赞了你一下嘛转发苦逼青年杨素晗回复雯子小菇凉主要你夸我我太激动了爱你爱你爱你雯子小菇凉回复苦逼青年杨素晗骗你干吗，真心电到了，越来越有范了，以后跟领导你混了阴险", "entity_list": [{"entity_text": "雯子小菇凉", "entity_type": "人名", "entity_index": [5, 10]}, {"entity_text": "杨素晗", "entity_type": "人名", "entity_index": [28, 31]}, {"entity_text": "雯子", "entity_type": "人名", "entity_index": [33, 35]}, {"entity_text": "雯子", "entity_type": "人名", "entity_index": [54, 56]}, {"entity_text": "杨素晗", "entity_type": "人名", "entity_index": [65, 68]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "下雨真烦，究竟要下到什么时候？？泪泪泪龙舟水都五洗甘厉害吧。鄙视鄙视我的妈丫，怎么上班？？我在", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "超搞···转发微博。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "如果还有两天假期，我发现自己可能还可以再宅两天挖鼻屎挖鼻屎", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "自娱自乐也是一种消遣。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "初夏清新；盛夏灿烂；末夏秋凉；细细品味每个日子的专属夏天；爱汽车不错", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "华克山庄免税店年末感恩转发微博赢取圣诞好礼笑哈哈朋友不必很多，知心就好；牵挂不必很多，动情就好；地址：", "entity_list": [{"entity_text": "华克山庄", "entity_type": "地点", "entity_index": [0, 4]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "新浪微博客户端版全新发布你有多久没旅行了？回味旅途点滴，一起来分享旅行中的风景、美食与自拍，完成任务还有大奖哦快开启旅行模式，带着微博去旅行飞机下载地址：", "entity_list": [{"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [0, 2]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "网友问怀孕两个月可不可以玩手机？，我的回答：没问题。可以玩。一次别玩太久，注意休息来自热门应用妈妈问答，详细内容", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "看见前女友和她的新欢走在一起的时候，已经无处可躲了，只好硬着头皮上去打招呼哎呀，好久不见，你儿子都这么高了。இɷஇ", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "支持非常给力的活动又是一段幸福的时光，又是一次真诚地祝福喜欢期待惊喜降临，地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "美食美味给力啊大家都来支持吧，好想得到啊，希望有好运。地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "亲们，该结婚的都滚去结婚了噢耶啦啦啦啦周康康康康要幺女。", "entity_list": [{"entity_text": "周康康康康", "entity_type": "人名", "entity_index": [19, 24]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "草你吗，跟里面的坑一样万恶，可恶的小笼包。哪里的死畜牲大便君人越来越冷漠大家都有老的一天是有因果轮回的宋灵姣夹夹夹生是天生畜生怒天气热了怎么办。。。。能说什么呢。。。。小小丸子童鞋转发微博", "entity_list": [{"entity_text": "宋灵姣", "entity_type": "人名", "entity_index": [51, 54]}, {"entity_text": "小小丸子", "entity_type": "人名", "entity_index": [84, 88]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "快来参加凡客凡儿到底大型主题活动，范儿不仅是潮流、个性范儿更是种生活态度自信展现诠释自我每期均有精彩互动活动，与专家交流穿搭秘诀，参与达人评选，获胜者更有机会获得凡客大奖。好看，好玩，得大奖，只有凡客凡儿立即参与", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "来学校的第一件事就是吃喝赌只能暗叹堕落啊", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "发表了博文：踏青偶记踏青偶记闷闷不乐，便开车奔老远的那座山去，求散散心，呼吸点儿新鲜空气。没有看到期待中红色枫叶，见不到想象中的层林尽染，本来有点失望。可错过一道风景，却遇到另一", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "很纯有木有熙熙鴨小碎", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "新版微博客户端，好听，好看，更好玩自定义个人封面；音、视频直接播放；好友圈好友消息不错过；支持屏蔽广告微博；广场音乐、电影、美食等频道升级。快试试一键听歌：下载地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "对科鲁兹说：万能微博元团购百万助购金活动，参与转发活动就可赢取万元助购传播奖，还有万元助购大奖我的朋友们快来参加吧", "entity_list": [{"entity_text": "科鲁兹", "entity_type": "人名", "entity_index": [1, 4]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "发表了博文做自己的英雄白色恤搭配什么都普通可是简单有时会与众不同拥有一颗平凡而真实的梦就拥有一整片天空我决定每天随身携带笑容只需要做感动自己最有用闭上眼吹着海风世界因为有", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "怎么没有新月冰冰如今生活好了，精神富足了，问君一句：肾好？甚好", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "对于脱毛，你了解几何？|背心，短袖，裙子，夏天的必备。而面对于夏天，很多并不是那么的自信，而且总是会担心自己腋下以及腿部的体毛会影响到自己的魅力，所以总是会寻找着很多的脱毛方法。林林总总，有着很多的脱毛办法。对于一", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "老是想通过别人的眼睛看世界，那我们的眼睛慢慢就会瞎了。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我正在申请微博达人，微博达人不仅拥有专属的身份标识，炫酷的微博达人勋章，还可以结识更多跟你有共同兴趣爱好的朋友，获得更多粉丝，拥有更多知己，更可优先享受新浪微博各种线上线下达人活动带来的生活便利和优惠。你也来申请吧", "entity_list": [{"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [76, 78]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "于丹经典语录：你是我的单项选择，其他的都是错误答案", "entity_list": [{"entity_text": "于丹", "entity_type": "人名", "entity_index": [0, 2]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "麻烦各位同学好友投上你神圣的一票：嘻嘻微博投票：第三届文明班级英语，我爱你", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我想问：那个人，你在哪？", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "迷茫无奈的汉希叫你嫌弃我右哼哼左哼哼", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我运动我美丽女人节，我想对你说京东晒·幸福关注广禾堂月子餐官方微博，玩游戏位准妈咪好友，赢免费游台湾我信赖的食品品牌我最喜爱的京东商城品牌春天来了淘宝女装海量新品，你最爱什么风格？", "entity_list": [{"entity_text": "京东商城", "entity_type": "组织机构", "entity_index": [63, 67]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "汗会不会跟内购一样上不去回头再来个已售罄黑线流浪艺术家明天会不会冷场啊，全世界都在观望刘强东搬个板凳坐在国美、苏宁店里，享受着空调暖气，走走路锻炼身体还可以赚钱退休后干什么？到京东做价格情报员", "entity_list": [{"entity_text": "刘强东", "entity_type": "人名", "entity_index": [43, 46]}, {"entity_text": "国美", "entity_type": "组织机构", "entity_index": [52, 54]}, {"entity_text": "苏宁", "entity_type": "组织机构", "entity_index": [55, 57]}, {"entity_text": "京东", "entity_type": "组织机构", "entity_index": [88, 90]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我亦领教过。从化资讯网注意", "entity_list": [{"entity_text": "从化资讯网", "entity_type": "组织机构", "entity_index": [6, 11]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "怒我妈感冒的时候经常吃宁财神凉茶，我", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "敢在我面前得瑟，吃我一记夺命剪刀脚～～", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "花心花心花心其实是个爱疯的妞宝宝要仔细开启学霸模式还是易昭蹲坑宋宋宋开启放假模式美人鱼宋院长开启学霸模式一起来玩捉迷藏吧小伙伴们有木有兴趣来嗨皮一下哈哈哈哈哈哈", "entity_list": [{"entity_text": "妞宝宝", "entity_type": "人名", "entity_index": [13, 16]}, {"entity_text": "易昭", "entity_type": "人名", "entity_index": [27, 29]}, {"entity_text": "宋院长", "entity_type": "人名", "entity_index": [43, 46]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "✒️提前收到的生日礼物好喜欢", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "想在老家让厨艺好的姑妈开家私房菜馆，接待来荔波旅游的游客，让他们尝到当地的美食，但是不知道会不会有生意，来荔波旅游的盆友们怎么样？期待回答呦", "entity_list": [{"entity_text": "荔波", "entity_type": "地理位置", "entity_index": [21, 23]}, {"entity_text": "荔波", "entity_type": "地理位置", "entity_index": [53, 55]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "育儿小贴士腹泻的孩子要吃些什么呢？继续喂养，母乳喂养儿继续喂母乳喂养，母乳不会加重腹泻。非母乳喂养儿喂米汤、面汤、稀释奶等。一直到腹泻停止后周内都要喂容易消化的食物，否则腹泻停止后还会再次腹泻。避免含粗纤维的蔬菜和水果，高糖食物也会加重腹泻。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "未成年者禁止进入不是我们减不了肥，而是敌人太过强大。。。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "快来和我一起使用安全管家，防骚扰、手机杀毒、流量监控，全方位保护手机不管你信不信，我反正信了。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "新浪微博四周年丫头哒哒你非带我认识新浪微博，夏至不再来你帮我弄好的新浪微博，徐爱强你是我最早认识的微友哈哈哈哈我在", "entity_list": [{"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [0, 2]}, {"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [17, 19]}, {"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [33, 35]}, {"entity_text": "徐爱强", "entity_type": "人名", "entity_index": [38, 41]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "他给她留言。你的，我都懂。我的，你有太多不懂。不懂就留到下辈子。都说这辈子两个人要相互欠对方东西。这样下辈子才能遇到。欠的越多越好。亲爱的。我欠你一个未来。所以，下辈子等我找到你，生死相依。她回复。上辈子我们也这么说过。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "丫头哒哒哈哈，吃货就是你拉哈哈哈哈", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我不想，我不想，我不想，我不想，我不想，我不想，我不想", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我参与了华广汽车学院团学实践部发起的投票•第六届汽车文化节之车王争霸赛，我投给了酱油王这个选项。你也快来表态吧：", "entity_list": [{"entity_text": "华广汽车学院团学实践部", "entity_type": "组织机构", "entity_index": [4, 15]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "在读我不是一本型录★★★★★", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "不是像，而是就是陵猫猫哈哈蔷薇女主播陵猫猫这么像你呢，哈哈", "entity_list": [{"entity_text": "陵猫猫", "entity_type": "人名", "entity_index": [18, 21]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "特百惠山楂小姐相亲记很棒的活动，奖品我很喜欢，感谢小编为粉丝们争取这么好的活动，期待可以收获一份惊喜地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "祝粉丝多多，人气旺旺，我顺便也沾沾你的喜气。地址：", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我参与了美薇婷的投票大家认为生男孩好还是女孩好，我投给了女孩，你也快来表态吧", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "成人礼成人礼。。大师兄〜〜〜〜璐温人意", "entity_list": [{"entity_text": "璐温人意", "entity_type": "人名", "entity_index": [15, 19]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "手机坏了之后、每次我想拿起手机看点什么或找点什么的时候、就只能再一次告诉自己别看了、你根本什么都看不了。也许这样也能治治我的手机强迫症、没有手机的时候那是小时候。泪泪泪好吧我承认我在自我安慰。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "月号，上午杭州走起。。等我来找你哟。亲爱的嘻嘻嘻嘻嘻嘻嘻嘻嘻嘻我在", "entity_list": [{"entity_text": "杭州", "entity_type": "地理位置", "entity_index": [5, 7]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "和我弟玩这个玩得不亦乐乎，孩纸们还知道这是啥么。抱抱", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我早就知道了包嘉俊周苏飘在水里的蛋炒饭那一抹容颜的老去", "entity_list": [{"entity_text": "包嘉俊", "entity_type": "人名", "entity_index": [6, 9]}, {"entity_text": "周苏飘", "entity_type": "人名", "entity_index": [9, 12]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "诗焱被骗了。。。心理学微小说一直以为是拼的。。。", "entity_list": [{"entity_text": "诗焱", "entity_type": "人名", "entity_index": [0, 2]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "这辈子最对不起你的事情。。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "昆凌和锦荣在一起无语了。。。汗汗汗", "entity_list": [{"entity_text": "昆凌", "entity_type": "人名", "entity_index": [0, 2]}, {"entity_text": "锦荣", "entity_type": "人名", "entity_index": [3, 5]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "有奖转发好活动给力，支持", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "都吾知咩財務，行去收錢黑囗黑面，再系甘就投訴你。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我叫赵新民什么意思啊？？？", "entity_list": [{"entity_text": "赵新民", "entity_type": "人名", "entity_index": [2, 5]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "可是我们很快乐呵呵骂人秘籍小时候，我们没有", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "要很多很多的爱。如果没有爱，那么就很多很多的钱，如果两件都没有，有健康也是好的。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "对南海杨知明说：补充上问：最多可减免班主任工作年限年我的理解：副高评审班主任年限为年，信息技术等学科可减免，但最起码有年班主任是不是这样的意思？", "entity_list": [{"entity_text": "南海", "entity_type": "地理位置", "entity_index": [1, 3]}, {"entity_text": "杨知明", "entity_type": "人名", "entity_index": [3, 6]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我懂你的心泪曾若彤苦逼青年杨素晗知道没？偷笑我为美发狂开心的时候要想起我，难受的时候要记得你还有我。", "entity_list": [{"entity_text": "曾若彤", "entity_type": "人名", "entity_index": [6, 9]}, {"entity_text": "杨素晗", "entity_type": "人名", "entity_index": [13, 16]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "吃完鸭脖辣得不行，喝水不能解决问题。我觉得我好聪明啊，我想到了刷牙结果鸭脖的辣和牙膏的清凉在嘴里激情碰撞的时候，那种感觉简直爽爆生病泪泪", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "分享歌曲，萧亚轩类似爱情我在这里", "entity_list": [{"entity_text": "萧亚轩", "entity_type": "人名", "entity_index": [5, 8]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
diff --git a/examples/ner/weibo_val.json b/examples/ner/weibo_val.json
new file mode 100644
index 0000000..4192fc2
--- /dev/null
+++ b/examples/ner/weibo_val.json
@@ -0,0 +1,50 @@
+{"task": "实体识别", "text": "口腔溃疡加上这玩意～酸酸甜甜好滋味。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "用最大努力去做值得的事，不计结果，笑看人生。哈哈哈哈哈哈我在", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "无论如何，不能让某个人成为你的全部，若是有天他们离开了，你将一无所有", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "总要相信那句付出会有回报。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "清清淡然朴普梅缘荷忆", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "老百姓心新乡新闻网话说这几天新乡天气还好吧偷笑", "entity_list": [{"entity_text": "新乡", "entity_type": "地理位置", "entity_index": [4, 6]}, {"entity_text": "新乡", "entity_type": "地理位置", "entity_index": [14, 16]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "太感谢你一直安慰我奇怪的黑发，不要说我鞭策你，我们一起努力和你一起真的很容易吃很多诶下星期一起羽毛球爱你爱你爱你爱你我下周就去买了那条裙子和你姐妹装啦", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "大学同宿舍三年，以为都还算了解你。断估唔到你已经系二岁孩子的爸了。许坚你始终都系甘耐人寻味。好高兴能参加你噶婚礼。祝福你我永远噶好舍友", "entity_list": [{"entity_text": "许坚", "entity_type": "人名", "entity_index": [33, 35]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "今天新入手的耳机我和我的小伙伴们都觉得好棒哟ω", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "超超超美的乐乐一看照片又长大啦去了给姨亲一口爱你", "entity_list": [{"entity_text": "乐乐", "entity_type": "人名", "entity_index": [5, 7]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我玩微城市，获得了金币，真不赖今天我玩微城市获得了金币。大家去玩微城市吧，每天登陆都会获得奖励开始游戏", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "创业最重要的原则先拥有一个客户再创业。点子一毛钱一打，凭一个想法打拼两年才赚到一块钱那太费劲了，我们和扎克伯格是完全不同的人。执行最重要？错唯一重要的是钱。有客户愿意付钱你的创业才不孤单，执行才会变得简单。", "entity_list": [{"entity_text": "扎克伯格", "entity_type": "人名", "entity_index": [51, 55]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "码通贩场贩在理查的置顶马给哥哥", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "上新啦这是一款天河石，它是信心的宝石。据西洋神秘传说，又绿又蓝的天河石，带有幸运，投机，化险为夷的能量，带着它，搓麻打牌很容易赢钱哦带着它，在职场中也会遇贵人哦能帮助摆脱负面能量，增进信心，达成目标亲，还在犹豫什么动起来", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "武大妹子录的，吊炸了给我最爱的贾老板翻唱歌曲中国原创音乐基地", "entity_list": [{"entity_text": "武大", "entity_type": "组织机构", "entity_index": [0, 2]}, {"entity_text": "贾老板", "entity_type": "人名", "entity_index": [15, 18]}, {"entity_text": "中国", "entity_type": "地理位置", "entity_index": [22, 24]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "哈哈，把手机都没收没收新浪校园挖鼻屎挖鼻屎这个你们觉得如何？手机签到制度桂林校园挖鼻屎辣么多人有两只手机呢", "entity_list": [{"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [11, 13]}, {"entity_text": "桂林", "entity_type": "地理位置", "entity_index": [36, 38]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "回复雯子小菇凉主要你夸我我太激动了爱你爱你爱你雯子小菇凉回复苦逼青年杨素晗骗你干吗，真心电到了，越来越有范了，以后跟领导你混了阴险", "entity_list": [{"entity_text": "雯子", "entity_type": "人名", "entity_index": [2, 4]}, {"entity_text": "雯子", "entity_type": "人名", "entity_index": [23, 25]}, {"entity_text": "杨素晗", "entity_type": "人名", "entity_index": [34, 37]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "下雪了，回不来了哈哈哈哈嘻嘻嘻嘻", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "对柜台熊孩子说：关注啦。。有什么情况？", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "秦时明月官方微博卫庄大人果然是忧国思民的典范哈哈", "entity_list": [{"entity_text": "卫庄大人", "entity_type": "人名", "entity_index": [8, 12]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "全球流行风尚八拜之交是指哪八拜这些你都知道吗？推荐关注神秘野史", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "这年头遇到个好人是件难事。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "做爱花的女性，女人如花。乐于直视他人的女性，自信的女人更美。到了时候就恋爱就生子的女人，会生活的女人更幸福。做个会做饭的女人，独立的女人更可爱。做爱读书的女人，睿智的女人更魅力。做深存感恩之心又独自远行的女人，这样的女人似一缕清风。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "今天的小老鼠好可爱������天气太冷了全部蜷在一堆雪我在", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我叫赵新民全球最热门视频呃，期待下一篇。。。", "entity_list": [{"entity_text": "赵新民", "entity_type": "人名", "entity_index": [2, 5]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "怎么还六月飞雪了呢，宁波晚报网友播报雨和雪都横着在飘也在沧海路和百丈东路交叉口吗？二二二理斯下雪了雨和雪都横着在飘", "entity_list": [{"entity_text": "宁波", "entity_type": "地理位置", "entity_index": [10, 12]}, {"entity_text": "晚报网", "entity_type": "组织机构", "entity_index": [12, 15]}, {"entity_text": "沧海路", "entity_type": "地点", "entity_index": [28, 31]}, {"entity_text": "百丈东路", "entity_type": "地点", "entity_index": [32, 36]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "阿袁在外地学习。一天，他发现生活费已提前用完，便忙给家里拍电报求援。电报上只有四个字：弹尽粮绝。没几天，阿袁收到家里的回电：顶住。", "entity_list": [{"entity_text": "阿袁", "entity_type": "人名", "entity_index": [0, 2]}, {"entity_text": "阿袁", "entity_type": "人名", "entity_index": [52, 54]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "刘心车推不动我要投诉堃说那个手柄拉下来就能推动了怎么办太丢人了", "entity_list": [{"entity_text": "刘心车", "entity_type": "人名", "entity_index": [0, 3]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "萧萧静的明天有明天徐妍徐妍", "entity_list": [{"entity_text": "萧萧静", "entity_type": "人名", "entity_index": [0, 3]}, {"entity_text": "徐妍", "entity_type": "人名", "entity_index": [9, 11]}, {"entity_text": "徐妍", "entity_type": "人名", "entity_index": [11, 13]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "匹克球星中国行我用分秒，完成了匹克球星安德鲁古德洛克的拼图游戏，申请参加月日兰德里匹克球迷见面会，夏木谢小玲好男人就是我我就是翁小贱，你们想不想与兰德里面对面？快来参加吧更有限量篮球好礼等你拿活动链接：", "entity_list": [{"entity_text": "匹克", "entity_type": "组织机构", "entity_index": [0, 2]}, {"entity_text": "匹克", "entity_type": "组织机构", "entity_index": [15, 17]}, {"entity_text": "匹克", "entity_type": "组织机构", "entity_index": [41, 43]}, {"entity_text": "中国", "entity_type": "地理位置", "entity_index": [4, 6]}, {"entity_text": "安德鲁古德洛克", "entity_type": "人名", "entity_index": [19, 26]}, {"entity_text": "兰德里", "entity_type": "人名", "entity_index": [38, 41]}, {"entity_text": "谢小玲", "entity_type": "人名", "entity_index": [51, 54]}, {"entity_text": "翁小贱", "entity_type": "人名", "entity_index": [63, 66]}, {"entity_text": "兰德里", "entity_type": "人名", "entity_index": [73, 76]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "突然很想念那些曾经亲密无间的朋友，一幕幕，不知道是不是有那么一时间我也曾被想起。。。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "分享个好玩的活动给你们，都来看看吧。如果有可能，你想住进哪部电影的房子里？", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "后半句，笑死人沧浪秋水一张帆啊白菜经典英文语录哈哈哈哈求白菜笑哈哈", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "九月菊花傲霜雪。秋天，枫叶被染红了，金黄的稻穗带来丰收的欢乐。更令人喜悦的是，菊花在这个时节盛开，千姿百态，万紫千红。菊花以自己的绚丽色彩、绰约风姿，把春天的色调再渡展现，把新生的魅力发挥出来。爹孩儿要去从军。军旅要吃很多苦的。孩儿就是去吃苦，我要比爹你更出色。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "他是韩国我是歌手拿冠军的那个吗？就是我们恨之入骨的那个打人的教官吗？怒嘻嘻练习生真的不是一般人能坚持下来的。每当遇到困难的时候想想自己喜欢的是以怎样的意志力坚持下来的就会觉得自己的问题真的都是渣渣。根本不算什么。", "entity_list": [{"entity_text": "韩国", "entity_type": "地理位置", "entity_index": [2, 4]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "你的良心何在，政府给你救助款都喂狗了。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "中秋节快乐，爸爸妈妈想你们了。哎不是唉这款不错唉", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "不可不学的瘦脸按摩法捏按面部肉多地方：涂上按摩霜，在面颊肉最多的地方捏捺，由内往外拉伸。揉按面颊：手掌紧贴双颊，由内到外按摩，大约分钟。托下颌：双手轻托下巴，有节律地做分钟。轻托脸部：双手放于下颌，轻轻按摩并提拉。双手轻拂面部：轻轻抚摸整个面部，停留分钟", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "赛琳曼琦张羽姑娘笑点低伦家唐唐唐唐秋云霖魏腾飞我想我是你的熊猫眼张先生", "entity_list": [{"entity_text": "赛琳曼琦", "entity_type": "人名", "entity_index": [0, 4]}, {"entity_text": "张羽", "entity_type": "人名", "entity_index": [4, 6]}, {"entity_text": "唐秋云", "entity_type": "人名", "entity_index": [16, 19]}, {"entity_text": "张先生", "entity_type": "人名", "entity_index": [32, 35]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "����������", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "新浪微博客户端全新升级信息流图片加载速度提升；发布器支持多张图片一键分享；新增、浪小花表情。观看视频，了解更多下载地址：", "entity_list": [{"entity_text": "浪小花", "entity_type": "人名", "entity_index": [40, 43]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "让我心碎，你做的比谁都到位。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "我想，我想，我想，我想，我想，我想，我想，我想，我想，我想，我想", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "必须转啊啊宇多田光之樱流泪泪泪明日香啊丽", "entity_list": [{"entity_text": "宇多田光", "entity_type": "人名", "entity_index": [5, 9]}, {"entity_text": "明日香", "entity_type": "人名", "entity_index": [15, 18]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "实际上，到最后我们都会在别人的上处于脱机状态。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "男女那回事最可怕的不是你娶了个你不爱的人，而是你发现她还是个灰常会化妆的女人。", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "必需收藏南瓜小狐狸已成精新浪娱乐分钟看年日本动画发展史", "entity_list": [{"entity_text": "新浪", "entity_type": "组织机构", "entity_index": [12, 14]}, {"entity_text": "日本", "entity_type": "地理位置", "entity_index": [20, 22]}], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "来听听我唱的常回家看看真情绵绵全是爱，击败了全国的人试听地址通过唱吧录制", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "不要解释那么多，直接扣扣我，迫不及待啦", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
+{"task": "实体识别", "text": "发表了一篇转载博文转载转载十类中国历代家训集锦", "entity_list": [], "choice": ["地理位置", "组织机构", "人名", "地点"]}
diff --git a/examples/text_classification/run_inference_bagualu.sh b/examples/text_classification/run_inference_bagualu.sh
new file mode 100644
index 0000000..fe3d80d
--- /dev/null
+++ b/examples/text_classification/run_inference_bagualu.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+WORK_DIR=$(dirname $(dirname $(dirname $(readlink -f "$0"))))
+echo "working directory: $WORK_DIR"
+
+cd $WORK_DIR
+
+TASK_DIR=$WORK_DIR/tasks/text_classification_example
+
+if [ ! -d $TASK_DIR ]; then
+    echo "task dir $TASK_DIR not exists, please train first."
+    exit 1
+fi
+
+export CUDA_VISIBLE_DEVICES=7 
+python gts_engine/gts_engine_inference.py \
+    --task_dir=$TASK_DIR \
+    --engine_type=bagualu \
+    --task_type=classification \
+    --input_path=examples/text_classification/tnews_test.json \
+    --output_path=tnews_output.json
diff --git a/examples/text_classification/run_train_bagualu.sh b/examples/text_classification/run_train_bagualu.sh
new file mode 100644
index 0000000..940b5b9
--- /dev/null
+++ b/examples/text_classification/run_train_bagualu.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+WORK_DIR=$(dirname $(dirname $(dirname $(readlink -f "$0"))))
+echo "working directory: $WORK_DIR"
+
+cd $WORK_DIR
+mkdir -p $WORK_DIR/tasks
+mkdir -p $WORK_DIR/pretrained
+
+PRETRAINED_DIR=$WORK_DIR/pretrained
+TASK_DIR=$WORK_DIR/tasks/text_classification_example
+mkdir -p $TASK_DIR
+
+export CUDA_VISIBLE_DEVICES=3 
+python gts_engine/gts_engine_train.py \
+    --engine_type=bagualu \
+    --train_mode=standard \
+    --task_dir=$TASK_DIR \
+    --task_type=classification \
+    --train_data=tnews_train.json \
+    --valid_data=tnews_val.json \
+    --test_data=tnews_test.json \
+    --label_data=tnews_label.json \
+    --data_dir=$WORK_DIR/examples/text_classification \
+    --save_path=$TASK_DIR/outputs \
+    --pretrained_model_dir=$PRETRAINED_DIR \
+    --train_batchsize=2 \
+    --valid_batchsize=4 \
+    --max_len=512 \
+    --max_epochs=1 \
+    --min_epochs=1 \
+    --seed=123
\ No newline at end of file
diff --git a/gts_engine/bagualu/__init__.py b/gts_engine/bagualu/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/gts_engine/bagualu/arguments/__init__.py b/gts_engine/bagualu/arguments/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/gts_engine/bagualu/arguments/ie/__init__.py b/gts_engine/bagualu/arguments/ie/__init__.py
new file mode 100644
index 0000000..b82fb7a
--- /dev/null
+++ b/gts_engine/bagualu/arguments/ie/__init__.py
@@ -0,0 +1,16 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from .arguments_std import TrainingArgumentsIEStd, InferenceArgumentsIEStd
diff --git a/gts_engine/bagualu/arguments/ie/arguments_std.py b/gts_engine/bagualu/arguments/ie/arguments_std.py
new file mode 100644
index 0000000..4f23ac7
--- /dev/null
+++ b/gts_engine/bagualu/arguments/ie/arguments_std.py
@@ -0,0 +1,295 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+from typing import Optional
+
+from ...lib.utils.path import mk_inexist_dir
+from ...lib.framework.base_arguments import BaseArguments, GeneralParser
+from ...lib.components.protocol_args import ProtocolArgsMixin
+
+
+class TrainingArgumentsIEStd(BaseArguments, ProtocolArgsMixin):
+    """ 信息抽取finetune参数 """
+
+    # 输出相关参数
+    @property
+    def ft_output_dir(self) -> str:
+        """ finetune输出路径 """
+        return os.path.join(self.protocol_args.output_dir)
+
+    @property
+    def log_dir(self) -> str:
+        return os.path.join(self.ft_output_dir, "log")
+
+    @property
+    def ckpt_dirpath(self) -> str:
+        """ checkpoint dirpath """
+        return os.path.join(self.ft_output_dir, "checkpoints")
+
+    @property
+    def tensorboard_dir(self) -> str:
+        """ tensorboard日志输出路径 """
+        return os.path.join(self.ft_output_dir, "tensorboard")
+
+    @property
+    def prediction_save_dir(self) -> str:
+        """ 预测结果保存路径 """
+        return os.path.join(self.ft_output_dir, "predictions")
+
+    @property
+    def model_save_dir(self) -> str:
+        """ 模型保存路径 """
+        return os.path.join(self.ft_output_dir, "model")
+
+    @property
+    def best_ckpt_path(self) -> str:
+        """ 最优checkpoint路径 """
+        return os.path.join(self.model_save_dir, "best.ckpt")
+
+    @property
+    def last_ckpt_path(self) -> str:
+        """ 最后一个checkpoint路径 """
+        return os.path.join(self.ckpt_dirpath, "last.ckpt")
+
+    @property
+    def onnx_saved_path(self) -> str:
+        """ onnx 模型保存路径 """
+        return os.path.join(self.model_save_dir, "model.onnx")
+
+    # 数据集相关参数
+
+    _train_data_path: Optional[str]
+    _dev_data_path: Optional[str]
+    _test_data_path: Optional[str]
+    _online_test_data_path: Optional[str]
+
+    @property
+    def train_data_path(self) -> str:
+        """ 训练集 """
+        default_path = os.path.join(self.protocol_args.input_dir, "train.jsonl")
+        return self._train_data_path if self._train_data_path else default_path
+
+    @property
+    def dev_data_path(self) -> str:
+        """ 验证集 """
+        default_path = os.path.join(self.protocol_args.input_dir, "val.jsonl")
+        return self._dev_data_path if self._dev_data_path else default_path
+
+    @property
+    def test_data_path(self) -> str:
+        """ 测试集 """
+        default_path = os.path.join(self.protocol_args.input_dir, "test.jsonl")
+        return self._test_data_path if self._test_data_path else default_path
+
+    @property
+    def online_test_data_path(self) -> str:
+        """ 无标注数据 """
+        default_path = os.path.join(self.protocol_args.input_dir, "test_online.jsonl")
+        return self._online_test_data_path if self._online_test_data_path else default_path
+
+    # 固定参数
+    logger: str = "ie_logger"
+    batch_size: int = 16
+    num_workers: int = 8
+    max_length: int = 512
+    learning_rate: float = 1e-5
+    weight_decay: float = 0.1
+    warmup: float = 0.06
+    loss: str = "dc"
+    loss_boost: float = 100000.0
+    log_step: int = 20
+    dc_smooth: float = 1.0
+    neg_drop: float = 0.4
+    distill_self: bool = False
+    sep_data: bool = False
+    sep_need_ori: bool = False
+    sep_need_neg: bool = False
+
+    cut_text: bool = True
+    cut_text_len: int = 400
+    cut_text_stride: int = 200
+
+    # trainer参数
+    max_steps: int = -1
+    max_epochs: int = 50
+    min_epochs: int = 1
+    precision: int = 16
+    check_val_every_n_epoch: int = 1
+    gradient_clip_val: float = 1.0
+    val_check_interval: float = 0.5
+    enable_progress_bar: bool = False
+    accumulate_grad_batches: int = 1
+    accelerator: str = "gpu"
+
+    # checkpoints参数
+    ckpt_monitor: str = "val_f1"
+    ckpt_mode: str = "max"
+    ckpt_save_last: bool = True
+    ckpt_filename: str = "model-{epoch:02d}-{val_f1:.4f}"
+    ckpt_save_top_k: int = 3
+    ckpt_every_n_train_steps: int = 100
+    ckpt_save_weights_only: bool = True
+
+    # 传入参数
+    dataset: str
+    entity_multi_label: bool
+    relation_multi_label: bool
+    threshold_ent: float
+    threshold_rel: float
+    gpus: int
+
+    def _add_args(self, parser: GeneralParser) -> None:
+        # 数据集参数
+        parser.add_argument("--dataset",
+                            dest="dataset",
+                            type=str,
+                            help="数据集名称",
+                            default="default")
+        parser.add_argument("--train_data_path",
+                            dest="_train_data_path",
+                            type=str,
+                            default=None,
+                            help="[可选]指定训练集路径")
+        parser.add_argument("--dev_data_path",
+                            dest="_dev_data_path",
+                            type=str,
+                            default=None,
+                            help="[可选]指定验证集路径")
+        parser.add_argument("--test_data_path",
+                            dest="_test_data_path",
+                            type=str,
+                            default=None,
+                            help="[可选]指定测试集路径")
+        parser.add_argument("--online_test_data_path",
+                            dest="_online_test_data_path",
+                            type=str,
+                            default=None,
+                            help="[可选]指定在线测试数据路径")
+
+        # 训练参数
+        parser.add_argument("--gpus",
+                            dest="gpus",
+                            type=str,
+                            default=None,
+                            help="[可选]指定gpu个数")
+        parser.add_argument("--entity_multi_label",
+                            action="store_true",
+                            dest="entity_multi_label",
+                            help="[可选]一个实体是否允许具备多种类型，默认不允许")
+        parser.add_argument("--relation_multi_label",
+                            action="store_true",
+                            dest="relation_multi_label",
+                            help="[可选]相同两个实体间是否允许多种关系，默认不允许")
+        parser.add_argument("--threshold_entity",
+                            dest="threshold_ent",
+                            type=float,
+                            default=0.5,
+                            help="[可选]实体阈值，默认0.5")
+        parser.add_argument("--threshold_relation",
+                            dest="threshold_rel",
+                            type=float,
+                            default=0.5,
+                            help="[可选]关系阈值，默认0.5")
+        parser.add_argument("--learning_rate",
+                            dest="learning_rate",
+                            type=float,
+                            default=1e-5,
+                            help="[可选]学习率，默认1e-5")
+        parser.add_argument("--accumulate_grad_batches",
+                            dest="accumulate_grad_batches",
+                            type=int,
+                            default=1,
+                            help="[可选]梯度累积，默认1")
+        parser.add_argument("--num_workers",
+                            dest="num_workers",
+                            type=int,
+                            default=8,
+                            help="[可选]工作数，默认8")
+        parser.add_argument("--max_length",
+                            dest="max_length",
+                            type=int,
+                            default=512,
+                            help="[可选]最大长度，默认512")
+        parser.add_argument("--val_check_interval",
+                            dest="val_check_interval",
+                            type=float,
+                            default=0.5,
+                            help="[可选]验证间隔，默认0.5")
+        parser.add_argument("--batch_size",
+                            dest="batch_size",
+                            type=int,
+                            default=16,
+                            help="[可选]train batch大小，默认16")
+        parser.add_argument("--max_epochs",
+                            dest="max_epochs",
+                            type=int,
+                            default=None,
+                            help="[可选]训练最大epoch数")
+        parser.add_argument("--min_epochs",
+                            dest="min_epochs",
+                            type=int,
+                            default=None,
+                            help="[可选]训练最小epoch数")
+
+    def _after_parse(self) -> None:
+        mk_inexist_dir(self.ft_output_dir)
+        mk_inexist_dir(self.log_dir)
+        mk_inexist_dir(self.ckpt_dirpath, clean=True)
+        mk_inexist_dir(self.tensorboard_dir)
+        mk_inexist_dir(self.prediction_save_dir, clean=True)
+        mk_inexist_dir(self.model_save_dir, clean=True)
+
+
+class InferenceArgumentsIEStd(BaseArguments):
+    """ 信息抽取finetune参数 """
+
+    task_dir: str
+    pretrained_model_root: str
+
+    max_length: int = 512
+    batch_size: int = 2
+
+    loss: str = "dc"
+    dc_smooth: float = 1.0
+    neg_drop: float = 0.4
+    distill_self: bool = False
+    entity_multi_label: bool = True
+    relation_multi_label: bool = True
+    threshold_ent: float = 0.5
+    threshold_rel: float = 0.5
+
+    @property
+    def model_save_dir(self) -> str:
+        """ 模型保存路径 """
+        return os.path.join(self.task_dir, "outputs/model")
+
+    @property
+    def best_ckpt_path(self) -> str:
+        """ 最优checkpoint路径 """
+        return os.path.join(self.model_save_dir, "best.ckpt")
+
+    def _add_args(self, parser: GeneralParser) -> None:
+        # 数据集参数
+        parser.add_argument("--task_dir",
+                            dest="task_dir",
+                            required=True,
+                            type=str,
+                            help="specific task directory")
+        parser.add_argument("--pretrained_model_root",
+                            dest="pretrained_model_root",
+                            required=True,
+                            type=str,
+                            help="path to pretrained model")
diff --git a/gts_engine/bagualu/arguments/text_classification/__init__.py b/gts_engine/bagualu/arguments/text_classification/__init__.py
new file mode 100644
index 0000000..f6f431d
--- /dev/null
+++ b/gts_engine/bagualu/arguments/text_classification/__init__.py
@@ -0,0 +1,1 @@
+from .arguments_std import TrainingArgumentsClfStd, InferenceArgumentsClfStd
\ No newline at end of file
diff --git a/gts_engine/bagualu/arguments/text_classification/arguments_std.py b/gts_engine/bagualu/arguments/text_classification/arguments_std.py
new file mode 100644
index 0000000..8a78057
--- /dev/null
+++ b/gts_engine/bagualu/arguments/text_classification/arguments_std.py
@@ -0,0 +1,57 @@
+from typing import Optional
+from pydantic import FilePath
+from pathlib import Path
+
+from ...lib.framework.base_arguments import GeneralParser
+from ...lib.components.lightning_callbacks.adaptive_val_intervals import (
+    ADAPTIVE_VAL_INTERVAL_MODE)
+from ...lib.framework.classification_finetune import (
+    BaseTrainingArgumentsClf,
+    BaseInferenceArgumentsClf)
+from ...lib.utils.detect_gpu_memory import detect_gpu_memory, decide_gpu
+
+
+class TrainingArgumentsClfStd(BaseTrainingArgumentsClf):
+
+    _aug_eda_path: Optional[FilePath]
+    precision: int=32
+    use_gradient_checkpointing: str="False"
+
+    @property
+    def aug_eda_path(self) -> FilePath:
+        """eda缓存文件"""
+        return self.input_dir / "eda_augment.json" if (
+            self._aug_eda_path is None) else self._aug_eda_path
+
+    @property
+    def memory_optimization_setting(self) -> None:
+        # 基于显卡显存大小，设定显存优化的参数
+        gpu_memory, gpu_cur_used_memory = detect_gpu_memory()
+        gpu_type = decide_gpu(gpu_memory)
+        self.precision = 16 if "low" in gpu_type else 32 
+        self.use_gradient_checkpointing = "True" if gpu_type == "lower_gpu" else "False"
+        print("基于当前显卡的显存大小 {}M, 设置training precision为{}, 设置use_gradient_checkpointing为{}".format(gpu_memory, self.precision, self.use_gradient_checkpointing))
+
+    def _add_args(self, parser: GeneralParser) -> None:
+        super()._add_args(parser)
+        parser.add_argument("--aug_eda_path", dest="_aug_eda_path",
+                            type=Path, default=None, help="[可选]指定eda文件缓存路径")
+        # 不传入时，才为false
+        parser.add_argument("--use_knn", dest="use_knn",
+                            default=False, action="store_true",
+                            help="whether or not to use knn component")
+        parser.add_argument("--rdrop_gate", dest="rdrop_gate",
+                            type=int, default=50,
+                            help="Determine whether to use rdrop based on the \
+                            number of categories in the classification task")
+        parser.add_argument("--use_rdrop", dest="use_rdrop",
+                            default=False, action="store_true",
+                            help="whether or not to use rdrop component")
+        parser.add_argument("--rdrop_alpha", dest="rdrop_alpha",
+                            type=int, default=5)
+    aug_eda_gate: bool = True
+    dev_resample_thres: int = 1000  # dev数据超过阈值进行重采样
+    validation_mode = ADAPTIVE_VAL_INTERVAL_MODE.ADAPTIVE
+
+class InferenceArgumentsClfStd(BaseInferenceArgumentsClf):
+    ...
diff --git a/gts_engine/bagualu/dataloaders/__init__.py b/gts_engine/bagualu/dataloaders/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/gts_engine/bagualu/dataloaders/ie/__init__.py b/gts_engine/bagualu/dataloaders/ie/__init__.py
new file mode 100644
index 0000000..1d5c172
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/__init__.py
@@ -0,0 +1,4 @@
+from .dataset import BagualuIEDataset
+from .data_model import BagualuIEDataModel
+from .dataset_utils import check_data, data_segment, data_segment_restore
+from .data_eval_utils import eval_entity_f1, eval_entity_relation_f1, eval_relation_f1
diff --git a/gts_engine/bagualu/dataloaders/ie/data_eval_utils.py b/gts_engine/bagualu/dataloaders/ie/data_eval_utils.py
new file mode 100644
index 0000000..002becd
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/data_eval_utils.py
@@ -0,0 +1,157 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import List, Tuple
+
+
+def eval_entity_f1(test_data: List[dict], pred_data: List[dict]) -> Tuple[float, tuple, tuple]:
+    """ 获取实体抽取f1
+
+    Args:
+        test_data (List[dict]): test data
+        pred_data (List[dict]): pred data
+
+    Returns:
+        Tuple[float, tuple, tuple]: (f1, recall, precise)
+    """
+    corr = 0
+    y_true = 0
+    y_pred = 0
+    assert len(test_data) == len(pred_data)
+    for test_sample, pred_sample in zip(test_data, pred_data):
+        assert test_sample['text'] == pred_sample['text']
+
+        y_true_set = {(e["entity_type"], e["entity_text"].strip()) for e in test_sample["entity_list"]}
+        y_true += len(y_true_set)
+
+        y_pred_set = {(e["entity_type"], e["entity_text"].strip()) for e in pred_sample["entity_list"]}
+        y_pred += len(y_pred_set)
+
+        corr += len(y_pred_set & y_true_set)
+
+
+    if y_pred <= 0:
+        precise = 0.
+    else:
+        precise = corr / y_pred
+
+    if y_true <= 0:
+        recall = 0.
+    else:
+        recall = corr / y_true
+    if precise + recall <= 0:
+        f1 = 0.
+    else:
+        f1 = 2 * precise * recall / (precise + recall)
+
+    return f1, (corr, y_true, recall), (corr, y_pred, precise)
+
+
+def eval_relation_f1(test_data: List[dict], pred_data: List[dict]) -> Tuple[float, tuple, tuple]:
+    """ 获取关系抽取f1
+
+    Args:
+        test_data (List[dict]): test data
+        pred_data (List[dict]): pred data
+
+    Returns:
+        Tuple[float, tuple, tuple]: (f1, recall, precise)
+    """
+    corr = 0
+    y_true = 0
+    y_pred = 0
+
+    for test_sample, pred_sample in zip(test_data, pred_data):
+
+        y_true_set = {
+            (spo["predicate"], spo["subject"]["entity_text"].strip(), spo["object"]["entity_text"].strip())
+            for spo in test_sample["spo_list"]
+        }
+        y_true += len(y_true_set)
+
+        y_pred_set = {
+            (spo["predicate"], spo["subject"]["entity_text"].strip(), spo["object"]["entity_text"].strip())
+            for spo in pred_sample["spo_list"]
+        }
+        y_pred += len(y_pred_set)
+
+        corr += len(y_pred_set & y_true_set)
+
+    if y_pred <= 0:
+        precise = 0.
+    else:
+        precise = corr / y_pred
+
+    if y_true <= 0:
+        recall = 0.
+    else:
+        recall = corr / y_true
+
+    if precise + recall <= 0:
+        f1 = 0.
+    else:
+        f1 = 2 * precise * recall / (precise + recall)
+
+    return f1, (corr, y_true, recall), (corr, y_pred, precise)
+
+
+def eval_entity_relation_f1(test_data: List[dict], pred_data: List[dict]) -> Tuple[float, tuple, tuple]:
+    """ 获取实体关系抽取f1
+
+    Args:
+        test_data (List[dict]): test data
+        pred_data (List[dict]): pred data
+
+    Returns:
+        Tuple[float, tuple, tuple]: (f1, recall, precise)
+    """
+    corr = 0
+    y_true = 0
+    y_pred = 0
+
+    for test_sample, pred_sample in zip(test_data, pred_data):
+
+        y_true_set = {
+            (spo["predicate"], spo["subject"]["entity_text"].strip(), spo["subject"]["entity_type"],
+            spo["object"]["entity_text"].strip(), spo["object"]["entity_type"])
+            for spo in test_sample["spo_list"]
+        }
+        y_true += len(y_true_set)
+
+        y_pred_set = {
+            (spo["predicate"], spo["subject"]["entity_text"].strip(), spo["subject"]["entity_type"],
+            spo["object"]["entity_text"].strip(), spo["object"]["entity_type"])
+            for spo in pred_sample["spo_list"]
+        }
+        y_pred += len(y_pred_set)
+
+        corr += len(y_pred_set & y_true_set)
+
+    if y_pred <= 0:
+        precise = 0.
+    else:
+        precise = corr / y_pred
+
+    if y_true <= 0:
+        recall = 0.
+    else:
+        recall = corr / y_true
+
+    if precise + recall <= 0:
+        f1 = 0.
+    else:
+        f1 = 2 * precise * recall / (precise + recall)
+
+    return f1, (corr, y_true, recall), (corr, y_pred, precise)
diff --git a/gts_engine/bagualu/dataloaders/ie/data_model.py b/gts_engine/bagualu/dataloaders/ie/data_model.py
new file mode 100644
index 0000000..6369e00
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/data_model.py
@@ -0,0 +1,89 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from typing import List
+
+from torch.utils.data import DataLoader
+import pytorch_lightning as pl
+from transformers import PreTrainedTokenizer
+
+from .item_encoder import ItemEncoder
+from .dataset import BagualuIEDataset
+from ...arguments.ie import TrainingArgumentsIEStd
+
+
+class BagualuIEDataModel(pl.LightningDataModule):
+    """ uniEXDataModel
+
+    Args:
+        tokenizer (PreTrainedTokenizer): tokenizer
+        args (GTSEngineArgs): arguments
+        train_data (List[dict]): train data
+        dev_data (List[dict]): dev data
+        test_data (List[dict]): test data
+    """
+    def __init__(self,
+                 tokenizer: PreTrainedTokenizer,
+                 args: TrainingArgumentsIEStd,
+                 train_data: List[dict] = None,
+                 dev_data: List[dict] = None,
+                 test_data: List[dict] = None):
+        super().__init__()
+
+        self.batch_size = args.batch_size
+
+        self.train_data = BagualuIEDataset(train_data if train_data else [], tokenizer, args.max_length)
+        self.valid_data = BagualuIEDataset(dev_data if dev_data else [], tokenizer, args.max_length)
+        self.test_data = BagualuIEDataset(test_data if test_data else [], tokenizer, args.max_length)
+
+        if not args.distill_self:
+            self.collate_fn = ItemEncoder.collate
+        else:
+            self.collate_fn = ItemEncoder.collate_expand
+
+    def train_dataloader(self) -> DataLoader:
+        """ train_dataloader
+
+        Returns:
+            DataLoader: data loader
+        """
+        return DataLoader(self.train_data,
+                          shuffle=True,
+                          collate_fn=self.collate_fn,
+                          batch_size=self.batch_size,
+                          pin_memory=False)
+
+    def val_dataloader(self) -> DataLoader:
+        """ val_dataloader
+
+        Returns:
+            DataLoader: data loader
+        """
+        return DataLoader(self.valid_data,
+                          shuffle=False,
+                          collate_fn=self.collate_fn,
+                          batch_size=self.batch_size,
+                          pin_memory=False)
+
+    def test_dataloader(self) -> DataLoader:
+        """ test_dataloader
+
+        Returns:
+            DataLoader: data loader
+        """
+        return DataLoader(self.test_data,
+                          shuffle=False,
+                          collate_fn=self.collate_fn,
+                          batch_size=self.batch_size,
+                          pin_memory=False)
diff --git a/gts_engine/bagualu/dataloaders/ie/dataset.py b/gts_engine/bagualu/dataloaders/ie/dataset.py
new file mode 100644
index 0000000..2a2977b
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/dataset.py
@@ -0,0 +1,44 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from typing import List
+
+from torch.utils.data import Dataset
+from transformers import PreTrainedTokenizer
+
+from .item_encoder import ItemEncoder
+
+
+class BagualuIEDataset(Dataset):
+    """ BagualuIEDataset
+
+    Args:
+        data (List[dict]): data
+        tokenizer (PreTrainedTokenizer): tokenizer
+        args (TrainingArgumentsIEStd): arguments
+    """
+    def __init__(self,
+                 data: List[dict],
+                 tokenizer: PreTrainedTokenizer,
+                 max_length: int) -> None:
+        super().__init__()
+
+        self.data = data
+        self.encoder = ItemEncoder(tokenizer, max_length)
+
+    def __len__(self):
+        return len(self.data)
+
+    def __getitem__(self, index: int):
+        return self.encoder.encode_item(self.data[index])
diff --git a/gts_engine/bagualu/dataloaders/ie/dataset_utils.py b/gts_engine/bagualu/dataloaders/ie/dataset_utils.py
new file mode 100644
index 0000000..1843869
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/dataset_utils.py
@@ -0,0 +1,325 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import collections
+from typing import List, Dict, Tuple
+
+
+def get_choice(spo_choice: list) -> tuple:
+    """ 把关系schema中的关系、实体获取出来
+
+    Args:
+        spo_choice (list): 关系schema
+
+    Returns:
+        tuple:
+            choice_ent (list)
+            choice_rel (list)
+            choice_head (list)
+            choice_tail (list)
+            entity2rel (dict)
+    """
+    choice_head = []
+    choice_tail = []
+    choice_ent = []
+    choice_rel = []
+    entity2rel = collections.defaultdict(list) # "subject|object" -> [relation]
+
+    for head, rel, tail in spo_choice:
+
+        if head not in choice_head:
+            choice_head.append(head)
+        if tail not in choice_tail:
+            choice_tail.append(tail)
+
+        if head not in choice_ent:
+            choice_ent.append(head)
+        if tail not in choice_ent:
+            choice_ent.append(tail)
+
+        if rel not in choice_rel:
+            choice_rel.append(rel)
+
+        entity2rel[head, tail].append(rel)
+
+    return choice_ent, choice_rel, choice_head, choice_tail, entity2rel
+
+
+def check_data(data: List) -> None:
+    """ 检查数据是否格式合法
+
+    Args:
+        data (List): 数据
+    """
+
+    def check_entity(entity: Dict,
+                     choice_entity: List[str],
+                     text: str) -> None:
+        """ 检查实体格式
+
+        Args:
+            entity (Dict): 实体
+            choice_entity (List[str]): 实体类型
+            text (str): 文本
+        """
+        assert "entity_text" in entity
+        assert "entity_type" in entity
+        assert "entity_index" in entity
+        assert text[entity["entity_index"][0]: entity["entity_index"][1]] == entity["entity_text"]
+        assert entity["entity_type"] in choice_entity
+
+    def check_spo(spo: Dict,
+                  choice_entity: List[str],
+                  entity2relation: Dict[Tuple[str, str], str],
+                  text: str) -> None:
+        """ 检查SPO格式
+
+        Args:
+            spo (Dict): SPO
+            choice_entity (List[str]): 实体类型
+            entity2relation (Dict[Tuple[str, str], str]): 实体-关系映射
+            text (str): 文本
+        """
+        assert "predicate" in spo
+        assert "subject" in spo
+        assert "object" in spo
+        check_entity(spo["subject"], choice_entity, text)
+        check_entity(spo["object"], choice_entity, text)
+        assert spo["predicate"] in entity2relation[spo["subject"]["entity_type"],
+                                                   spo["object"]["entity_type"]]
+
+    for item in data:
+
+        assert "task" in item and isinstance(item["task"], str)
+        assert "text" in item and isinstance(item["text"], str)
+        assert "choice" in item
+
+        if item["task"] == "实体识别":
+            assert "entity_list" in item
+            choice_entity = item["choice"]
+            for entity in item["entity_list"]:
+                check_entity(entity, choice_entity, item["text"])
+
+        elif item["task"] == "关系抽取":
+            assert "spo_list" in item
+            choice_entity = get_choice(item["choice"])[0]
+            entity2relation = get_choice(item["choice"])[-1]
+            for spo in item["spo_list"]:
+                check_spo(spo, choice_entity, entity2relation, item["text"])
+
+        else:
+            raise ValueError(f"task type `{item['task']}` is not supported yet.")
+
+
+def data_segment(ori_data: List[dict],
+                 cut_text_len: int = 400,
+                 cut_text_stride: int = 200) -> List[dict]:
+    """ 数据按照文本切割
+
+    Args:
+        ori_data (List[dict]): 原始数据
+        cut_text_len (int, optional): 切割文本窗口大小. Defaults to 400.
+        cut_text_stride (int, optional): 切割文本步长. Defaults to 200.
+
+    Returns:
+        List[dict]: 切割后的文本
+    """
+    data = []
+
+    for item_id, item in enumerate(ori_data):
+
+        task = item["task"]
+        text = item["text"]
+        entity_list = item.get("entity_list", [])
+        spo_list = item.get("spo_list", [])
+        choice = item["choice"]
+
+        win_size = min(len(text), cut_text_len) # 实际窗口大小
+
+        for win_end in range(win_size, len(text) + 1, cut_text_stride):
+
+            win_start = win_end - win_size
+            text_win = text[win_start: win_end] # 窗口文本
+
+            # 处理窗口包含的实体
+            entity_list_win = []
+            for entity in entity_list:
+
+                entity_start, entity_end = entity["entity_index"]
+                entity_text, entity_type = entity["entity_text"], entity["entity_type"]
+
+                if entity_start >= win_start and entity_end <= win_end: # 包含的实体
+                    entity_index = [entity_start - win_start, entity_end - win_start] # 窗口中的相对索引
+                    assert text_win[entity_index[0]: entity_index[1]] == entity["entity_text"]
+                    entity_list_win.append({
+                        "entity_text": entity_text,
+                        "entity_type": entity_type,
+                        "entity_index": entity_index
+                    })
+
+            # 处理窗口包含的关系
+            spo_list_win = []
+            for spo in spo_list:
+
+                predicate = spo["predicate"]
+                subj_start, subj_end = spo["subject"]["entity_index"]
+                subj_text, subj_type = spo["subject"]["entity_text"], spo["subject"]["entity_type"]
+                obj_start, obj_end = spo["object"]["entity_index"]
+                obj_text, obj_type = spo["object"]["entity_text"], spo["object"]["entity_type"]
+
+                if subj_start >= win_start and subj_end <= win_end and \
+                        obj_start >= win_start and obj_end <= win_end: # 包含的关系
+                    subj_index = [subj_start - win_start, subj_end - win_start]
+                    obj_index = [obj_start - win_start, obj_end - win_start]
+                    assert text_win[subj_index[0]: subj_index[1]] == subj_text
+                    assert text_win[obj_index[0]: obj_index[1]] == obj_text
+                    spo_list_win.append({
+                        "predicate": predicate,
+                        "subject": {
+                            "entity_text": subj_text,
+                            "entity_type": subj_type,
+                            "entity_index": subj_index,
+                        },
+                        "object": {
+                            "entity_text": obj_text,
+                            "entity_type": obj_type,
+                            "entity_index": obj_index,
+                        }
+                    })
+
+            data.append({
+                "id": item_id,
+                "task": task,
+                "text": text_win,
+                "entity_list": entity_list_win,
+                "spo_list": spo_list_win,
+                "choice": choice,
+                "bias": win_start,
+                "full_text": text,
+                "full_entity_list": entity_list,
+                "full_spo_list": spo_list,
+            })
+
+    return data
+
+
+def data_segment_restore(ori_data: List[dict]) -> List[dict]:
+    """ 切割后的数据进行预测后，对预测结果进行合并
+
+    Args:
+        ori_data (List[dict]): 原始数据
+
+    Returns:
+        List[dict]: 合并后数据
+    """
+    ori_data.sort(key=lambda x: (x["id"], x["bias"]))  # 保证相同ID的样本连续，且合并后样本顺序与原始数据相同
+
+    data = []
+    for item in ori_data:
+        task = item["task"]
+        text = item["full_text"]
+        entity_list_win = item["entity_list"]
+        spo_list_win = item["spo_list"]
+        choice = item["choice"]
+        bias = item["bias"]
+
+        if bias == 0:
+            new_item = {
+                "task": task,
+                "text": text,
+                "entity_list": [],
+                "spo_list": [],
+                "choice": choice,
+            }
+            data.append(new_item)
+        else:
+            new_item = data[-1]
+            assert new_item["text"] == text
+
+        # 合并实体列表
+        for entity in entity_list_win:
+            entity_text, entity_type, score = entity["entity_text"], \
+                                              entity["entity_type"], entity["score"]
+            entity_start, entity_end = entity["entity_index"][0] + bias, \
+                                       entity["entity_index"][1] + bias
+            assert entity_text == text[entity_start: entity_end]
+
+            # 判断该实体是否已经被前一个窗口包含，若包含则更新score，不包含则加入
+            is_included = False
+            entity_key = (entity_text, entity_type, entity_start, entity_end) # 判断实体被包含的key
+            for pre_entity in new_item["entity_list"]:
+                pre_entity_key =  (pre_entity["entity_text"], pre_entity["entity_type"],
+                                   pre_entity["entity_index"][0], pre_entity["entity_index"][1])
+                if entity_key == pre_entity_key:
+                    pre_entity["score"] = max(pre_entity["score"], score) # score取最高
+                    is_included = True
+                    break
+
+            if not is_included:
+                new_item["entity_list"].append({
+                    "entity_text": entity_text,
+                    "entity_type": entity_type,
+                    "entity_index": [entity_start, entity_end],
+                    "score": score,
+                })
+
+        # 合并关系列表
+        for spo in spo_list_win:
+            predicate, subj, obj, score = spo["predicate"], spo["subject"], \
+                                          spo["object"], spo["score"]
+            subj_text, subj_type, subj_score = subj["entity_text"], \
+                                               subj["entity_type"], subj["score"]
+            subj_start, subj_end = subj["entity_index"][0] + bias, \
+                                   subj["entity_index"][1] + bias
+            obj_text, obj_type, obj_score = obj["entity_text"], obj["entity_type"], obj["score"]
+            obj_start, obj_end = obj["entity_index"][0] + bias, obj["entity_index"][1] + bias
+            assert subj_text == text[subj_start: subj_end]
+            assert obj_text == text[obj_start: obj_end]
+
+            # 判断该关系是否已经被前一个窗口包含，若包含则更新score，不包含则加入
+            is_included = False
+            spo_key = (predicate, subj_text, subj_type, subj_start, subj_end,
+                       obj_text, obj_type, obj_start, obj_end)
+            for pre_spo in new_item["spo_list"]:
+                pre_subj, pre_obj = pre_spo["subject"], pre_spo["object"]
+                pre_spo_key = (pre_spo["predicate"], pre_subj["entity_text"],
+                               pre_subj["entity_type"], pre_subj["entity_index"][0],
+                               pre_subj["entity_index"][1], pre_obj["entity_text"],
+                               pre_obj["entity_type"], pre_obj["entity_index"][0],
+                               pre_obj["entity_index"][1])
+                if spo_key == pre_spo_key:
+                    pre_spo["score"] = max(pre_spo["score"], score) # score取最高
+                    is_included = True
+                    break
+
+            if not is_included:
+                new_item["spo_list"].append({
+                    "predicate": predicate,
+                    "subject": {
+                        "entity_text": subj_text,
+                        "entity_type": subj_type,
+                        "entity_index": [subj_start, subj_end],
+                        "score": subj_score
+                    },
+                    "object": {
+                        "entity_text": obj_text,
+                        "entity_type": obj_type,
+                        "entity_index": [obj_start, obj_end],
+                        "score": obj_score
+                    },
+                    "score": score
+                })
+
+    return data
diff --git a/gts_engine/bagualu/dataloaders/ie/item_decoder.py b/gts_engine/bagualu/dataloaders/ie/item_decoder.py
new file mode 100644
index 0000000..e2bb307
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/item_decoder.py
@@ -0,0 +1,321 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# from collections import defaultdict
+from typing import List, Tuple, Dict
+
+import numpy as np
+from transformers import PreTrainedTokenizer
+
+from .item_encoder import entity_based_tokenize, get_entity_indices
+from .dataset_utils import get_choice
+from ...arguments.ie import TrainingArgumentsIEStd
+
+
+class ItemDecoder(object):
+    """ Decoder
+
+    Args:
+        tokenizer (PreTrainedTokenizer): tokenizer
+        args (TrainingArgumentsIEStd): arguments
+    """
+    def __init__(self,
+                 tokenizer: PreTrainedTokenizer,
+                 args: TrainingArgumentsIEStd) -> None:
+        self.tokenizer = tokenizer
+        self.max_length = args.max_length
+        self.threshold_entity = args.threshold_ent
+        self.threshold_rel = args.threshold_rel
+        self.entity_multi_label = args.entity_multi_label
+        self.relation_multi_label = args.relation_multi_label
+
+    def extract_entity_index(self,
+                             entity_logits: np.ndarray,
+                             ) -> List[Tuple[int, int]]:
+        """ extract entity index
+
+        Args:
+            entity_logits (np.ndarray): entity_logits
+
+        Returns:
+            List[Tuple[int, int]]: result
+        """
+
+        l, _, d = entity_logits.shape
+        result = []
+        for i in range(l):
+            for j in range(i, l):
+                if self.entity_multi_label:
+                    for k in range(d):
+                        entity_score = float(entity_logits[i, j, k])
+                        if entity_score > self.threshold_entity:
+                            result.append((i, j, k, entity_score))
+
+                else:
+                    k = np.argmax(entity_logits[i, j])
+                    entity_score = float(entity_logits[i, j, k])
+                    if entity_score > self.threshold_entity:
+                        result.append((i, j, k, entity_score))
+
+        return result
+
+    @staticmethod
+    def extract_entity(text: str,
+                       entity_idx: List[int],
+                       entity_type: str,
+                       entity_score: float,
+                       text_start_id: int,
+                       offset_mapping: List[List[int]]) -> dict:
+        """ extract entity
+
+        Args:
+            text (str): text
+            entity_idx (List[int]): entity indices
+            entity_type (str): entity type
+            entity_score (float): entity score
+            text_start_id (int): text_start_id
+            offset_mapping (List[List[int]]): offset mapping
+
+        Returns:
+            dict: entity
+        """
+        entity_start, entity_end = entity_idx[0] - text_start_id, entity_idx[1] - text_start_id
+
+        start_split = offset_mapping[entity_start] if 0 <= entity_start < len(offset_mapping) else []
+        end_split = offset_mapping[entity_end] if 0 <= entity_end < len(offset_mapping) else []
+
+        if not start_split or not end_split:
+            return None
+
+        start_idx, end_idx = start_split[0], end_split[-1]
+        entity_text = text[start_idx: end_idx]
+
+        if not entity_text:
+            return None
+
+        entity = {
+            "entity_text": entity_text,
+            "entity_type": entity_type,
+            "score": entity_score,
+            "entity_index": [start_idx, end_idx]
+        }
+
+        return entity
+
+    def decode_ner(self,
+                   text: str,
+                   choice: List[str],
+                   sample_span_logits: np.ndarray,
+                   offset_mapping: List[List[int]]
+                  ) -> List[dict]:
+        """ NER decode
+
+        Args:
+            text (str): text
+            choice (List[str]): choice
+            sample_span_logits (np.ndarray): sample span_logits
+            offset_mapping (List[List[int]]): offset mapping
+
+
+        Returns:
+            List[dict]: decoded entity list
+        """
+        entity_list = []
+
+        entity_idx_list = self.extract_entity_index(sample_span_logits)
+
+        for entity_start, entity_end, entity_type_idx, entity_score in entity_idx_list:
+
+            entity = self.extract_entity(text,
+                                         [entity_start, entity_end],
+                                         choice[entity_type_idx],
+                                         entity_score,
+                                         text_start_id=1,
+                                         offset_mapping=offset_mapping)
+
+            if entity is None:
+                continue
+
+            if entity not in entity_list:
+                entity_list.append(entity)
+
+        return entity_list
+
+    def decode_spo(self,
+                   text: str,
+                   choice: List[List[str]],
+                   sample_span_logits: np.ndarray,
+                   offset_mapping: List[List[int]]) -> tuple:
+        """ SPO decode
+
+        Args:
+            text (str): text
+            choice (List[List[str]]): choice
+            sample_span_logits (np.ndarray): sample span_logits
+            offset_mapping (List[List[int]): offset mapping
+
+        Returns:
+            List[dict]: decoded spo list
+            List[dict]: decoded entity list
+        """
+        spo_list = []
+        entity_list = []
+
+        choice_ent, choice_rel, choice_head, choice_tail, entity2rel = get_choice(choice)
+
+        entity_logits = sample_span_logits[:, :, : len(choice_ent)] # (seq_len, seq_len, num_entity)
+        relation_logits = sample_span_logits[:, :, len(choice_ent): ] # (seq_len, seq_len, num_relation)
+
+        entity_idx_list = self.extract_entity_index(entity_logits)
+
+        head_list = []
+        tail_list = []
+        for entity_start, entity_end, entity_type_idx, entity_score in entity_idx_list:
+
+            entity_type = choice_ent[entity_type_idx]
+
+            entity = self.extract_entity(text,
+                                         [entity_start, entity_end],
+                                         entity_type,
+                                         entity_score,
+                                         text_start_id=1,
+                                         offset_mapping=offset_mapping)
+
+            if entity is None:
+                continue
+
+            if entity_type in choice_head:
+                head_list.append((entity_start, entity_end, entity_type, entity))
+            if entity_type in choice_tail:
+                tail_list.append((entity_start, entity_end, entity_type, entity))
+
+        for head_start, head_end, subject_type, subject_dict in head_list:
+            for tail_start, tail_end, object_type, object_dict in tail_list:
+
+                if subject_dict == object_dict:
+                    continue
+
+                if (subject_type, object_type) not in entity2rel.keys():
+                    continue
+
+                relation_candidates = entity2rel[subject_type, object_type]
+                rel_idx = [choice_rel.index(r) for r in relation_candidates]
+
+                so_rel_logits = relation_logits[:, :, rel_idx]
+
+                if self.relation_multi_label:
+                    for idx, predicate in enumerate(relation_candidates):
+                        rel_score = so_rel_logits[head_start, tail_start, idx] + \
+                                    so_rel_logits[head_end, tail_end, idx]
+                        predicate_score = float(rel_score / 2)
+
+                        if predicate_score <= self.threshold_rel:
+                            continue
+
+                        if subject_dict not in entity_list:
+                            entity_list.append(subject_dict)
+                        if object_dict not in entity_list:
+                            entity_list.append(object_dict)
+
+                        spo = {
+                            "predicate": predicate,
+                            "score": predicate_score,
+                            "subject": subject_dict,
+                            "object": object_dict,
+                        }
+
+                        if spo not in spo_list:
+                            spo_list.append(spo)
+
+                else:
+
+                    hh_idx = np.argmax(so_rel_logits[head_start, head_end])
+                    tt_idx = np.argmax(so_rel_logits[tail_start, tail_end])
+                    hh_score = so_rel_logits[head_start, tail_start, hh_idx] + so_rel_logits[head_end, tail_end, hh_idx]
+                    tt_score = so_rel_logits[head_start, tail_start, tt_idx] + so_rel_logits[head_end, tail_end, tt_idx]
+
+                    predicate = relation_candidates[hh_idx] if hh_score > tt_score else relation_candidates[tt_idx]
+
+                    predicate_score = float(max(hh_score, tt_score) / 2)
+
+                    if predicate_score <= self.threshold_rel:
+                        continue
+
+                    if subject_dict not in entity_list:
+                        entity_list.append(subject_dict)
+                    if object_dict not in entity_list:
+                        entity_list.append(object_dict)
+
+                    spo = {
+                        "predicate": predicate,
+                        "score": predicate_score,
+                        "subject": subject_dict,
+                        "object": object_dict,
+                    }
+
+                    if spo not in spo_list:
+                        spo_list.append(spo)
+
+        return spo_list, entity_list
+
+    def decode(self,
+               item: Dict,
+               span_logits: np.ndarray,
+               label_mask:  np.ndarray,
+               ):
+        """ decode
+
+        Args:
+            task (str): task name
+            choice (list): choice
+            text (str): text
+            span_logits (np.ndarray): sample span_logits
+            label_mask (np.ndarray): label_mask
+
+        Raises:
+            NotImplementedError: raised if task name is not supported
+
+        Returns:
+            List[dict]: decoded entity list
+            List[dict]: decoded spo list
+        """
+        task, choice, text = item["task"], item["choice"], item["text"]
+        entity_indices = get_entity_indices(item.get("entity_list", []), item.get("spo_list", []))
+        _, offset_mapping = entity_based_tokenize(text, self.tokenizer, entity_indices,
+                                                  return_offsets_mapping=True)
+
+        assert span_logits.shape == label_mask.shape
+
+        span_logits = span_logits + (label_mask - 1) * 100000
+
+        spo_list = []
+        entity_list = []
+
+        if task in {"实体识别", "抽取任务"}:
+            entity_list = self.decode_ner(text,
+                                          choice,
+                                          span_logits,
+                                          offset_mapping)
+
+        elif task in {"关系抽取"}:
+            spo_list, entity_list = self.decode_spo(text,
+                                                    choice,
+                                                    span_logits,
+                                                    offset_mapping)
+
+        else:
+            raise NotImplementedError
+
+        return entity_list, spo_list
diff --git a/gts_engine/bagualu/dataloaders/ie/item_encoder.py b/gts_engine/bagualu/dataloaders/ie/item_encoder.py
new file mode 100644
index 0000000..20e23f0
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/ie/item_encoder.py
@@ -0,0 +1,534 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# pylint: disable=no-member
+
+from typing import List, Tuple, Dict, Union
+
+import numpy as np
+import torch
+import torch.nn as nn
+from transformers import PreTrainedTokenizer
+
+from .dataset_utils import get_choice
+
+
+def get_entity_indices(entity_list: List[dict], spo_list: List[dict]) -> List[List[int]]:
+    """ 获取样本中包含的实体位置信息
+
+    Args:
+        entity_list (List[dict]): 实体列表
+        spo_list (List[dict]): 三元组列表
+
+    Returns:
+        List[List[int]]: 实体位置信息
+    """
+    entity_indices = []
+
+    # 实体中的实体位置
+    for entity in entity_list:
+        entity_index = entity["entity_index"]
+        entity_indices.append(entity_index)
+
+    # 三元组中的实体位置
+    for spo in spo_list:
+        sub_idx = spo["subject"]["entity_index"]
+        obj_idx = spo["object"]["entity_index"]
+        entity_indices.append(sub_idx)
+        entity_indices.append(obj_idx)
+
+    return entity_indices
+
+
+def entity_based_tokenize(text: str,
+                          tokenizer: PreTrainedTokenizer,
+                          enitity_indices: List[Tuple[int, int]],
+                          max_len: int = -1,
+                          return_offsets_mapping: bool = False) \
+    -> Union[List[int], Tuple[List[int], List[Tuple[int, int]]]]:
+    """ 基于实体位置信息的编码，确保实体为连续1到多个token的合并，同时利用预训练模型词根信息
+
+    Args:
+        text (str): 文本
+        tokenizer (PreTrainedTokenizer): tokenizer
+        enitity_indices (List[Tuple[int, int]]): 实体位置信息
+        max_len (int, optional): 长度限制. Defaults to -1.
+        return_offsets_mapping (bool, optional): 是否返回offsets_mapping. Defaults to False.
+
+    Returns:
+        Union[List[int], Tuple[List[int], List[Tuple[int, int]]]]: 编码id
+    """
+    # 根据实体位置遍历出需要对文本进行切割的点
+    split_points = sorted(list({i for idx in enitity_indices for i in idx} | {0, len(text)}))
+    # 对文本进行切割
+    text_parts = []
+    for i in range(0, len(split_points) - 1):
+        text_parts.append(text[split_points[i]: split_points[i + 1]])
+
+    # 对切割后的文本进行编码
+    bias = 0
+    text_ids = []
+    offset_mapping = []
+    for part in text_parts:
+
+        part_encoded = tokenizer(part, add_special_tokens=False, return_offsets_mapping=True)
+        part_ids, part_mapping = part_encoded["input_ids"], part_encoded["offset_mapping"]
+
+        text_ids.extend(part_ids)
+        for start, end in part_mapping:
+            offset_mapping.append((start + bias, end + bias))
+
+        bias += len(part)
+
+    if max_len > 0:
+        text_ids = text_ids[: max_len]
+
+    # 是否返回offsets_mapping
+    if return_offsets_mapping:
+        return text_ids, offset_mapping
+    return text_ids
+
+
+class ItemEncoder(object):
+    """ Item Encoder
+
+    Args:
+        tokenizer (PreTrainedTokenizer): tokenizer
+        max_length (int): max length
+    """
+    def __init__(self, tokenizer: PreTrainedTokenizer, max_length: int) -> None:
+        self.tokenizer = tokenizer
+        self.max_length = max_length
+
+    def search_index(self,
+                     entity_idx: List[int],
+                     offset_mapping: List[Tuple[int, int]],
+                     bias: int = 0) -> Tuple[int, int]:
+        """ 查找实体在tokens中的索引
+
+        Args:
+            entity_idx (List[int]): entity index
+            offset_mapping (List[Tuple[int, int]]): text
+            bias (int): bias
+
+        Returns:
+            Tuple[int]: (start_idx, end_idx)
+        """
+        entity_start, entity_end = entity_idx
+        start_idx, end_idx = -1, -1
+
+        for token_idx, (start, end) in enumerate(offset_mapping):
+            if start == entity_start:
+                start_idx = token_idx
+            if end == entity_end:
+                end_idx = token_idx
+        assert start_idx >= 0 and end_idx >= 0
+
+        return start_idx + bias, end_idx + bias
+
+    @staticmethod
+    def get_position_ids(text_len: int,
+                        ent_ranges: List,
+                        rel_ranges: List) -> np.ndarray:
+        """ 获取position_ids
+
+        Args:
+            text_len (int): input length
+            ent_ranges (List[List[int, int]]): each entity ranges idx
+            rel_ranges (List[List[int, int]]): each relation ranges idx.
+
+        Returns:
+            np.ndarray: position_ids
+        """
+        # 一切从0开始算position，@liuhan
+        text_pos_ids = list(range(text_len))
+
+        ent_pos_ids, rel_pos_ids = [], []
+        for s, e in ent_ranges:
+            ent_pos_ids.extend(list(range(e - s)))
+        for s, e in rel_ranges:
+            rel_pos_ids.extend(list(range(e - s)))
+        position_ids = text_pos_ids + ent_pos_ids + rel_pos_ids
+
+        return position_ids
+
+    @staticmethod
+    def get_att_mask(input_len: int,
+                     ent_ranges: List,
+                     rel_ranges: List= None,
+                     choice_ent: List[str] = None,
+                     choice_rel: List[str] = None,
+                     entity2rel: dict = None,
+                     full_attent: bool = False) -> np.ndarray:
+        """ 获取att_mask，不同choice之间的attention_mask置零
+
+        Args:
+            input_len (int): input length
+            ent_ranges (List[List[int, int]]): each entity ranges idx
+            rel_ranges (List[List[int, int]]): each relation ranges idx. Defaults to None.
+            choice_ent (List[str], optional): choice entity. Defaults to None.
+            choice_rel (List[str], optional): choice relation. Defaults to None.
+            entity2rel (dict, optional): entity to relations. Defaults to None.
+            full_attent (bool, optional): is full attention or not. Defaults to None.
+        Returns:
+            np.ndarray: attention mask
+        """
+
+        # attention_mask.shape = (input_len, input_len)
+        attention_mask = np.ones((input_len, input_len))
+        if full_attent and not rel_ranges: # full-attention且没有关系情况下，返回全1
+            return attention_mask
+
+        # input_ids: [CLS] text [SEP] [unused1] ent1 [unused2] rel1 [unused3] event1
+        text_len = ent_ranges[0][0] # text长度
+        # 将text-实体之间的attention置零，text看不到实体,不受传入的entity个数、顺序影响 @liuhan
+        attention_mask[:text_len, text_len:] = 0
+
+        # 将实体-实体、实体关系之间的attention_mask置零
+        attention_mask[text_len:, text_len: ] = 0
+
+        # 将每个实体与自己的attention_mask置一
+        for s, e in ent_ranges:
+            attention_mask[s: e, s: e] = 1
+
+        # 没有关系的话，直接返回
+        if not rel_ranges:
+            return attention_mask
+
+        # 处理有关系情况
+
+        # 关系自身attention_mask置1
+        for s, e in rel_ranges:
+            attention_mask[s: e, s: e] = 1
+
+        # 将有关联的实体-关系置一
+        for head_tail, relations in entity2rel.items():
+            for entity_type in head_tail:
+                ent_idx = choice_ent.index(entity_type)
+                ent_s, _ = ent_ranges[ent_idx] # ent_s, ent_e
+                for relation_type in relations:
+                    rel_idx = choice_rel.index(relation_type)
+                    rel_s, rel_e = rel_ranges[rel_idx]
+                    attention_mask[rel_s: rel_e, ent_s] = 1 # 关系只看实体第一个的[unused1]
+
+        if full_attent: # full-attention且有关系情况下，让文本能看见关系
+            for s, e in rel_ranges:
+                attention_mask[: text_len, s: e] = 1
+
+        return attention_mask
+
+    def encode(self,
+               text: str,
+               task_name: str,
+               choice: List[str],
+               entity_list: List[dict],
+               spo_list: List[dict],
+               full_attent: bool = False,
+               with_label: bool = True) -> Dict[str, torch.Tensor]:
+        """ encode
+
+        Args:
+            text (str): text
+            task_name (str): task name
+            choice (List[str]): choice
+            entity_list (List[dict]): entity list
+            spo_list (List[dict]): spo list
+            full_attent (bool): full attention
+            with_label (bool): encoded with label. Defaults to True.
+
+        Returns:
+            Dict[str, torch.Tensor]: encoded
+        """
+        choice_ent, choice_rel, entity2rel = choice, [], {}
+        if isinstance(choice, list):
+            if isinstance(choice[0], list): # 关系抽取 & 实体识别
+                choice_ent, choice_rel, _, _, entity2rel = get_choice(choice)
+        elif isinstance(choice, dict):
+            # 事件类型
+            raise ValueError('event extract not supported now!')
+        else:
+            raise NotImplementedError
+
+        input_ids = []
+        text_ids = [] # text部分id
+        ent_ids = [] # entity部分id
+        rel_ids = [] # relation部分id
+        entity_labels_idx = []
+        relation_labels_idx = []
+
+        sep_ids = self.tokenizer.encode("[SEP]", add_special_tokens=False) # [SEP]的编码
+        cls_ids = self.tokenizer.encode("[CLS]", add_special_tokens=False) # [CLS]的编码
+        entity_op_ids = self.tokenizer.encode("[unused1]", add_special_tokens=False) # [unused1]的编码
+        relation_op_ids = self.tokenizer.encode("[unused2]", add_special_tokens=False) # [unused2]的编码
+
+        # 任务名称的编码
+        task_ids = self.tokenizer.encode(task_name, add_special_tokens=False)
+
+        # 实体标签的编码
+        for c in choice_ent:
+            c_ids = self.tokenizer.encode(c, add_special_tokens=False)[: self.max_length]
+            ent_ids += entity_op_ids + c_ids
+
+        # 关系标签的编码
+        for c in choice_rel:
+            c_ids = self.tokenizer.encode(c, add_special_tokens=False)[: self.max_length]
+            rel_ids += relation_op_ids + c_ids
+
+        # text的编码
+        entity_indices = get_entity_indices(entity_list, spo_list)
+        text_max_len = self.max_length - len(task_ids) - 3
+        text_ids, offset_mapping = entity_based_tokenize(text, self.tokenizer, entity_indices,
+                                                         max_len=text_max_len,
+                                                         return_offsets_mapping=True)
+        text_ids = cls_ids + text_ids + sep_ids
+
+        input_ids = text_ids + task_ids + sep_ids + ent_ids + rel_ids
+
+        token_type_ids = [0] * len(text_ids) + [0] * (len(task_ids) + 1) + \
+            [1] * len(ent_ids) + [1] * len(rel_ids)
+
+        entity_labels_idx = [i for i, id_ in enumerate(input_ids) if id_ == entity_op_ids[0]]
+        relation_labels_idx = [i for i, id_ in enumerate(input_ids) if id_ == relation_op_ids[0]]
+
+        ent_ranges = [] # 每个实体的起始范围
+        for i in range(len(entity_labels_idx) - 1):
+            ent_ranges.append([entity_labels_idx[i], entity_labels_idx[i + 1]])
+        if not relation_labels_idx:
+            ent_ranges.append([entity_labels_idx[-1], len(input_ids)])
+        else:
+            ent_ranges.append([entity_labels_idx[-1], relation_labels_idx[0]])
+        assert len(ent_ranges) == len(choice_ent)
+
+        rel_ranges = [] # 每个关系的起始范围
+        for i in range(len(relation_labels_idx) - 1):
+            rel_ranges.append([relation_labels_idx[i], relation_labels_idx[i + 1]])
+        if relation_labels_idx:
+            rel_ranges.append([relation_labels_idx[-1], len(input_ids)])
+        assert len(rel_ranges) == len(choice_rel)
+
+        # 所有unused的位置
+        label_token_idx = entity_labels_idx + relation_labels_idx
+        task_num_labels = len(label_token_idx)
+        input_len = len(input_ids)
+        text_len = len(text_ids)
+
+        # 计算mask
+        attention_mask = self.get_att_mask(input_len,
+                                           ent_ranges,
+                                           rel_ranges,
+                                           choice_ent,
+                                           choice_rel,
+                                           entity2rel,
+                                           full_attent)
+        # 计算label-mask
+        label_mask = np.ones((text_len, text_len, task_num_labels))
+        for i in range(text_len):
+            for j in range(text_len):
+                if j < i:
+                    for l in range(len(entity_labels_idx)):
+                        # entity部分的下三角可mask
+                        label_mask[i, j, l] = 0
+
+        # 计算position_ids
+        position_ids = self.get_position_ids(len(text_ids) + len(task_ids) + 1,
+                                             ent_ranges,
+                                             rel_ranges)
+
+        assert len(input_ids) == len(position_ids) == len(token_type_ids)
+
+        if not with_label:
+            return {
+                "input_ids": torch.tensor(input_ids).long(),
+                "attention_mask": torch.tensor(attention_mask).float(),
+                "position_ids": torch.tensor(position_ids).long(),
+                "token_type_ids": torch.tensor(token_type_ids).long(),
+                "label_token_idx": torch.tensor(label_token_idx).long(),
+                "label_mask":  torch.tensor(label_mask).float(),
+                "text_len": torch.tensor(text_len).long(),
+                "ent_ranges": ent_ranges,
+                "rel_ranges": rel_ranges,
+            }
+
+        # 输入的span_labels，只保留text部分
+        span_labels = np.zeros((text_len, text_len, task_num_labels))
+
+        # 将实体转成span
+        for entity in entity_list:
+
+            entity_type = entity["entity_type"]
+            entity_index = entity["entity_index"]
+
+            start_idx, end_idx = self.search_index(entity_index, offset_mapping, 1)
+
+            if start_idx < text_len and end_idx < text_len:
+                ent_label = choice_ent.index(entity_type)
+                span_labels[start_idx, end_idx, ent_label] = 1
+
+        # 将三元组转成span
+        for spo in spo_list:
+
+            sub_idx = spo["subject"]["entity_index"]
+            obj_idx = spo["object"]["entity_index"]
+
+            # 获取头实体、尾实体的开始、结束index
+            sub_start_idx, sub_end_idx = self.search_index(sub_idx, offset_mapping, 1)
+            obj_start_idx, obj_end_idx = self.search_index(obj_idx, offset_mapping, 1)
+            # 实体label置1
+            if sub_start_idx < text_len and sub_end_idx < text_len:
+                sub_label = choice_ent.index(spo["subject"]["entity_type"])
+                span_labels[sub_start_idx, sub_end_idx, sub_label] = 1
+
+            if obj_start_idx < text_len and obj_end_idx < text_len:
+                obj_label = choice_ent.index(spo["object"]["entity_type"])
+                span_labels[obj_start_idx, obj_end_idx, obj_label] = 1
+
+            # 有关系的sub/obj实体的start/end在realtion对应的label置1
+            if spo["predicate"] in choice_rel:
+                pre_label = choice_rel.index(spo["predicate"]) + len(choice_ent)
+                if sub_start_idx < text_len and obj_start_idx < text_len:
+                    span_labels[sub_start_idx, obj_start_idx, pre_label] = 1
+                if sub_end_idx < text_len and obj_end_idx < text_len:
+                    span_labels[sub_end_idx, obj_end_idx, pre_label] = 1
+
+        return {
+            "input_ids": torch.tensor(input_ids).long(),
+            "attention_mask": torch.tensor(attention_mask).float(),
+            "position_ids": torch.tensor(position_ids).long(),
+            "token_type_ids": torch.tensor(token_type_ids).long(),
+            "label_token_idx": torch.tensor(label_token_idx).long(),
+            "span_labels": torch.tensor(span_labels).float(),
+            "label_mask":  torch.tensor(label_mask).float(),
+            "text_len": torch.tensor(text_len).long(),
+            "ent_ranges": ent_ranges,
+            "rel_ranges": rel_ranges,
+        }
+
+    def encode_item(self, item: dict, with_label: bool = True) -> Dict[str, torch.Tensor]:  # pylint: disable=unused-argument
+        """ encode
+
+        Args:
+            item (dict): item
+            with_label (bool): encoded with label. Defaults to True.
+
+        Returns:
+            Dict[str, torch.Tensor]: encoded
+        """
+        return self.encode(text=item["text"],
+                           task_name=item["task"],
+                           choice=item["choice"],
+                           entity_list=item.get("entity_list", []),
+                           spo_list=item.get("spo_list", []),
+                           full_attent=item.get('full_attent', False),
+                           with_label=with_label)
+
+    @staticmethod
+    def collate(batch: List[Dict[str, torch.Tensor]]) -> Dict[str, torch.Tensor]:
+        """
+        Aggregate a batch data.
+        batch = [ins1_dict, ins2_dict, ..., insN_dict]
+        batch_data = {"sentence":[ins1_sentence, ins2_sentence...],
+        "input_ids":[ins1_input_ids, ins2_input_ids...], ...}
+        """
+        input_ids = nn.utils.rnn.pad_sequence(
+            sequences=[encoded["input_ids"] for encoded in batch],
+            batch_first=True,
+            padding_value=0)
+
+        label_token_idx = nn.utils.rnn.pad_sequence(
+            sequences=[encoded["label_token_idx"] for encoded in batch],
+            batch_first=True,
+            padding_value=0)
+
+        token_type_ids = nn.utils.rnn.pad_sequence(
+            sequences=[encoded["token_type_ids"] for encoded in batch],
+            batch_first=True,
+            padding_value=0)
+
+        position_ids = nn.utils.rnn.pad_sequence(
+            sequences=[encoded["position_ids"] for encoded in batch],
+            batch_first=True,
+            padding_value=0)
+
+        text_len = torch.tensor([encoded["text_len"] for encoded in batch]).long()
+        max_text_len = text_len.max()
+
+        batch_size, batch_max_length = input_ids.shape
+        _, batch_max_labels = label_token_idx.shape
+
+        attention_mask = torch.zeros((batch_size, batch_max_length, batch_max_length))
+        label_mask = torch.zeros((batch_size,
+                                  batch_max_length,
+                                  batch_max_length,
+                                  batch_max_labels))
+        for i, encoded in enumerate(batch):
+            input_len = encoded["attention_mask"].shape[0]
+            attention_mask[i, :input_len, :input_len] = encoded["attention_mask"]
+            _, cur_text_len, label_len = encoded['label_mask'].shape
+            label_mask[i, :cur_text_len, :cur_text_len, :label_len] = encoded['label_mask']
+        label_mask = label_mask[:, :max_text_len, :max_text_len, :]
+
+        batch_data = {
+            "input_ids": input_ids,
+            "attention_mask": attention_mask,
+            "position_ids": position_ids,
+            "token_type_ids": token_type_ids,
+            "label_token_idx": label_token_idx,
+            "label_mask": label_mask,
+            'text_len': text_len
+        }
+
+        if "span_labels" in batch[0].keys():
+            span_labels = torch.zeros((batch_size,
+                                       batch_max_length,
+                                       batch_max_length,
+                                       batch_max_labels))
+            for i, encoded in enumerate(batch):
+                input_len, _, sample_num_labels = encoded["span_labels"].shape
+                span_labels[i, :input_len, :input_len, :sample_num_labels] = encoded["span_labels"]
+            batch_data["span_labels"] = span_labels[:, :max_text_len, :max_text_len, :]
+
+        return batch_data
+
+    @staticmethod
+    def collate_expand(batch: List[Dict[str, torch.Tensor]]) -> Dict[str, torch.Tensor]:
+        """
+        Aggregate a batch data and expand to full attention
+        batch = [ins1_dict, ins2_dict, ..., insN_dict]
+        batch_data = {"sentence":[ins1_sentence, ins2_sentence...],
+        "input_ids":[ins1_input_ids, ins2_input_ids...], ...}
+        """
+        mask_atten_batch = ItemEncoder.collate(batch)
+        full_atten_batch = ItemEncoder.collate(batch)
+        # 对full_atten_batch进行改造
+        atten_mask = full_atten_batch['attention_mask']
+        b, _, _ = atten_mask.size()
+        for i in range(b):
+            ent_ranges, rel_ranges = batch[i]['ent_ranges'], batch[i]['rel_ranges']
+            text_len = ent_ranges[0][0] # text长度
+
+            if not rel_ranges:
+                assert len(ent_ranges) == 1, 'ent_ranges:%s' % ent_ranges
+                s, e = ent_ranges[0]
+                atten_mask[i, : text_len, s: e] = 1
+            else:
+                assert len(rel_ranges) == 1 and len(ent_ranges) <= 2, \
+                    'ent_ranges:%s, rel_ranges:%s' % (ent_ranges, rel_ranges)
+                s, e = rel_ranges[0]
+                atten_mask[i, : text_len, s: e] = 1
+        full_atten_batch['attention_mask'] = atten_mask
+        collate_batch = {}
+        for key, value in mask_atten_batch.items():
+            collate_batch[key] = torch.cat((value, full_atten_batch[key]), 0)
+        return collate_batch
diff --git a/gts_engine/bagualu/dataloaders/text_classification/__init__.py b/gts_engine/bagualu/dataloaders/text_classification/__init__.py
new file mode 100644
index 0000000..c51747e
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/text_classification/__init__.py
@@ -0,0 +1,1 @@
+from .data_module_std import DataModuleClfStd
\ No newline at end of file
diff --git a/gts_engine/bagualu/dataloaders/text_classification/data_module_std.py b/gts_engine/bagualu/dataloaders/text_classification/data_module_std.py
new file mode 100644
index 0000000..745490e
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/text_classification/data_module_std.py
@@ -0,0 +1,104 @@
+from typing import List
+import math
+import os
+from torch.utils.data import DataLoader
+
+from ...lib.framework.classification_finetune import BaseDataModuleClf, DataReaderClf
+from ...lib.framework.classification_finetune.consts import LabeledSample
+from ...lib.components import EDA
+from ...lib.components.samplers import PairBatchSampler
+
+from ...dataloaders.text_classification.datasets_std import TrainDatasetClfStd, TestDatasetClfStd, InfDatasetClfStd
+from ...arguments.text_classification.arguments_std import TrainingArgumentsClfStd
+
+
+class DataModuleClfStd(BaseDataModuleClf):
+    
+    _args: TrainingArgumentsClfStd
+    
+    def _get_train_dataset(self, sample_list):
+        return TrainDatasetClfStd(
+            sample_list, self._tokenizer, self._prompt, 
+            self._args.training_label_prompt, 
+            self._args.label_guided_rate, 
+            self._args.max_length, 
+            self._args.wwm_mask_rate
+        )
+
+    def _get_test_dataset(self, sample_list):
+        return TestDatasetClfStd(
+            sample_list, self._tokenizer, self._prompt,
+            self._args.inference_label_prompt,
+            self._args.prefix_prompt,
+            self._args.max_length
+        )
+    
+    def _get_inf_dataset(self, sample_list):
+        return InfDatasetClfStd(
+            sample_list, self._tokenizer, self._prompt,
+            self._args.inference_label_prompt,
+            self._args.prefix_prompt,
+            self._args.max_length
+        )
+    
+    def _load_train_sample_list(self):
+        sample_list: List[LabeledSample] = []
+        if self._args.train_data_path is not None:
+            sample_list += list(DataReaderClf.read_labeled_sample(self._args.train_data_path, self._prompt.label2token))
+        else:
+            raise Exception("no training data is passed")
+        if self._args.aug_eda_gate:
+            if self._args.aug_eda_path is None:
+                raise Exception("eda augmentation file path is not passed")
+            eda: EDA[LabeledSample] = EDA(self._tokenizer, logger_name=self._args.logger)
+            sample_list += eda.eda_aug(sample_list, self._args.aug_eda_path)
+        return sample_list
+
+    def _load_raw_train_sample_list(self) -> List[LabeledSample]:
+        sample_list: List[LabeledSample] = []
+        if self._args.train_data_path is not None:
+            sample_list += list(DataReaderClf.read_labeled_sample(self._args.train_data_path, self._prompt.label2token))
+        else:
+            raise Exception("no training data is passed")
+        return sample_list
+
+    def raw_train_dataloader(self, load_ratio: float = 1):
+        sample_list = self._load_raw_train_sample_list()
+        self._logger.info(f"number of training sample: {len(sample_list)}")
+        lg_dataset = self._get_train_dataset(sample_list)
+        return DataLoader(dataset=lg_dataset, batch_size=self._args.train_batch_size, num_workers=6 * self._args.device_num, shuffle=True)
+
+    def reparse_args(self):
+        """根据数据信息更改训练参数"""
+        if len(self._prompt.label_ids) <= 5:
+            self._args.label_guided_rate = 0.3
+        step_per_epoch = self.train_sample_num // self._args.train_batch_size
+        steps = step_per_epoch * self._args.epoch
+        min_steps = min(1400 // self._args.device_num, 40 * step_per_epoch)
+        if steps < min_steps:
+            self._args.epoch = math.ceil(min_steps // step_per_epoch)
+        if self.dev_sample_num == 0:
+            self._args.epoch = int(0.75 * self._args.epoch)
+        self._logger.info(f"reparsing epoch: {self._args.epoch}")
+        self._logger.info(f"batch size per device: {self._args.train_batchsize_per_device}")
+        self._logger.info(f"total batch size: {self._args.train_batch_size}")
+
+        if len(self._prompt.label_ids) < self._args.rdrop_gate:
+            # 类别数<rdrop gate时，使用rdrop
+            self._args.use_rdrop = True
+        if self._args.dev_data_path is not None and os.path.exists(self._args.dev_data_path) and len(self._prompt.label_ids) > self._args.rdrop_gate:
+            # 类别数大于rdrop gate时(knn在小类别任务有负向作用)，且有验证集时，才使用knn
+            self._args.use_knn = True
+            
+    def train_dataloader(self, load_ratio: float = 1):
+        sample_list = self.train_sample_list[:int(len(self.train_sample_list) * load_ratio)]
+        self._logger.info(f"number of training sample: {len(sample_list)}")
+        lg_dataset = self._get_train_dataset(sample_list)
+        # cs-kd: 实现pairbatch,实现同类别采样
+        sampler = lambda lg_dataset : PairBatchSampler(lg_dataset, self._args.train_batch_size)
+        if self._args.use_knn:
+            return DataLoader(dataset=lg_dataset,
+                            batch_sampler=sampler(lg_dataset),  # type: ignore
+                            num_workers=6 * self._args.device_num)
+        else:
+            return DataLoader(dataset=lg_dataset, batch_size=self._args.train_batch_size, num_workers=6 * self._args.device_num, shuffle=True)
\ No newline at end of file
diff --git a/gts_engine/bagualu/dataloaders/text_classification/datasets_std.py b/gts_engine/bagualu/dataloaders/text_classification/datasets_std.py
new file mode 100644
index 0000000..c5c1083
--- /dev/null
+++ b/gts_engine/bagualu/dataloaders/text_classification/datasets_std.py
@@ -0,0 +1,160 @@
+from dataclasses import asdict
+from typing import Sequence
+import numpy as np
+import torch
+
+from ...lib.framework.classification_finetune import BaseDatasetClf
+from ...lib.framework.classification_finetune.consts import LabeledSample, PreEncodedTrainSample, EncodedTrainSample, InfSampleProto, EncodedInfSample
+from ...lib.framework.classification_finetune.mask_tools import wwm_masking
+from ...lib.components.text_tools import segment_text
+
+class TrainDatasetClfStd(BaseDatasetClf):
+    
+    def __init__(self, sample_list, tokenizer, prompt, training_label_prompt: str, label_guided_rate: float, max_length: int, wwm_mask_rate: float):
+        self._training_label_prompt = training_label_prompt
+        self._label_guided_rate = label_guided_rate
+        self._max_length = max_length
+        self._wwm_mask_rate = wwm_mask_rate
+        super().__init__(sample_list, tokenizer, prompt)
+    
+    def _encode_before_iter(self, sample: LabeledSample, idx: int) -> PreEncodedTrainSample:
+        words = segment_text(sample.text)
+        inference_prompt = segment_text(self._prompt.prompt, self._tokenizer) + self._tokenizer.tokenize(self._training_label_prompt)
+        prompt_t = self._prompt.prompt + sample.label
+        training_prompt = segment_text(prompt_t, self._tokenizer)
+        return PreEncodedTrainSample(**asdict(sample), words=words, inference_prompt=inference_prompt, training_prompt=training_prompt)
+    
+    def _encode_on_iter(self, sample: PreEncodedTrainSample, idx: int) -> EncodedTrainSample:
+        rands = np.random.random(1)
+        if rands[0] < self._label_guided_rate:
+            prompt_tokens = sample.inference_prompt
+        else:
+            prompt_tokens = sample.training_prompt
+        prompt_tokens.insert(0, "[CLS]")
+        prompt_tokens.append("[SEP]")
+
+        max_second_len = self._max_length - len(prompt_tokens) - 3
+        sentence_tokens = sample.words[:max_second_len]
+        sentence_tokens.append("[SEP]")
+
+        new_tokens = prompt_tokens + sentence_tokens
+        token_type_ids = [0] * len(prompt_tokens) + \
+            [1] * (len(sentence_tokens))
+        attention_mask = [1] * len(new_tokens)
+        assert len(token_type_ids) == len(attention_mask)
+        while len(token_type_ids) < self._max_length:
+            token_type_ids.append(0)
+            attention_mask.append(0)
+        wwm_output = wwm_masking(
+            new_tokens=new_tokens,
+            index=idx,
+            tokenizer=self._tokenizer,
+            prompt_mode=self._prompt,
+            max_length=self._max_length,
+            mask_rate=self._wwm_mask_rate
+        )
+        source = wwm_output.input_ids
+        target = wwm_output.mlm_labels
+        attention_mask = wwm_output.input_mask
+        mask_positions = wwm_output.masked_lm_positions
+        assert len(source) == len(target) == len(token_type_ids)
+        while len(mask_positions) <= 30:
+            mask_positions.append(-100)
+        end_token = ["[SEP]"]
+        end_id_list = self._tokenizer.convert_tokens_to_ids(end_token)
+        assert isinstance(end_id_list, list)
+        end_id = end_id_list[0]
+        seq_actual_len = len(source) - source[::-1].index(end_id)
+        mask_positions = mask_positions[:30]
+        return EncodedTrainSample(
+            input_ids=torch.tensor(source).long(),
+            input_seg=torch.tensor(token_type_ids).long(),
+            input_mask=torch.tensor(attention_mask).long(),
+            mask_positions=torch.tensor(mask_positions).long(),
+            labels=torch.tensor(target).long(),
+            label_id=torch.tensor(sample.label_id).long(),
+            label_id_clf=torch.tensor(sample.label_id_clf).long(),
+            id=sample.id,
+            weight=torch.tensor(1).float(),
+            soft_label=torch.tensor(sample.soft_label).float(),
+            seq_len=seq_actual_len,
+            my_id=idx
+        )
+        
+class TestDatasetClfStd(BaseDatasetClf):
+    
+    def __init__(self, sample_list, tokenizer, prompt, inference_label_prompt: str, prefix_prompt: str, max_length: int):
+        self._inference_label_prompt = inference_label_prompt
+        self._prefix_prompt = prefix_prompt
+        self._max_length = max_length
+        super().__init__(sample_list, tokenizer, prompt)
+    
+    def _encode_before_iter(self, sample: LabeledSample, idx: int) -> EncodedTrainSample:
+        mask_positions = []
+        predict_prompt = self._inference_label_prompt
+        prompt = self._prefix_prompt + predict_prompt
+        encode_dict = self._tokenizer.encode_plus(prompt,
+                                                 text_pair=sample.text,
+                                                 truncation="only_second",
+                                                 max_length=self._max_length,
+                                                 pad_to_max_length=True)
+        encode_sent = encode_dict["input_ids"]
+        token_type_ids = encode_dict["token_type_ids"]
+        attention_mask = encode_dict["attention_mask"]
+        source, target = encode_sent[:], [-100] * len(encode_sent) # type: ignore
+
+        while len(mask_positions) <= 30:
+            mask_positions.append(-100)
+
+        end_token = ["[SEP]"]
+        end_id = self._tokenizer.convert_tokens_to_ids(end_token)[0]  # type: ignore
+        seq_actual_len = len(source) - source[::-1].index(end_id)
+
+        return EncodedTrainSample(
+            input_ids=torch.tensor(source).long(),
+            input_seg=torch.tensor(token_type_ids).long(),
+            input_mask=torch.tensor(attention_mask).long(),
+            labels=torch.tensor(target).long(),
+            mask_positions=torch.tensor(mask_positions[:30]).long(),
+            seq_len=seq_actual_len,
+            label_id=torch.tensor(sample.label_id).long(),
+            label_id_clf=torch.tensor(sample.label_id_clf).long(),
+            id=sample.id,
+            weight=torch.tensor(1).float(),
+            soft_label=torch.tensor(sample.soft_label).float(),
+            my_id=idx
+        )
+    
+class InfDatasetClfStd(BaseDatasetClf):
+    
+    def __init__(self, sample_list, tokenizer, prompt, inference_label_prompt: str, prefix_prompt: str, max_length: int):
+        self._inference_label_prompt = inference_label_prompt
+        self._prefix_prompt = prefix_prompt
+        self._max_length = max_length
+        super().__init__(sample_list, tokenizer, prompt)
+    
+    def _encode_before_iter(self, sample: InfSampleProto, idx: int) -> EncodedInfSample:
+        mask_positions = []
+        predict_prompt = self._inference_label_prompt
+        prompt = self._prefix_prompt + predict_prompt
+        encode_dict = self._tokenizer.encode_plus(sample.text,
+                                                 max_length=self._max_length,
+                                                 pad_to_max_length=True)
+        encode_sent = encode_dict["input_ids"]
+        token_type_ids = encode_dict["token_type_ids"]
+        attention_mask = encode_dict["attention_mask"]
+        source, target = encode_sent[:], [-100] * len(encode_sent) # type: ignore
+
+        while len(mask_positions) <= 30:
+            mask_positions.append(-100)
+
+        end_token = ["[SEP]"]
+        end_id = self._tokenizer.convert_tokens_to_ids(end_token)[0]  # type: ignore
+        seq_actual_len = len(source) - source[::-1].index(end_id)
+
+        return EncodedInfSample(
+            input_ids=torch.tensor(source).long(),
+            input_seg=torch.tensor(token_type_ids).long(),
+            input_mask=torch.tensor(attention_mask).long(),
+            my_id=idx
+        )
\ No newline at end of file
diff --git a/gts_engine/bagualu/entrances/__init__.py b/gts_engine/bagualu/entrances/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/gts_engine/bagualu/entrances/ie/__init__.py b/gts_engine/bagualu/entrances/ie/__init__.py
new file mode 100644
index 0000000..4676ed1
--- /dev/null
+++ b/gts_engine/bagualu/entrances/ie/__init__.py
@@ -0,0 +1,1 @@
+from .gts_engine_interface_std import GtsEngineInterfaceIEStd
diff --git a/gts_engine/bagualu/entrances/ie/gts_engine_interface_std.py b/gts_engine/bagualu/entrances/ie/gts_engine_interface_std.py
new file mode 100644
index 0000000..49bc2e1
--- /dev/null
+++ b/gts_engine/bagualu/entrances/ie/gts_engine_interface_std.py
@@ -0,0 +1,121 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import List, Optional
+from pathlib import Path
+
+from pydantic import BaseModel, DirectoryPath, FilePath
+
+from gts_common.arguments import GtsEngineArgs
+from ...lib.framework.base_gts_engine_interface import (TRAIN_MODE,
+                                                       BaseGtsEngineInterface)
+from .training_pipeline_std import TrainingPipelineIEStd
+from .inference_manager_std import InferenceManagerIEStd
+
+
+class TypeCheckedTrainArgs(BaseModel):
+    """ GTS-Engine相关参数进行runtime类型检查 """
+    task_dir: DirectoryPath
+    pretrained_model_dir: DirectoryPath
+    data_dir: DirectoryPath
+    save_path: DirectoryPath
+    train_data_path: FilePath
+    valid_data_path: FilePath
+    test_data_path: Optional[FilePath]
+    gpus: int
+    train_mode: TRAIN_MODE
+    seed: int
+    lr: float
+    num_workers: int
+    max_len: int
+    val_check_interval: float
+    train_batchsize: int
+    max_epochs: int
+    min_epochs: int
+
+
+class TypeCheckedInferenceArgs(BaseModel):
+    """ GTS-Engine相关参数进行runtime类型检查 """
+    task_dir: DirectoryPath
+    pretrained_model_root: DirectoryPath
+
+
+class GtsEngineInterfaceIEStd(BaseGtsEngineInterface):
+    """ GtsEngineInterfaceIEStd """
+
+    @property
+    def _training_pipeline_type(self):
+        return TrainingPipelineIEStd
+
+    def _parse_training_args(self, args: GtsEngineArgs) -> List[str]:
+        # args类型检查
+        checked_args = TypeCheckedTrainArgs(
+            task_dir=Path(args.task_dir),
+            pretrained_model_dir=Path(args.pretrained_model_dir),
+            data_dir=Path(args.data_dir),
+            save_path=Path(args.save_path),
+            train_data_path=args.train_data_path,
+            valid_data_path=args.valid_data_path,
+            test_data_path=args.test_data_path,
+            gpus=args.gpus,
+            train_mode=TRAIN_MODE(args.train_mode),
+            seed=args.seed,
+            lr=args.lr,
+            num_workers=args.num_workers,
+            max_len=args.max_len,
+            val_check_interval=args.val_check_interval,
+            train_batchsize=args.train_batchsize,
+            max_epochs=args.max_epochs,
+            min_epochs=args.min_epochs,
+        )
+
+        # 用户参数转化为实际参数
+        args_parse_list: List[str] = []
+        args_parse_list.extend(["--gts_input_path", str(checked_args.task_dir)])
+        args_parse_list.extend(["--gts_pretrained_model_path",
+                                str(checked_args.pretrained_model_dir)])
+        args_parse_list.extend(["--gts_output_dir", str(checked_args.save_path)])
+        args_parse_list.extend(["--gts_train_level", "1"])
+        args_parse_list.extend(["--gpus", str(checked_args.gpus)])
+        args_parse_list.extend(["--train_data_path", str(checked_args.train_data_path)])
+        args_parse_list.extend(["--dev_data_path", str(checked_args.valid_data_path)])
+        args_parse_list.extend(["--test_data_path", str(checked_args.test_data_path)])
+        args_parse_list.extend(["--learning_rate",str(checked_args.lr)])
+        args_parse_list.extend(["--num_workers", str(checked_args.num_workers)])
+        args_parse_list.extend(["--max_length", str(checked_args.max_len)])
+        args_parse_list.extend(["--val_check_interval", str(checked_args.val_check_interval)])
+        args_parse_list.extend(["--batch_size", str(checked_args.train_batchsize)])
+        args_parse_list.extend(["--max_epochs", str(checked_args.max_epochs)])
+        args_parse_list.extend(["--min_epochs", str(checked_args.min_epochs)])
+        args_parse_list.extend(["--entity_multi_label"])
+        args_parse_list.extend(["--relation_multi_label"])
+        return args_parse_list
+
+    @property
+    def _inference_manager_type(self):
+        return InferenceManagerIEStd
+
+    def _parse_inference_args(self, args: GtsEngineArgs) -> List[str]:
+        # args类型检查
+        checked_args = TypeCheckedInferenceArgs(
+            task_dir=Path(args.task_dir),
+            pretrained_model_root=Path(args.pretrained_model_dir)
+        )
+
+        # 用户参数转化为实际参数
+        args_parse_list: List[str] = []
+        args_parse_list.extend(["--task_dir", str(checked_args.task_dir)])
+        args_parse_list.extend(["--pretrained_model_root", str(checked_args.pretrained_model_root)])
+        return args_parse_list
diff --git a/gts_engine/bagualu/entrances/ie/inference_manager_std.py b/gts_engine/bagualu/entrances/ie/inference_manager_std.py
new file mode 100644
index 0000000..09a7de4
--- /dev/null
+++ b/gts_engine/bagualu/entrances/ie/inference_manager_std.py
@@ -0,0 +1,68 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import List
+
+from transformers import AutoTokenizer
+
+from ...lib.framework import BaseInferenceManager
+from ...lib.framework.mixin import OptionalLoggerMixin
+from ...models.ie import BagualuIEModel, BagualuIELitModel, BagualuIEExtractModel
+from ...arguments.ie import InferenceArgumentsIEStd
+from ...dataloaders.ie import data_segment, data_segment_restore
+
+
+class InferenceManagerIEStd(BaseInferenceManager, OptionalLoggerMixin):
+    """ InferenceManagerIEStd """
+
+    _args: InferenceArgumentsIEStd # inference所需参数
+    _inference_model: BagualuIEModel # inference模型
+    _extract_model: BagualuIEExtractModel # 抽取模型
+
+    def prepare_inference(self) -> None:
+        """ prepare inference """
+        # load model
+        self._inference_model = BagualuIELitModel.load_from_checkpoint(self._args.best_ckpt_path, # pylint: disable=protected-access
+                                                                       args=self._args,
+                                                                       logger=None)._model
+        self.info(f"loaded model from {self._args.best_ckpt_path}")
+
+        # load tokenizer
+        added_token = [f"[unused{i + 1}]" for i in range(99)]
+        tokenizer = AutoTokenizer.from_pretrained(self._args.model_save_dir,
+                                                  additional_special_tokens=added_token)
+        self.info(f"loaded tokenzier from {self._args.model_save_dir}")
+
+        # instantiate extract model
+        self._extract_model = BagualuIEExtractModel(tokenizer, self._args)
+
+    def inference(self, sample: List[dict]) -> List[dict]:
+        """ inference
+
+        Args:
+            sample (List[dict]): input samples for inference
+
+        Returns:
+            List[dict]: inference results
+        """
+        sample = data_segment(sample)
+        batch_size = self._args.batch_size
+        result = []
+        for i in range(0, len(sample), batch_size):
+            batch_data = sample[i: i + batch_size]
+            batch_result = self._extract_model.extract(batch_data, self._inference_model)
+            result.extend(batch_result)
+        result = data_segment_restore(result)
+        return result
diff --git a/gts_engine/bagualu/entrances/ie/training_pipeline_std.py b/gts_engine/bagualu/entrances/ie/training_pipeline_std.py
new file mode 100644
index 0000000..ca0136e
--- /dev/null
+++ b/gts_engine/bagualu/entrances/ie/training_pipeline_std.py
@@ -0,0 +1,314 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import datetime
+import shutil
+from logging import Logger
+from typing import List, Optional
+
+import torch
+from transformers import AutoTokenizer, PreTrainedTokenizer
+from pytorch_lightning import Trainer
+from pytorch_lightning.loggers import TensorBoardLogger
+from pytorch_lightning.callbacks import ModelCheckpoint, EarlyStopping
+
+from ...arguments.ie import TrainingArgumentsIEStd
+from ...models.ie import BagualuIELitModel, BagualuIEExtractModel, BagualuIEOnnxConfig
+from ...lib.framework.base_training_pipeline import BaseTrainingPipeline
+from ...lib.utils import LoggerManager
+from ...lib.utils.json_processor import (load_json_list,
+                                         dump_json,
+                                         dump_json_list)
+from ...dataloaders.ie import (BagualuIEDataModel,
+                               check_data,
+                               data_segment,
+                               data_segment_restore,
+                               eval_entity_f1,
+                               eval_entity_relation_f1)
+
+
+class TrainingPipelineIEStd(BaseTrainingPipeline):
+    """ training pipeline """
+
+    _args: TrainingArgumentsIEStd # training所需参数
+
+    def __init__(self, args_parse_list: Optional[List[str]] = None):
+        super().__init__(args_parse_list)
+
+        self._logger: Logger
+        self._tokenizer: PreTrainedTokenizer
+        self._data_module: BagualuIEDataModel
+        self._lit_model: BagualuIELitModel
+        self._trainer: Trainer
+        self._extract_model: BagualuIEExtractModel
+
+    def _before_training(self) -> None:
+        self._set_logger()
+        self._logger.info("use saving path: %s", self._args.ft_output_dir)
+        self._generate_tokenizer()
+        self._logger.info("phase before_training finished")
+
+    def _train(self) -> None:
+        self._get_data_module()
+        self._get_training_lightning()
+        self._get_trainer()
+        self._fit()
+        self._logger.info("phase train finished")
+
+    def _after_training(self) -> None:
+        best_ckpt = self._select_best_model_from_ckpts()
+        self._get_inf_model(best_ckpt)
+        self._extract_model = BagualuIEExtractModel(self._tokenizer, self._args)
+        self._generate_prediction_file()
+        self._export_onnx()
+        self._logger.info("phase after_training finished")
+
+    def _set_logger(self) -> None:
+        now_date = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
+        task_name = f"{self._args.dataset}_{now_date}"
+        log_file = os.path.join(self._args.log_dir, task_name + ".log")
+        LoggerManager.set_logger(self._args.logger, log_file)
+        self._logger = LoggerManager.get_logger(self._args.logger)
+        self._logger.info("log_file: %s", log_file)
+
+    def _generate_tokenizer(self):
+        self._logger.info("generate tokenizer...")
+        added_token = [f"[unused{i + 1}]" for i in range(99)]
+        tokenizer = AutoTokenizer.from_pretrained(self._args.pretrained_model_root,
+                                                  additional_special_tokens=added_token)
+        self._tokenizer = tokenizer
+
+    def _get_data_module(self):
+        train_data_path = self._args.train_data_path
+        dev_data_path = self._args.dev_data_path
+        test_data_path = self._args.test_data_path
+
+        # train data
+        if train_data_path and os.path.exists(train_data_path):
+            train_data = list(load_json_list(train_data_path))
+            self._logger.info("load train data from %s", train_data_path)
+            check_data(train_data)
+            train_data = data_segment(train_data)
+        else:
+            train_data = None
+            self._logger.error("training data path [%s] is not valid", train_data_path)
+
+        # dev data
+        if dev_data_path and os.path.exists(dev_data_path):
+            dev_data = list(load_json_list(dev_data_path))
+            self._logger.info("load dev data from %s", dev_data_path)
+            check_data(dev_data)
+            dev_data = data_segment(dev_data)
+        else:
+            dev_data = None
+            self._logger.warning("dev data path [%s] is not valid", dev_data_path)
+
+        # test data
+        if test_data_path and os.path.exists(test_data_path):
+            test_data = list(load_json_list(test_data_path))
+            self._logger.info("load test data from %s", test_data_path)
+            check_data(test_data)
+            test_data = data_segment(test_data)
+        else:
+            test_data = None
+            self._logger.warning("test data path [%s] is not valid", test_data_path)
+
+        self._data_module =  BagualuIEDataModel(self._tokenizer,
+                                                self._args,
+                                                train_data=train_data,
+                                                dev_data=dev_data,
+                                                test_data=test_data)
+
+    def _get_training_lightning(self):
+        self._lit_model = BagualuIELitModel(self._args, logger=self._logger)
+
+    def _get_trainer(self):
+        # add checkpoint callback
+        checkpoint_callback = ModelCheckpoint(monitor=self._args.ckpt_monitor,
+                                              save_top_k=self._args.ckpt_save_top_k,
+                                              mode=self._args.ckpt_mode,
+                                              save_last=self._args.ckpt_save_last,
+                                              every_n_train_steps=self._args.ckpt_every_n_train_steps,
+                                              save_weights_only=self._args.ckpt_save_weights_only,
+                                              dirpath=self._args.ckpt_dirpath,
+                                              filename=self._args.ckpt_filename)
+
+        # add early stop callback
+        early_stop_callback = EarlyStopping(monitor=self._args.ckpt_monitor,
+                                            mode=self._args.ckpt_mode,
+                                            patience=10)
+
+        # add logger
+        logger = TensorBoardLogger(save_dir=self._args.tensorboard_dir)
+
+        # trainer
+        trainer = Trainer(logger=logger,
+                          precision=self._args.precision,
+                          callbacks=[checkpoint_callback, early_stop_callback],
+                          accelerator=self._args.accelerator,
+                          devices=self._args.gpus,
+                          max_steps=self._args.max_steps,
+                          max_epochs=self._args.max_epochs,
+                          min_epochs=self._args.min_epochs,
+                          check_val_every_n_epoch=self._args.check_val_every_n_epoch,
+                          gradient_clip_val=self._args.gradient_clip_val,
+                          val_check_interval=self._args.val_check_interval,
+                          enable_progress_bar=self._args.enable_progress_bar,
+                          accumulate_grad_batches=self._args.accumulate_grad_batches,)
+
+        self._trainer = trainer
+
+    def _fit(self) -> None:
+        self._lit_model.num_data = len(self._data_module.train_data)
+        self._trainer.fit(self._lit_model, self._data_module)
+
+    def _select_best_model_from_ckpts(self) -> str:
+
+        # try to get best checkpoint path from ModelCheckpoint callback
+        self._logger.info("select model from checkpoints...")
+        for callback in self._trainer.callbacks:
+            if isinstance(callback, ModelCheckpoint):
+                best_model_path = callback.best_model_path
+                if best_model_path and os.path.exists(best_model_path):
+                    shutil.copy(best_model_path, self._args.best_ckpt_path)
+                    self._logger.info("best model path %s, copied to %s",
+                                      best_model_path,
+                                      self._args.best_ckpt_path)
+                    return self._args.best_ckpt_path
+
+        # if no best checkpoint found (e.g. no dev data), use the last checkpoint
+        if self._args.last_ckpt_path and os.path.exists(self._args.last_ckpt_path):
+            shutil.copy(self._args.last_ckpt_path, self._args.best_ckpt_path)
+            self._logger.warning("no ModelCheckpoint available, use last checkpoint")
+            return self._args.best_ckpt_path
+
+        self._logger.error("Unexpected error: No checkpoint available")
+        return self._args.best_ckpt_path
+
+    def _get_inf_model(self, checkpoint_path: str):
+        if not checkpoint_path or not os.path.exists(checkpoint_path):
+            self._logger.info("checkpoint [%s] is not valid. use final checkpoint.",
+                              checkpoint_path)
+        else:
+            self._logger.info("generating inference model from %s...", checkpoint_path)
+            self._lit_model = BagualuIELitModel.load_from_checkpoint(checkpoint_path,
+                                                                     args=self._args,
+                                                                     logger=None)
+        self._lit_model.eval()
+
+    def _generate_prediction_file(self):
+
+        # generate predictions from dev data
+        # if os.path.exists(self._args.dev_data_path):
+        #     self._logger.info("predicting on dev data..")
+        #     data = list(load_json_list(self._args.dev_data_path))
+        #     pred_data = self._generate_prediction_results(data)
+        #     dump_json_list(pred_data, os.path.join(self._args.prediction_save_dir,
+        #                                            "dev_prediction_results.json"))
+        #     eval_results = self._generate_evaluation_results(data, pred_data)
+        #     dump_json(eval_results,
+        #               os.path.join(self._args.prediction_save_dir,
+        #                            "dev_evaluation_results.json"),
+        #               indent=4)
+
+        # generate predictions from test data
+        if os.path.exists(self._args.test_data_path):
+            self._logger.info("predicting on test data..")
+            data = list(load_json_list(self._args.test_data_path))
+            pred_data = self._generate_prediction_results(data)
+            dump_json_list(pred_data, os.path.join(self._args.prediction_save_dir,
+                                                   "test_prediction_results.json"))
+            eval_results = self._generate_evaluation_results(data, pred_data)
+            dump_json(eval_results,
+                      os.path.join(self._args.prediction_save_dir,
+                                   "test_evaluation_results.json"),
+                      indent=4)
+
+    def _generate_prediction_results(self, data: List[dict]) -> List[dict]:
+        check_data(data)
+        data = data_segment(data)
+
+        batch_size = self._args.batch_size
+
+        result = []
+        for i in range(0, len(data), batch_size):
+            batch_data = data[i: i + batch_size]
+            batch_result = self._extract_model.extract(batch_data, self._lit_model._model) # pylint: disable=protected-access
+            result.extend(batch_result)
+
+        result = data_segment_restore(result)
+
+        return result
+
+    def _generate_evaluation_results(self, true_data: List[dict], pred_data: List[dict]):
+        eval_results = dict()
+
+        ner_true_data = [d for d in true_data if d["task"] in {"实体识别"}]
+        ner_pred_data = [d for d in pred_data if d["task"] in {"实体识别"}]
+        if ner_true_data:
+            f1, recall, precise = eval_entity_f1(ner_true_data, ner_pred_data)
+            eval_results["entity"] = {
+                "num_ner_samples": len(ner_true_data),
+                "num_ground_truth_entities": recall[1],
+                "num_predicted_enttities": precise[1],
+                "num_correctly_predicted_entities": precise[0],
+                "precision": precise[-1],
+                "recall": recall[-1],
+                "f1": f1
+            }
+
+        re_true_data = [d for d in true_data if d["task"] in {"关系抽取"}]
+        re_pred_data = [d for d in pred_data if d["task"] in {"关系抽取"}]
+        if re_true_data:
+            f1, recall, precise = eval_entity_relation_f1(re_true_data, re_pred_data)
+            eval_results["relation"] = {
+                "num_re_samples": len(re_true_data),
+                "num_ground_truth_relations": recall[1],
+                "num_predicted_relations": precise[1],
+                "num_correctly_predicted_relations": precise[0],
+                "precision": precise[-1],
+                "recall": recall[-1],
+                "f1": f1
+            }
+
+        return eval_results
+
+    def _export_onnx(self) -> None:
+
+        # save tokenizer
+        self._tokenizer.save_pretrained(self._args.model_save_dir)
+        self._logger.info("tokenizer saved to %s", self._args.model_save_dir)
+
+        # load model config
+        model_config = BagualuIEOnnxConfig()
+        bert_model = self._lit_model._model.cpu() # pylint: disable=protected-access
+        bert_model.eval()
+
+        # get dummy model input
+        input_sample = next(iter(self._data_module.train_dataloader()))
+        model_input = [input_sample[name] for name in model_config.input_names]
+
+        torch.onnx.export(
+            bert_model,
+            tuple(model_input),
+            f=self._args.onnx_saved_path,
+            input_names=model_config.input_names,
+            output_names=model_config.output_names,
+            dynamic_axes=model_config.dynamic_axes,
+            do_constant_folding=True,
+            opset_version=13
+        )
+        self._logger.info("onnx model saved to %s", self._args.onnx_saved_path)
diff --git a/gts_engine/bagualu/entrances/text_classification/__init__.py b/gts_engine/bagualu/entrances/text_classification/__init__.py
new file mode 100644
index 0000000..b9a5852
--- /dev/null
+++ b/gts_engine/bagualu/entrances/text_classification/__init__.py
@@ -0,0 +1,1 @@
+from .gts_engine_interface_std import GtsEngineInterfaceClfStd
\ No newline at end of file
diff --git a/gts_engine/bagualu/entrances/text_classification/gts_engine_interface_std.py b/gts_engine/bagualu/entrances/text_classification/gts_engine_interface_std.py
new file mode 100644
index 0000000..2407352
--- /dev/null
+++ b/gts_engine/bagualu/entrances/text_classification/gts_engine_interface_std.py
@@ -0,0 +1,139 @@
+from typing import List, Optional, Dict
+from pydantic import BaseModel, DirectoryPath, FilePath
+from pathlib import Path
+
+from ...lib.framework.base_gts_engine_interface import TRAIN_MODE, BaseGtsEngineInterface, GtsEngineArgs
+from ...lib.utils.json_processor import dump_json, load_json
+
+from .training_pipeline_std import TrainingPipelineClfStd
+from .inference_manager_std import InferenceManagerClfStd
+
+
+class TypeCheckedTrainArgs(BaseModel):
+    """GTS-Engine相关参数进行runtime类型检查与转换"""
+    task_dir: DirectoryPath
+    pretrained_model_dir: DirectoryPath
+    data_dir: DirectoryPath
+    save_path: DirectoryPath
+    train_data_path: FilePath
+    valid_data_path: FilePath
+    test_data_path: Optional[FilePath]
+    label_data_path: Optional[FilePath]
+    gpus: int
+    train_mode: TRAIN_MODE
+    seed: int
+    num_workers: int
+    train_batchsize: int
+    valid_batchsize: int
+    test_batchsize: int
+    max_length: int
+    learning_rate: float
+    
+class TypeCheckedInfArgs(BaseModel):
+    model_save_dir: DirectoryPath
+    label2id_path: FilePath
+
+
+class GtsEngineInterfaceClfStd(BaseGtsEngineInterface):
+
+    @property
+    def _training_pipeline_type(self):
+        return TrainingPipelineClfStd
+
+    def _parse_training_args(self, args: GtsEngineArgs) -> List[str]:
+        # args类型检查
+        type_checked_args = TypeCheckedTrainArgs(
+            task_dir=Path(args.task_dir),
+            pretrained_model_dir=Path(args.pretrained_model_dir),
+            data_dir=Path(args.data_dir),
+            save_path=Path(args.save_path),
+            train_data_path=args.train_data_path,
+            valid_data_path=args.valid_data_path,
+            test_data_path=args.test_data_path,
+            # 将在prepare_training()中通过label_data生成label2id.json
+            label_data_path=self.__get_label2id_path(args),
+            gpus=args.gpus,
+            train_mode=TRAIN_MODE(args.train_mode),
+            seed=args.seed,
+            num_workers=args.num_workers,
+            train_batchsize=args.train_batchsize,
+            valid_batchsize=args.valid_batchsize,
+            test_batchsize=args.test_batchsize,
+            max_length=args.max_len,
+            learning_rate=args.lr,
+        )
+        args_parse_list: List[str] = []
+        args_parse_list.extend(
+            ["--gts_input_path", str(type_checked_args.task_dir)])
+        args_parse_list.extend(
+            ["--gts_pretrained_model_path", str(type_checked_args.pretrained_model_dir)])
+        args_parse_list.extend(
+            ["--gts_output_dir", str(type_checked_args.save_path)])
+        args_parse_list.extend(["--gts_train_level", "1"])
+        args_parse_list.extend(["--gpu_num", str(type_checked_args.gpus)])
+        args_parse_list.extend(["--run_mode", "online"])
+        args_parse_list.extend(
+            ["--train_data_path", str(type_checked_args.train_data_path)])
+        args_parse_list.extend(
+            ["--dev_data_path", str(type_checked_args.valid_data_path)])
+        args_parse_list.extend(
+            ["--aug_eda_path", str(self.__get_eda_cache_path(args))])
+        args_parse_list.extend(
+            ["--num_workers", str(type_checked_args.num_workers)])
+        args_parse_list.extend(
+            ["--train_batchsize", str(type_checked_args.train_batchsize)])
+        args_parse_list.extend(
+            ["--valid_batchsize", str(type_checked_args.valid_batchsize)])
+        args_parse_list.extend(
+            ["--test_batchsize", str(type_checked_args.test_batchsize)])
+        args_parse_list.extend(
+            ["--max_length", str(type_checked_args.max_length)])
+        args_parse_list.extend(
+            ["--learning_rate", str(type_checked_args.learning_rate)])
+        if type_checked_args.test_data_path is not None:
+            args_parse_list.extend(
+                ["--test_data_path", str(type_checked_args.test_data_path)])
+        try:
+            assert type_checked_args.label_data_path is not None
+            args_parse_list.extend(
+                ["--label2id_path", str(type_checked_args.label_data_path)])
+        except:
+            raise Exception(
+                "you should pass label_data file in classification task")
+        args_parse_list.extend(
+            ["--log_dir", str(type_checked_args.task_dir / "logs")])
+        return args_parse_list
+
+    def prepare_training(self, args: GtsEngineArgs) -> None:
+        # 将label_data数据转为label2id格式
+        assert args.label_data_path is not None
+        label_data = load_json(args.label_data_path,
+                               type_=Dict[str, List[str]])
+        label2id = {
+            label: {"id": idx, "label_desc_zh": label}
+            for idx, label in enumerate(label_data["labels"])
+        }
+        dump_json(label2id, self.__get_label2id_path(args))
+
+    @property
+    def _inference_manager_type(self):
+        return InferenceManagerClfStd
+
+    def _parse_inference_args(self, args: GtsEngineArgs) -> List[str]:
+        type_checked_args = TypeCheckedInfArgs(
+            label2id_path=self.__get_label2id_path(args),
+            model_save_dir=Path(args.task_dir) / "outputs" /
+            "student_output" / "finetune_output"
+        )
+        args_parse_list: List[str] = []
+        args_parse_list.extend(
+            ["--model_save_dir", str(type_checked_args.model_save_dir)])
+        args_parse_list.extend(
+            ["--label2id_path", str(type_checked_args.label2id_path)])
+        return args_parse_list
+
+    def __get_label2id_path(self, args: GtsEngineArgs) -> FilePath:
+        return Path(args.task_dir) / f"{args.train_data.split('.')[0]}_label2id.json"
+
+    def __get_eda_cache_path(self, args: GtsEngineArgs) -> FilePath:
+        return Path(args.task_dir) / f"{args.train_data.split('.')[0]}_eda_augment.json"
diff --git a/gts_engine/bagualu/entrances/text_classification/inference_manager_std.py b/gts_engine/bagualu/entrances/text_classification/inference_manager_std.py
new file mode 100644
index 0000000..5381df1
--- /dev/null
+++ b/gts_engine/bagualu/entrances/text_classification/inference_manager_std.py
@@ -0,0 +1,27 @@
+from typing import List
+
+from ...lib.framework.classification_finetune import BaseInferenceManagerClf
+from ...lib.framework.classification_finetune.consts import InferenceManagerInputSampleList
+
+from ...models.text_classification.lightnings_std import InferenceLightningClfStd
+from ...dataloaders.text_classification.datasets_std import InfDatasetClfStd
+from ...arguments.text_classification.arguments_std import InferenceArgumentsClfStd
+
+class InferenceManagerClfStd(BaseInferenceManagerClf):
+    
+    _args: InferenceArgumentsClfStd
+
+    def _get_inf_lightning(self) -> InferenceLightningClfStd:
+        return InferenceLightningClfStd(
+            self._prompt, 
+            self._args,
+            self._tokenizer
+        )
+    
+    def _get_dataset(self, sample: List[InferenceManagerInputSampleList]) -> InfDatasetClfStd:
+        return InfDatasetClfStd(
+            sample, self._tokenizer, self._prompt, 
+            self._args.inference_label_prompt,
+            self._args.prefix_prompt,
+            self._args.max_length
+        )
\ No newline at end of file
diff --git a/gts_engine/bagualu/entrances/text_classification/training_pipeline_std.py b/gts_engine/bagualu/entrances/text_classification/training_pipeline_std.py
new file mode 100644
index 0000000..ad8e749
--- /dev/null
+++ b/gts_engine/bagualu/entrances/text_classification/training_pipeline_std.py
@@ -0,0 +1,166 @@
+from pytorch_lightning import Trainer, seed_everything
+from pytorch_lightning.callbacks import Callback, ModelSummary, ModelCheckpoint
+from typing import List, Literal, Tuple, Dict, Union
+import torch
+import os
+import shutil
+import numpy as np
+
+from ...lib.framework.classification_finetune import BaseTrainingPipelineClf
+from ...lib.framework.classification_finetune.consts import InferenceManagerInputSample
+from ...lib.framework.consts import TRAINING_STAGE
+from ...lib.components.lightning_callbacks.adaptive_val_intervals import AdaptiveValIntervalTrainLoss, ADAPTIVE_VAL_INTERVAL_MODE, AdaptiveValIntervalFixed
+from ...lib.components.knn_tools import BasicDatastore, grid_search_for_hyper, get_datastore
+
+from ...arguments.text_classification.arguments_std import TrainingArgumentsClfStd
+from ...dataloaders.text_classification.data_module_std import DataModuleClfStd
+from ...models.text_classification.lightnings_std import TrainLightningClfStd, PredictLightningClfStd
+
+class TrainingPipelineClfStd(BaseTrainingPipelineClf):
+    
+    _args: TrainingArgumentsClfStd
+    _mode_name: str = "ft_std"
+    
+    def _get_data_module(self) -> DataModuleClfStd:
+        return DataModuleClfStd(self._args, self._prompt, self._tokenizer)
+    
+    def _get_training_lightning(self) -> TrainLightningClfStd:
+        return TrainLightningClfStd(self._args, self._data_module.class_num, self._data_module.train_sample_num)
+    
+    def _get_trainer(self) -> Trainer:
+        callbacks = self.__get_callbacks(self._data_module.dev_sample_num)
+        return Trainer(
+            accelerator="gpu", 
+            devices=self._args.gpu_num, 
+            callbacks=callbacks,
+            max_epochs=self._args.epoch,
+            gradient_clip_val=self._args.clip_norm,
+            default_root_dir=str(self._output_dir),
+            enable_progress_bar=False,
+            auto_select_gpus=True,
+            strategy="dp",
+            logger=self._get_lightning_loggers(),
+            precision=self._args.precision,
+        )
+        
+    def _fit(self) -> None:
+        seed_everything(self._args.seed)
+        self._logger.info('-' * 30 + 'Args' + '-' * 30)
+        vars_str = ''
+        for k, v in vars(self._args).items():
+            vars_str += str(k)+':'+str(v)+'\t'
+        self._logger.info(vars_str)
+        self._logger.info('\n' + '-' * 64)
+
+        self._trainer.fit(
+            model=self._training_lightning, 
+            train_dataloaders=self._data_module.train_dataloader(load_ratio=self._args.load_data_ratio),
+            val_dataloaders=self._data_module.val_dataloader(
+                stage=TRAINING_STAGE.VALIDATION, 
+                load_ratio=self._args.load_data_ratio, 
+                resample_thres=self._args.dev_resample_thres),
+        )
+        
+    def _get_inf_lightning(self, use_knn=False, datastore=None, best_hyper=None) -> PredictLightningClfStd:
+        if use_knn:
+            return PredictLightningClfStd(self._prompt, self._args, self._tokenizer, datastore=datastore, best_hyper=best_hyper)
+        else:
+            return PredictLightningClfStd(self._prompt, self._args, self._tokenizer)
+    
+    def __get_callbacks(self, dev_num: int) -> List[Callback]:
+        model_summary = ModelSummary(max_depth=2)
+        if self._args.validation_mode == ADAPTIVE_VAL_INTERVAL_MODE.ADAPTIVE and self._data_module.class_num > 2: 
+            adaptive_test_interval_cls = AdaptiveValIntervalTrainLoss
+        else:
+            adaptive_test_interval_cls = AdaptiveValIntervalFixed
+        adaptive_test_interval = adaptive_test_interval_cls(
+            int(self._data_module.train_sample_num * self._args.load_data_ratio),
+            self._args.train_batch_size,
+            self._args.logger
+        )
+        if dev_num == 0: # 无验证数据时，保存最终模型
+            model_checkpoint = ModelCheckpoint(
+                monitor="step",
+                mode="max", 
+                save_top_k=1,
+                save_on_train_epoch_end=True, 
+                verbose=True,
+                dirpath=self._output_dir
+            )
+        elif dev_num <= self._args.dev_resample_thres: # 验证数据较少时，保存dev_loss最小模型
+            model_checkpoint = ModelCheckpoint(
+                monitor="dev_acc",
+                mode="max", 
+                save_top_k=1,
+                save_on_train_epoch_end=False, 
+                verbose=True,
+                dirpath=self._output_dir
+            )
+        else: # 验证数据较多时，重采样验证数据，保存dev_loss最小的五个模型，训练完成后再用全样本筛选出最优模型
+            self._logger.info("dev sample is too large, resample dev data...")
+            model_checkpoint = ModelCheckpoint(
+                monitor="dev_acc",
+                mode="max", 
+                save_top_k=5,
+                save_on_train_epoch_end=False, 
+                verbose=True,
+                dirpath=self._output_dir
+            )
+        return [
+            model_summary,
+            adaptive_test_interval,
+            model_checkpoint
+        ]
+
+    def _after_training(self) -> None:
+        # select best model from ckpts
+        self._logger.info("select model from checkpoints...")
+        best_ckpt = self._select_best_model_from_ckpts()
+        # The best ckpt model to make predictions on test_public, and outputs the accuracy
+        if self._args.debug:
+            if self._args.test_data_path is None:
+                self._logger.info(
+                    "test_data_path is not passed, skip testing...")
+            elif not self._args.test_data_path.exists():
+                self._logger.info(f"test_data_path {self._args.test_data_path}"
+                                  f" does not exist, skip testing...")
+            else:
+                self._logger.info("implement test on training model...")
+                self._implement_test(best_ckpt)
+        # generating inference model, test predicting, and save onnx model
+        self._logger.info("generating inference model...")
+        state_dict = self._load_ckpt(best_ckpt).get_model_state_dict()
+        self._inference_lightning = self._get_inf_lightning()
+        self._inference_lightning.load_model_from_state_dict(state_dict)
+        self._inference_lightning.model.eval()
+        
+        if self._args.use_knn:
+            # Create a knn datastore with train data
+            self._logger.info("generating knn datastore...")
+            datastore = get_datastore(model=self._inference_lightning.model.cuda(), 
+                                        data_model=self._get_data_module().raw_train_dataloader())
+            best_hyper,_,_ = grid_search_for_hyper(datastore, 
+                                                    self._get_data_module().val_dataloader(stage=TRAINING_STAGE.VALIDATION), 
+                                                    self._inference_lightning.model.cuda())
+            self._logger.info("best_hyper of knn is {}, base dev dataset".format(best_hyper))
+            
+            # construct knn_based inference model
+            self._inference_lightning = self._get_inf_lightning(use_knn=self._args.use_knn, datastore=datastore, best_hyper=best_hyper)
+            self._inference_lightning.load_model_from_state_dict(state_dict)
+            self._inference_lightning.model.eval()
+        self._logger.info("generate prediction file...")
+        self._generate_prediction_file()
+        self._logger.info("save model files...")
+        torch.save(state_dict, os.path.join(self._output_dir, "finetune_pytorch.bin"))
+        self._logger.info("export model to onnx...")
+        self._export_onnx()
+
+        self._logger.info("generate and export evaluation results...")
+        self._save_eval_results()
+        
+        self._logger.info("cleaning checkpoints...")
+        for ckpt in self._ckpt_file_list:
+            os.remove(ckpt)
+        if os.path.exists(lightning_logs_path := os.path.join(self._output_dir, "lightning_logs")):
+            shutil.rmtree(lightning_logs_path)
+        self._copy_output_files()
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/__init__.py b/gts_engine/bagualu/lib/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/gts_engine/bagualu/lib/components/__init__.py b/gts_engine/bagualu/lib/components/__init__.py
new file mode 100644
index 0000000..80f27dc
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/__init__.py
@@ -0,0 +1,4 @@
+from ..utils.logger_manager import LoggerManager
+from .protocol_args import ProtocolArgs, ProtocolArgsMixin
+from .tokenizer_generator import TokenizerGenerator
+from .eda import EDA
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/eda.py b/gts_engine/bagualu/lib/components/eda.py
new file mode 100644
index 0000000..d8a87ae
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/eda.py
@@ -0,0 +1,76 @@
+from typing import Generator, Optional, List, Dict, Protocol, Sequence, Type, TypeVar, Generic, Union
+import os
+from transformers.tokenization_utils import PreTrainedTokenizer
+from dataclasses import asdict, dataclass
+import multiprocessing as mp
+from tqdm import tqdm
+import time
+import copy
+from pydantic import FilePath
+
+from ..utils.json_processor import load_json_list, dump_json_list
+from .text_tools import segment_text
+from ..framework.mixin import OptionalLoggerMixin
+
+class SampleProto(Protocol):
+    """传入的数据需要有text字段"""
+    text: str
+    
+_SampleType = TypeVar("_SampleType", bound=SampleProto)
+
+class EDA(OptionalLoggerMixin, Generic[_SampleType]):
+    #############################################################################################
+    ######################################## public ##########################################
+    def __init__(
+        self, 
+        tokenizer: PreTrainedTokenizer,
+        alpha: float = 0.1, 
+        logger_name: Optional[str] = None
+    ):
+        from textda.data_expansion import data_expansion # 加载时间较长，放在类初始化时进行
+        self.__data_expansion = data_expansion
+        OptionalLoggerMixin.__init__(self, logger_name)
+        self._alpha = alpha
+        self._tokenizer = tokenizer
+    
+    def eda_aug(self, sample_list: Sequence[_SampleType], aug_path: Union[FilePath, str], aug_num: int = 10) -> Sequence[_SampleType]:
+        self.__sample_cls = type(sample_list[0])
+        if os.path.exists(aug_path):
+            self.info("EDA file exists, load data...")
+            return self.__load_data(aug_path)
+        else:
+            self.info("generating EDA data...")
+            start = time.time()
+            eda_sample_list = self.__generate_data(sample_list, aug_num)
+            dump_json_list(eda_sample_list, aug_path)
+            end = time.time()
+            self.info(f"generating EDA data...finished, consuming {end - start:.4f}s")
+            return eda_sample_list
+        
+    #############################################################################################
+    ######################################## private ##########################################    
+    
+    def __load_data(self, aug_path: Union[FilePath, str]) -> List[_SampleType]:
+        eda_sample_list = list(load_json_list(aug_path, type_=self.__sample_cls))
+        return eda_sample_list
+    
+    def __generate_data(self, sample_list: Sequence[_SampleType], aug_num: int = 10) -> List[_SampleType]:
+        res: List[_SampleType] = []
+        pool = mp.Pool(processes=8)
+        iters = pool.imap(self._process_sample, sample_list)
+        with tqdm(total=len(sample_list), desc="EDA Augmentation") as p_bar:
+            for eda_sample_list in iters:
+                res += eda_sample_list
+                p_bar.update(1)
+            self.info(p_bar.__str__())
+        pool.close()
+        pool.join()
+        return res
+        
+    def _process_sample(self, sample: _SampleType, aug_num: int = 10) -> List[_SampleType]:
+        def create_eda_sample(new_text: str) -> _SampleType:
+            new_sample = copy.deepcopy(sample)
+            new_sample.text = new_text
+            return new_sample
+            
+        return [create_eda_sample(text) for text in self.__data_expansion(sample.text, num_aug=aug_num)]
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/es_search/__init__.py b/gts_engine/bagualu/lib/components/es_search/__init__.py
new file mode 100644
index 0000000..4fd47fb
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/es_search/__init__.py
@@ -0,0 +1,2 @@
+# from .es_search_engine import ESSearchEngine
+# from .es_search_args import ESSearchArgs, SEARCH_CLIENT
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/es_search/es_search_args.py b/gts_engine/bagualu/lib/components/es_search/es_search_args.py
new file mode 100644
index 0000000..de63811
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/es_search/es_search_args.py
@@ -0,0 +1,59 @@
+
+from typing import Optional
+from enum import Enum
+
+from ...framework import BaseArguments
+
+class SEARCH_CLIENT(Enum):
+    WEAVIATE = 0
+    WENTIAN = 1
+
+class ESSearchArgs(BaseArguments):
+    """es搜索参数
+    """
+    #############################################################################################
+    ######################################## 外部参数 ##########################################
+    augm_es_path: Optional[str]
+    
+    def _add_args(self, parser) -> None:
+        parser.add_argument("--augm_es_path", dest="augm_es_path", type=str, default=None, help="[可选]指定es search缓存文件路径")
+        
+    #############################################################################################
+    ######################################## 固定参数 ##########################################
+    
+    dev_step = 20  
+    """间隔dev_step步做一次验证"""
+    required_minimum_data = 1000 
+    """tapt所需最少的数据量"""
+    es_search_num = 20 
+    """es 默认检索的数据量"""
+    search_batch_size = 3 
+    max_augment_data = 12000 
+    """最大检索的数据量"""
+    addtion_search = 20 
+    """es额外需要增加的检索数据量"""
+    es_search_query = 2000 
+    """最多检索的query的个数"""
+    es_request_timeout = 1800 
+    """es请求超时的数据量"""
+    es_num_thread = 16 
+    """es并行处理的线程数"""
+    es_max_runtime = 90 
+    """es检索的"""
+    stratification_sample = 1 
+    tapt_max_runtime = 180 
+    """tapt模块整体的时间"""
+    add_label_description: bool = True
+    
+    start_time: float
+    es_cllient = SEARCH_CLIENT.WEAVIATE
+    
+    
+    #############################################################################################
+    ######################################## info ##########################################
+    @property
+    def _arg_name(self):
+        return "ES搜索参数"
+    
+
+    
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/es_search/es_search_client.py b/gts_engine/bagualu/lib/components/es_search/es_search_client.py
new file mode 100644
index 0000000..f4269a0
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/es_search/es_search_client.py
@@ -0,0 +1,207 @@
+from abc import ABCMeta, abstractmethod
+from logging import Logger
+from typing import Protocol, Optional, List, Type, Tuple
+import time
+from dataclasses import dataclass
+import weaviate
+from alibabacloud_tea_util import models as util_models
+from alibabacloud_ha3engine import models, client
+import json
+
+
+
+from .es_search_args import ESSearchArgs, SEARCH_CLIENT
+from ...components import ProtocolArgs, LoggerManager
+from ...framework.consts import RUN_MODE
+from ...framework.mixin import OptionalLoggerMixin
+
+
+
+#############################################################################################
+## types
+#############################################################################################
+
+class ArgumentsProto(Protocol):
+    train_data_path: Optional[str]
+    unlabeld_data_path: Optional[str]
+    start_time: float
+    @property
+    def run_mode(self) -> RUN_MODE: ...
+    es_search_args: ESSearchArgs
+    logger: Optional[str]
+    
+@dataclass
+class SearchedSample:
+    id: int
+    content: str
+
+Id_QueryVector = Tuple[int, List[float]]
+
+#############################################################################################
+## base
+#############################################################################################
+
+
+class SearchClient(OptionalLoggerMixin, metaclass=ABCMeta):
+    #############################################################################################
+    ##################################### public #############################################
+    def __init__(self, args: ArgumentsProto) -> None:
+        OptionalLoggerMixin.__init__(self, args.logger)
+        self._args = args
+        
+    def es_search(self, id_query_vector: Id_QueryVector) -> Optional[List[SearchedSample]]:
+        if self._already_run_time_min > self._args.es_search_args.es_max_runtime:
+            self.info("es search exceeds maximum time")
+            return
+        content_list = self._fetch_data(id_query_vector[1])
+        es_result = self._select_and_tokenize(id_query_vector[0], content_list)
+        
+        return es_result
+    
+    #############################################################################################
+    ##################################### protected #############################################
+    @property
+    def _already_run_time_min(self) -> float:
+        curr_time = time.time()
+        already_run_time = curr_time -  self._args.es_search_args.start_time
+        return already_run_time / 60
+    
+    def _select_and_tokenize(self, id: int, content_list: List[str]) -> List[SearchedSample]:
+        content_list = sorted(content_list, key=lambda content : len(content), reverse=True)
+        es_search_length = min(self._args.es_search_args.es_search_num - self._args.es_search_args.addtion_search, 
+                               len(content_list))
+        content_list = content_list[:es_search_length]
+        return [SearchedSample(id, content) for content in content_list]
+    
+    #############################################################################################
+    ##################################### virtual #############################################
+    @abstractmethod
+    def initializer(self) -> None:
+        """初始化client，作为类静态属性
+        """
+        pass
+    
+    
+    @abstractmethod
+    def _fetch_data(self, query_vector: List[float]) -> List[str]:
+        """从数据库检索数据
+
+        Args:
+            query_vector (List[float]): 
+
+        Returns:
+            List[str]: 检索出的未处理的字符串列表
+        """
+        pass
+    
+#############################################################################################
+## Derived
+#############################################################################################
+class WeaviateClient(SearchClient):
+    
+    @property
+    def _server_url(self) -> str:
+        if self._args.run_mode == RUN_MODE.OFFLINE:
+            return ""  # 开源暂时隐藏
+        elif self._args.run_mode == RUN_MODE.ONLINE:
+            return ""  # 开源暂时隐藏
+        else:
+            raise Exception("mode is not supported")
+    
+    def __init__(self, args=None):
+        super().__init__(args)
+    
+    def initializer(self) -> None:
+        WeaviateClient.client = None
+        try:
+            WeaviateClient.client = weaviate.Client(self._server_url, timeout_config=(5, 90)) # type: ignore
+        except TypeError:
+            raise Exception('Error, can\'t connect to vector database server, is it running?')
+
+    def _fetch_data(self, query_vector: List[float]) -> List[str]:
+        # weavaite中的表名
+        table_name = "Generation"
+        # 对应的字段名
+        # 存储向量的字段
+        vector_name = "vector"
+        # 存储文本的字段
+        text_name = "content"
+        # 存储文本长度信息的字段
+        len_name = "counter"
+        # 执行查询
+        # 构建向量查询条件
+        nearVector = {vector_name: query_vector}
+        # 需要返回的数据数量
+        top = self._args.es_search_args.es_search_num
+        # top = 10
+        # 文本限制的长度，字数，不是字节数
+
+        # with_where可增加多个，但目前只支持文本长度过滤
+        # operator可以为LessThan、GreaterThan、Equal
+        if WeaviateClient.client:
+            res = WeaviateClient.client.query.get(
+                    table_name, [text_name]).with_near_vector(
+                    nearVector).with_limit(
+                    top).with_additional(
+                    vector_name).do()
+        else:
+            raise Exception("client is not initialised")
+        data = res['data']['Get']['Generation']
+        es_result = ["".join(data_[text_name]) for data_ in data]
+        return es_result
+
+class WentianClient(SearchClient):
+    def __init__(self, args=None):
+        super().__init__(args)
+        self.info('endpoint : {}'.format(self._args.es_search_args.endpoint))
+        
+    def initializer(self) -> None:
+        endpoint = "" \
+                if self._args.run_mode == RUN_MODE.OFFLINE else ""  # 开源暂时隐藏
+        wentian_config = models.Config(
+            endpoint=endpoint,
+            instance_id="ha-cn-zvp2qr1sk01",
+            protocol="http",
+            access_user_name="ccnl",
+            access_pass_word="2022idea"
+        )
+        # 如用户请求时间较长. 可通过此配置增加请求等待时间. 单位 ms
+        # 此参数可在 push_documents_with_options 方法中使用
+        runtime = util_models.RuntimeOptions(
+            connect_timeout=5000,
+            read_timeout=10000,
+            autoretry=False,
+            ignore_ssl=False,
+            max_idle_conns=50
+        )
+        
+        # 初始化 Ha3Engine Client
+        WentianClient.ha3EngineClient = client.Client(wentian_config)
+
+
+    def _fetch_data(self, query_vector: List[float]) -> List[str]:
+        vector = ','.join(str(x) for x in query_vector)
+        queryinfo="&&query=generation_base_index:\'{}&n={}\'".format(vector, str(self._args.es_search_args.es_search_num))
+        query_str=f"kvpairs=first_formula:proxima_score(generation_base_index)&&sort=-RANK&&config=start:0,hit:{self._args.es_search_args.es_search_num},format:json&&cluster=general&&filter=fieldlen(content)>24"
+        optionsHeaders = {}
+        haSearchQuery = models.SearchQuery(query=query_str+queryinfo)
+        haSearchRequestModel = models.SearchRequestModel(optionsHeaders, haSearchQuery)
+        hastrSearchResponseModel = WentianClient.ha3EngineClient.search(haSearchRequestModel)
+        data = json.loads(hastrSearchResponseModel.body.__str__()) # type: ignore
+        content_items = data['result']['items']
+        es_result = []
+        for item in content_items:
+            es_result.append(item['fields']['content'])
+        return es_result
+
+
+#############################################################################################
+## factory
+#############################################################################################
+def search_client_factory(client: SEARCH_CLIENT) -> Type[SearchClient]:
+    if client == SEARCH_CLIENT.WEAVIATE:
+        return WeaviateClient
+    elif client == SEARCH_CLIENT.WENTIAN:
+        return WentianClient
+    else:
+        raise Exception("client is not supported")
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/es_search/es_search_engine.py b/gts_engine/bagualu/lib/components/es_search/es_search_engine.py
new file mode 100644
index 0000000..a438136
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/es_search/es_search_engine.py
@@ -0,0 +1,191 @@
+from dataclasses import dataclass, asdict
+import os
+from random import shuffle
+from typing import Callable, Dict, List, Optional, NamedTuple, Protocol, Generator, Tuple
+from copy import copy
+from sentence_transformers import SentenceTransformer
+import torch
+import time
+import multiprocessing as mp
+import json
+from tqdm import tqdm
+
+from .es_search_args import ESSearchArgs
+from .es_search_client import search_client_factory
+from ...framework.classification_finetune.consts import Label2Token, LabeledSample
+from ...framework.classification_finetune import DataReaderClf
+from ...framework.consts import RUN_MODE
+from ...components import ProtocolArgs, LoggerManager
+from ...framework.mixin import OptionalLoggerMixin
+
+
+os.environ["TOKENIZERS_PARALLELISM"] = "false"
+
+
+#############################################################################################
+## types
+#############################################################################################
+
+class ArgumentsProto(Protocol):
+    train_data_path: Optional[str]
+    unlabeld_data_path: Optional[str]
+    es_search_args: ESSearchArgs
+    start_time: float
+    run_mode: RUN_MODE
+    logger: Optional[str]
+    gpu_id: int
+    pretrained_model_root: str
+
+Id_Text = Tuple[int, str]
+Id_QueryVector = Tuple[int, List[float]]
+
+#############################################################################################
+## engine
+#############################################################################################
+
+class ESSearchEngine(OptionalLoggerMixin):
+    """es检索器
+    """
+    
+    #############################################################################################
+    ######################################## public ##########################################
+    
+    def __init__(
+        self, 
+        args: ArgumentsProto, 
+        label2token: Label2Token
+    ):
+        OptionalLoggerMixin.__init__(self, args.logger)
+        self._args = args
+        self._label2token = label2token
+    
+    def es_search_augment(self) -> None:
+        # 执行条件
+        if self._args.es_search_args.augm_es_path is None:
+            raise Exception("path for es search file is not passed")
+        if os.path.exists(self._args.es_search_args.augm_es_path):
+            self.info("es file already exists, skip es search")
+            return
+        # 加载和计算数据
+        text_dict = self.__load_text_dict()
+        search_text_list = self.__gen_search_text_list(text_dict)
+        self._args.es_search_args.es_search_num = self.__cal_es_search_num(search_text_list)
+        if len(search_text_list) == 0 or self._args.es_search_args.es_search_num == 0:
+            self.info("no text needs to be augmented, skip es search")
+            return
+        self.info(f"length of the text list for es search(`len(search_text_list)`): {len(search_text_list)}")
+        self.info(f"search number for each text(`es_search_args.es_search_num`)：{self._args.es_search_args.es_search_num}")
+        # 生成搜索向量
+        cuda_id = f"cuda:{self._args.gpu_id}"
+        device = torch.device(cuda_id if torch.cuda.is_available() else "cpu")
+        model = self.__load_sbert_client(device)
+        query_vector_list = self.__gen_query_vector_list(search_text_list, model, device)
+        self.info(f"query vectors num: {len(query_vector_list)}")
+        # 启动搜索
+        self._args.es_search_args.start_time = time.time()
+        client = search_client_factory(self._args.es_search_args.es_cllient)(self._args)
+        es_search_result_num = 0
+        with open(self._args.es_search_args.augm_es_path, 'w', encoding='utf8') as output_file, \
+                mp.Pool(processes=self._args.es_search_args.es_num_thread, initializer=client.initializer) as pool:
+            iters = pool.imap(client.es_search, query_vector_list)
+            for searched_sample_list in tqdm(iters, desc='es Augmentation'):
+                if searched_sample_list is not None:
+                    output_file.writelines([json.dumps(asdict(searched_sample), ensure_ascii=False) + "\n" 
+                                            for searched_sample in searched_sample_list])
+                    es_search_result_num += len(searched_sample_list)
+        es_end_time = time.time()
+        # 输出信息
+        self.info('the spend time of es search :%d' % (es_end_time - self._args.es_search_args.start_time))
+        self.info('es search num:%d' % (es_search_result_num))
+        del model
+            
+    #############################################################################################
+    ######################################## private ##########################################
+            
+    def __load_text_dict(self) -> Dict[str, List[Id_Text]]:
+        """按label加载数据字典
+        {
+            "label1": [(id1, text1), (id2, text2), ...]
+            "label2": ...
+            ...
+            "unlabeled": ...
+        }
+        """
+        text_dict: Dict[str, List[Id_Text]] = {}
+        if self._args.train_data_path is not None and os.path.exists(self._args.train_data_path):
+            if self._args.es_search_args.add_label_description:
+                sample_handler: Callable[[LabeledSample], Id_Text] = lambda sample : (sample.id, sample.text + ',这是' + sample.label)
+            else:
+                sample_handler: Callable[[LabeledSample], Id_Text] = lambda sample : (sample.id, sample.text)
+            for sample in DataReaderClf.read_labeled_sample(self._args.train_data_path, self._label2token):
+                if sample.label not in text_dict.keys():
+                    text_dict[sample.label] = []
+                text_dict[sample.label].append(sample_handler(sample))
+        if self._args.unlabeld_data_path is not None and os.path.exists(self._args.unlabeld_data_path):
+            text_dict["unlabeled"] = [(sample.id, sample.text) for sample in DataReaderClf.read_unlabeled_sample(self._args.unlabeld_data_path)]
+            
+        return text_dict
+    
+    def __gen_search_text_list(self, text_dict: Dict[str, List[Id_Text]]) -> List[Id_Text]:
+        """根据条件加载需要进行搜索的句子列表
+        """
+        text_num = sum([len(val) for val in text_dict.values()])
+        search_text_list: List[Id_Text] = []
+        # 如果训练文本数量足够，则无需增强
+        if text_num > self._args.es_search_args.max_augment_data or text_num == 0:
+            pass
+        # 如果训练文本数量不足但超出搜索query上限，需要筛选用于搜索的text
+        elif self._args.es_search_args.stratification_sample and text_num > self._args.es_search_args.es_search_query:
+            labeled_text_dict = copy(text_dict)
+            unlabeled_list = []
+            if "unlabeled" in labeled_text_dict.keys():
+                unlabeled_list = labeled_text_dict.pop("unlabeled")
+            label_num = len(labeled_text_dict)
+            search_num_each_label = int(self._args.es_search_args.es_search_query / label_num) \
+                                if label_num > 0 else 0
+            # 优先使用标注数据
+            for text_list in labeled_text_dict.values():
+                shuffle(text_list)
+                search_text_list += text_list[:search_num_each_label]
+            # 多余的query用无标注数据填充
+            unlabeled_search_num = self._args.es_search_args.es_search_query - len(search_text_list)
+            if unlabeled_search_num > 0:
+                search_text_list += unlabeled_list[:unlabeled_search_num]
+        # 否则使用所有的text进行搜索
+        else:
+            for text_list in text_dict.values():
+                search_text_list += text_list
+        
+        return search_text_list
+    
+    def __cal_es_search_num(self, search_text_list: List[Id_Text]) -> int:
+        """计算每条text需要搜索的结果数量"""
+        es_search_num = int(self._args.es_search_args.max_augment_data / len(search_text_list)) + \
+                        self._args.es_search_args.addtion_search \
+                    if len(search_text_list) > 0 else 0
+        return min(4000, es_search_num) # 每条数据搜索量不超过4k
+     
+    def __load_sbert_client(self, device) -> SentenceTransformer:
+        """加载sbert"""
+        self.info("loading sbert for es search...")
+        if self._args.run_mode == RUN_MODE.OFFLINE:
+            model = SentenceTransformer('paraphrase-multilingual-MiniLM-L12-v2', device=device) 
+        else:
+            model_root = self._args.pretrained_model_root
+            model_path = os.path.join(model_root, 'paraphrase-multilingual-MiniLM-L12-v2')
+            self.info(f"sentence transformer path: {model_path}")
+            model = SentenceTransformer(model_path, device=device)
+        self.info("loading sbert for es search...finished")
+        return model
+    
+    def __gen_query_vector_list(self, search_text_list: List[Id_Text], model: SentenceTransformer, device) -> List[Id_QueryVector]:
+        """计算查询向量"""
+        id_query_vector_list: List[Tuple[int, torch.Tensor]] = []
+        batch_size = self._args.es_search_args.search_batch_size
+        for idx in range(0, len(search_text_list), batch_size):
+            id_text_batch = search_text_list[idx : idx + batch_size]
+            text_batch = [id_text[1] for id_text in id_text_batch]
+            query_vector_batch: List[torch.Tensor] = model.encode(text_batch, device=device) # type: ignore
+            id_query_vector_list += [(id_text_batch[i][0], query_vector_batch[i]) for i in range(len(id_text_batch))]
+        return [(id_query_vector[0], id_query_vector[1].tolist()) for id_query_vector in id_query_vector_list]
+        
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/knn_tools.py b/gts_engine/bagualu/lib/components/knn_tools.py
new file mode 100644
index 0000000..06314a7
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/knn_tools.py
@@ -0,0 +1,324 @@
+#encoding=utf8
+
+import os
+import json
+import itertools
+import torch
+import numpy as np
+from tqdm import tqdm
+from sklearn.metrics import pairwise_distances, accuracy_score
+
+class PrototypeEnhancer(object):
+    def __init__(self):
+        pass
+
+    def enhance(self, label_embeds_dict):
+        label_enhanced_embeds_dict = {}
+        enhance_num = 0
+        for label, label_embeds in label_embeds_dict.items():
+            label_embeds = np.vstack(label_embeds)
+            embed_size = label_embeds.shape[0]
+            label_enhanced_embeds_dict[label] = [np.mean(label_embeds, axis=0)]
+            enhance_num += 1
+            for c_embeds in itertools.combinations(label_embeds, embed_size - 1):
+                if len(c_embeds) <= 0:
+                    continue
+                c_embeds = np.vstack(c_embeds)
+                label_enhanced_embeds_dict[label].append(np.mean(c_embeds, axis=0))
+                enhance_num += 1
+        print("enhance num:", enhance_num)
+        return label_enhanced_embeds_dict
+
+class WhiteningTransformer(object):
+    def __init__(self):
+        self.kernel = None
+        self.bias = None
+
+    def preprocess(self, embeds, n_components=64):
+        """计算whitening的kernel和bias
+        vecs.shape = [num_samples, embedding_size]，
+        最后的变换：y = (x + bias).dot(kernel)
+        """
+        mu = embeds.mean(axis=0, keepdims=True)
+        cov = np.cov(embeds.T)
+        u, s, vh = np.linalg.svd(cov)
+        W = np.dot(u, np.diag(1 / np.sqrt(s)))
+        if n_components > 0:
+            W = W[:, :n_components]
+        self.kernel = W
+        self.bias = -mu
+        return self.kernel ,self.bias
+
+    def transform_raw(self, embeds):
+        embeds = (embeds + self.bias).dot(self.kernel)
+        norms = (embeds**2).sum(axis=1, keepdims=True)**0.5
+        embeds = embeds / np.clip(norms, 1e-8, np.inf)
+        return embeds
+
+    def transform(self, embeds, kernel, bias):
+        embeds = torch.matmul(embeds + bias, kernel)  # type: ignore
+        norms = (embeds**2).sum(dim=1, keepdims=True)**0.5  # type: ignore
+        embeds = embeds / torch.clamp(norms, 1e-8, np.inf)
+        return embeds
+
+class BasicDatastore(object):
+    def __init__(self, n_components=64):
+        self.embeds = None
+        self.probs = None
+        self.dataset_name = None
+        self.dataset_path = None
+        self.embed_transformer = WhiteningTransformer()
+        self.data_enhancer = PrototypeEnhancer()
+        self.n_components = n_components
+        self.kernel = None
+        self.bias = None
+    
+    def get_embed_transformer(self):
+        return self.embed_transformer
+
+    def get_datastore_embed(self):
+        return self.embeds
+
+    def get_datastore_prob(self):
+        return self.probs
+
+    def get_dataset_name(self):
+        return self.dataset_name
+
+    def get_dataset_path(self):
+        return self.dataset_path
+
+    def get_kernel(self):
+        return self.kernel
+
+    def get_bias(self):
+        return self.bias
+
+    def transform_embed(self, embeds):
+        return self.embed_transformer.transform(embeds, self.kernel, self.bias)
+    
+    def transform_embed_raw(self, embeds):
+        return self.embed_transformer.transform_raw(embeds)
+
+    def construct(self, classify_model, dataset, dataset_name="labeled"):
+        """ 建立需要索引的datastore
+        """
+        classify_model.eval()
+        embeds = []
+        probs = []
+        device = None
+        label_embeds_dict = {} # 每个label下所有样本的向量表示
+        for batch in dataset:
+            labels = batch['label_id_clf']
+            inference_output = classify_model(input_ids=batch['input_ids'],
+                    input_mask=batch['input_mask'],
+                    input_seg=batch['input_seg'],
+                    prompt_gate=False)
+            device = inference_output['positions'].device
+            sample_embeds = np.array(inference_output['embeds'].detach().cpu())
+            inner_probs = np.array(inference_output['probs'].detach().cpu())
+            if dataset_name != "unlabeled":
+                hard_probs = np.eye(inner_probs.shape[1])[labels]
+                batch_probs = hard_probs
+                probs.append(batch_probs)
+            else:
+                probs.append(inner_probs)
+            embeds.append(sample_embeds)
+            if dataset_name != "unlabeled":
+                for label, embed in zip(labels, sample_embeds):
+                    if label not in label_embeds_dict:
+                        label_embeds_dict[label] = []
+                    label_embeds_dict[label].append(embed)   
+        # 构建句子向量变换器
+        kernel ,bias = self.embed_transformer.preprocess(np.vstack(embeds), self.n_components)
+        # 在样本空间中采样增强datastore
+        label_enhanced_embeds_dict = {}
+        if self.data_enhancer:
+            label_enhanced_embeds_dict = self.data_enhancer.enhance(label_embeds_dict)
+        for label, enhanced_embeds in label_enhanced_embeds_dict.items():
+            enhanced_embeds = np.vstack(enhanced_embeds)
+            embeds.append(enhanced_embeds)
+            indices = label.detach().cpu().tolist()* enhanced_embeds.shape[0]
+            probs.append(np.eye(inner_probs.shape[1])[indices])  # type: ignore
+        embeds = np.vstack(embeds)
+        probs = np.vstack(probs)
+
+        self.kernel = torch.tensor(kernel).float().to(device=device)
+        self.bias = torch.tensor(bias).float().to(device=device)
+        embeds = self.embed_transformer.transform(torch.tensor(embeds).float().to(device=device), self.kernel, self.bias)
+        self.embeds = embeds
+        self.probs = torch.tensor(probs).float().to(device=device)
+        self.dataset_name = dataset_name
+        
+        datastore_dict = {
+            "embeds": self.embeds,
+            "probs": self.probs,
+            "dataset_name": self.dataset_name,
+            "kernel": self.kernel,
+            "bias": self.bias,
+            "embed_transformer": self.embed_transformer,
+        }
+
+        print("datastore[%s] embed size:" % dataset_name, embeds.shape)
+        return datastore_dict
+
+def get_datastore(model, data_model):
+    """ 加载or计算索引 """
+    
+    datastore = BasicDatastore()
+    datastore.construct(
+        classify_model=model,
+        dataset=data_model  # type: ignore
+    )
+    return datastore
+
+def softmax_2D(x):
+    """Compute the softmax in a numerically stable way."""
+    x = x - np.max(x, axis=1, keepdims=True)
+    exp_x = np.exp(x)
+    softmax_x = exp_x / np.sum(exp_x, axis=1, keepdims=True)
+    return softmax_x
+
+def get_knn_probs_raw(distances, datastore_probs, k, remove_self=False):
+    """numpy算子，计算knn的概率
+
+    Args:
+        distances: pair-wise距离矩阵
+        datastore_probs: 索引对应的样本概率分布
+        k: knn的k值
+        remove_self (bool, optional): 计算knn时是否移除自身，当query样本和datastore样本来源相同时，需要设置为True. Defaults to False.
+    """
+    nearest_indices = np.argpartition(distances, kth=k, axis=1)[:, :k] # 最近的k个下标
+    nearest_distances = np.take_along_axis(distances, nearest_indices, axis=1) # 最近的k个距离
+    if remove_self:
+        valid_distances = (nearest_distances >= 1e-5)
+        # 有可能存在多个距离等于0的情况，只保留一个
+        invalid_row_indices = np.where(np.count_nonzero(valid_distances, axis=1) != k-1)[0] # 存在问题的行
+        for invalid_row_index in invalid_row_indices:
+            invalid_col_indices = np.where(valid_distances[invalid_row_index] == False)[0] # 存在问题的列
+            for invalid_col_index in invalid_col_indices[1:]:
+                valid_distances[invalid_row_index][invalid_col_index] = True # 只保留一个
+        nearest_indices = nearest_indices[valid_distances].reshape(distances.shape[0], k-1)
+        nearest_distances = nearest_distances[valid_distances].reshape(distances.shape[0], k-1)
+    nearest_distances = softmax_2D(-1.0 * nearest_distances) # norm distance
+    # nearest_distances = 1.0 - softmax_2D(nearest_distances)
+    nearest_prob = np.take(datastore_probs, nearest_indices, axis=0)
+    nearest_prob = np.transpose(nearest_prob, (0, 2, 1))
+    nearest_distances = np.expand_dims(nearest_distances, axis=2)
+    nearest_distances = np.repeat(nearest_distances, datastore_probs.shape[-1], axis=2)
+    nearest_distances = np.transpose(nearest_distances, (0, 2, 1))
+    nearest_prob = nearest_prob * nearest_distances
+    nearest_prob = np.sum(nearest_prob, axis=2)
+    return nearest_prob
+
+def get_knn_probs(distances, datastore_probs, k):
+    """torch算子，计算knn的概率
+
+    Args:
+        distances: pair-wise距离矩阵
+        datastore_probs: 索引对应的样本概率分布
+        k: knn的k值
+    """
+    nearest_indices = torch.topk(distances, k=k, dim=1, largest=False).indices # 最近的k个下标
+    nearest_distances = torch.gather(distances, 1, nearest_indices) # 最近的k个距离
+    m = torch.nn.Softmax(1)
+    nearest_distances = m(-1.0*nearest_distances)
+    nearest_prob = torch.gather(datastore_probs.unsqueeze(0).repeat(nearest_indices.shape[0],1,1), 1, \
+                                        nearest_indices.unsqueeze(-1).repeat(1,1,datastore_probs.shape[-1]))
+    nearest_prob = nearest_prob.permute(0,2,1)
+    nearest_distances = nearest_distances.unsqueeze(2).repeat(1,1,datastore_probs.shape[-1]).permute(0,2,1)
+    nearest_prob = nearest_prob * nearest_distances
+    nearest_prob = nearest_prob.sum(2)
+    return nearest_prob
+
+def grid_search_for_hyper(datastores, dev_dataloader, classify_model):
+    classify_model.eval()
+    datastores= [datastores]
+    """ 加载验证数据 """
+    y_true = []
+    classify_probs = []
+    sample_embeds = []
+    for batch in dev_dataloader:
+        labels = batch['label_id_clf']
+        inference_output = classify_model(input_ids=batch['input_ids'],
+                input_mask=batch['input_mask'],
+                input_seg=batch['input_seg'],
+                prompt_gate=False)
+        embeds = np.array(inference_output['embeds'].detach().cpu())
+        inner_probs = np.array(inference_output['probs'].detach().cpu())
+        y_true.extend(labels)
+        classify_probs.append(inner_probs)
+        sample_embeds.append(embeds)
+    sample_embeds = np.vstack(sample_embeds)
+    classify_probs = np.vstack(classify_probs)
+
+    """ 超参搜索 """
+    best_acc = 0.0
+    best_hyper = {"lambda_values": [], "k_values": []}
+    best_y_pred = []
+    
+    # compute distances
+    distance_cache = []
+    for datastore in datastores:
+        query_embeds = datastore.transform_embed_raw(sample_embeds)
+        datastore_embeds = datastore.get_datastore_embed()
+        distances = pairwise_distances(query_embeds, datastore_embeds.detach().cpu().numpy(), metric="euclidean", n_jobs=1)
+        distance_cache.append(distances)
+
+    # search for best hyper parameters
+    knn_cache = []
+    for index in range(len(datastores)):
+        knn_cache.append({})
+    k_search_list = [np.arange(0, 32, 1)]
+    lambda_search_list = [np.linspace(0, 1, 21)]
+    for hyper_tuple in itertools.product(*lambda_search_list, *k_search_list):
+        lambda_values = list(map(lambda x: round(x, 2), hyper_tuple[:len(lambda_search_list)]))
+        k_values = list(hyper_tuple[len(lambda_search_list):])
+        y_pred = []
+        if sum(lambda_values) > 1:
+            continue
+        final_prob = (1 - sum(lambda_values)) * classify_probs
+        for index in range(len(datastores)):
+            datastore = datastores[index]
+            if k_values[index] == 0 or lambda_values[index] <= 1e-4:
+                continue
+            if k_values[index] in knn_cache[index]:
+                knn_probs = knn_cache[index][k_values[index]]
+            else:
+                datastore_probs = datastore.get_datastore_prob()
+                knn_probs = get_knn_probs_raw(distance_cache[index], datastore_probs.detach().cpu().numpy(), k_values[index])
+                knn_cache[index][k_values[index]] = knn_probs
+            final_prob = final_prob + lambda_values[index] * knn_probs
+        y_pred = list(np.argmax(final_prob, axis=1))
+        current_acc = accuracy_score(y_true, y_pred)
+        if current_acc > best_acc:
+            best_acc = current_acc
+            best_y_pred = y_pred
+            best_hyper = {"lambda_values": lambda_values[0], "k_values": k_values[0]}
+            print("best acc:", round(best_acc * 100, 2), "\tbest hyper:", best_hyper)
+
+    return best_hyper, y_true, best_y_pred
+    
+def pdist(a: torch.Tensor, b: torch.Tensor, p: int = 2) -> torch.Tensor:
+    a=a.unsqueeze(1)
+    b=b.unsqueeze(0)
+    return (a-b).abs().pow(p).sum(-1).pow(1/p)
+
+def inference_with_knn(datastore, classify_probs, sample_embeds, best_hyper, whitening=True):
+    datastore_embeds = datastore.get_datastore_embed()
+    sample_embeds = sample_embeds.to(datastore_embeds.device)
+    classify_probs = classify_probs.to(datastore_embeds.device)
+    # embedding预处理
+    query_embeds = datastore.transform_embed(sample_embeds)
+    # 先算好distances
+    distances = pdist(query_embeds, datastore_embeds)
+    lambda_values = best_hyper["lambda_values"]
+    k_values = best_hyper["k_values"]
+    if k_values == 0 or lambda_values <= 1e-4:
+        return classify_probs
+    else:
+        final_prob = (1 - lambda_values) * classify_probs
+        datastore_probs = datastore.get_datastore_prob()
+        knn_probs = get_knn_probs(distances, datastore_probs, k_values)
+        final_prob = final_prob + lambda_values * knn_probs
+    return final_prob
diff --git a/gts_engine/bagualu/lib/components/lightning_callbacks/__init__.py b/gts_engine/bagualu/lib/components/lightning_callbacks/__init__.py
new file mode 100644
index 0000000..c8ae0b1
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/lightning_callbacks/__init__.py
@@ -0,0 +1,1 @@
+from .lightning_progress_bar import LitProgressBar
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/lightning_callbacks/adaptive_val_intervals.py b/gts_engine/bagualu/lib/components/lightning_callbacks/adaptive_val_intervals.py
new file mode 100644
index 0000000..5a2af0b
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/lightning_callbacks/adaptive_val_intervals.py
@@ -0,0 +1,182 @@
+from typing import Dict, Optional, List
+from enum import Enum
+import pytorch_lightning as pl
+import numpy as np
+from pytorch_lightning.trainer.states import TrainerFn
+from torch import Tensor
+from math import ceil
+from pytorch_lightning.callbacks import Callback
+from abc import ABCMeta, abstractmethod
+
+from ...utils.statistics import ExponentialSmoothingList, DynamicMax, interval_mean
+from ...framework.mixin import OptionalLoggerMixin
+
+#############################################################################################
+## Base
+#############################################################################################
+
+class BaseAdaptiveValidationInterval(Callback, OptionalLoggerMixin, metaclass=ABCMeta):
+    
+    def __init__(self, logger_name: Optional[str] = None):
+        Callback.__init__(self)
+        OptionalLoggerMixin.__init__(self, logger_name)
+        
+    def setup(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule", stage: Optional[str] = None) -> None:
+        if not trainer.state.fn == TrainerFn.FITTING:
+            return
+        self._disable_validation(trainer)  
+    
+    def on_batch_start(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
+        if not trainer.state.fn == TrainerFn.FITTING:
+            return
+        if self._get_validation_flag(trainer, pl_module):
+            self.info("trigger validation")
+            self._enable_validation(trainer)
+    
+    def on_validation_epoch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
+        if not trainer.state.fn == TrainerFn.FITTING:
+            return
+        self._disable_validation(trainer)
+    
+    def _disable_validation(self, trainer: "pl.Trainer"):
+        """使loop不进行验证"""
+        trainer.fit_loop.epoch_loop._should_check_val_fx = lambda : False   
+    
+    def _enable_validation(self, trainer: "pl.Trainer"):
+        """使loop进行验证"""
+        trainer.fit_loop.epoch_loop._should_check_val_fx = lambda : True 
+        
+    @abstractmethod
+    def _get_validation_flag(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> bool: ...
+
+#############################################################################################
+## Subs
+#############################################################################################
+
+class ADAPTIVE_VAL_INTERVAL_MODE(Enum):
+    ADAPTIVE = 1
+    FIXED = 2
+
+
+class AdaptiveValIntervalFixed(BaseAdaptiveValidationInterval):
+    
+    def __init__(self, train_sample_num: int, batch_size: int, logger_name: Optional[str] = None):
+        super().__init__(logger_name)
+        batch_num_per_epoch = train_sample_num // batch_size
+        val_times_per_epoch = 7
+        self.__val_interval = max(1, batch_num_per_epoch // val_times_per_epoch)
+    
+    def _get_validation_flag(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> bool:
+        return trainer.global_step % self.__val_interval == 0
+    
+    
+class AdaptiveValIntervalDevLoss(BaseAdaptiveValidationInterval):
+    
+    def __init__(self, train_sample_num: int, batch_size: int, logger_name: Optional[str] = None):
+        super().__init__(logger_name)
+        self.__batch_size = batch_size
+        self.__train_sample_num = train_sample_num
+        self.__current_dev_times_per_epoch = 1
+        self.__min_dev_times_per_epoch = 3
+        self.__max_dev_times_per_epoch = 7
+        self.__batch_num_each_epoch = self.__train_sample_num // self.__batch_size
+        self.__val_interval = max(int(self.__batch_num_each_epoch // self.__current_dev_times_per_epoch), 1)
+        self.__next_validation = self.__val_interval
+        self.__dev_loss_list: List[float] = []
+        
+    def _get_validation_flag(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> bool:
+        return trainer.global_step == self.__next_validation
+    
+    def __step(self, loss: float) -> None:
+        self.__dev_loss_list.append(loss)
+        if self.__loss_rising_flag():
+            self.__current_dev_times_per_epoch -= 2
+            self.__current_dev_times_per_epoch = max(self.__current_dev_times_per_epoch, self.__min_dev_times_per_epoch)
+        else:
+            self.__current_dev_times_per_epoch += 2
+            self.__current_dev_times_per_epoch = min(self.__current_dev_times_per_epoch, self.__max_dev_times_per_epoch)
+        self.__val_interval = max(int(self.__batch_num_each_epoch // self.__current_dev_times_per_epoch), 5)
+    
+    def __loss_rising_flag(self, window=2, pre_sample_len=4):
+        """
+        判断当前loss是否处于上涨状态
+        """
+        dev_sample_start = len(self.__dev_loss_list) - window if (len(self.__dev_loss_list) - window) >= 0 else 0
+        pre_sample_start = len(self.__dev_loss_list) - window - pre_sample_len if (len(self.__dev_loss_list) - window - pre_sample_len) >= 0 else 0
+        pre_sample_end = pre_sample_start + window if pre_sample_start + window <= len(self.__dev_loss_list) else len(self.__dev_loss_list)
+        pre_step_loss_avg = np.mean(self.__dev_loss_list[pre_sample_start:pre_sample_end])
+        dev_step_loss_avg = np.mean(self.__dev_loss_list[dev_sample_start:])
+        return True if pre_step_loss_avg - dev_step_loss_avg < 0.00 else False
+    
+    def on_validation_epoch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
+        if trainer.state.fn != TrainerFn.FITTING:
+            return
+        super().on_validation_epoch_end(trainer, pl_module)
+        dev_loss = float(trainer.logged_metrics["dev_loss"].item())
+        self.__step(dev_loss)
+        self.__next_validation = trainer.global_step + self.__val_interval
+        self.info(f"current validation interval: {self.__val_interval}, next validation will be at step {self.__next_validation}")
+
+
+class AdaptiveValIntervalTrainLoss(BaseAdaptiveValidationInterval):
+    
+    def __init__(self, train_sample_num: int, batch_size: int, logger_name: Optional[str] = None):
+        super().__init__(logger_name)
+        self.__dev_gate: bool = False
+        self.__batch_num_per_epoch = train_sample_num // batch_size
+        val_times_per_epoch = 7
+        self.__val_interval = max(self.__batch_num_per_epoch // val_times_per_epoch, 1)
+        self.__train_loss_smooth_list = ExponentialSmoothingList(level=2, alpha=0.9, warmup_steps=10)
+        self.__diff_smooth_list = ExponentialSmoothingList(level=2, alpha=0.9, warmup_steps=10)
+        self.__diff_ratio_max = DynamicMax(top_n=3)
+        self.__sample_gap = max(min(15 // max(int((batch_size // 8) ** 0.5), 1), self.__batch_num_per_epoch // 8), 1)
+        self.__trigger_count = 0 # 连续触发次数
+        self.__trigger_count_thres = 2 # 连续触发次数阈值
+        self.__ratio_thres = 0.8 # 比例阈值 # TODO: 根据batch_size放缩，需要更多实验支撑
+        self.__warmup_steps = self.__batch_num_per_epoch // 2
+                
+    def _get_validation_flag(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> bool:
+        return self.__dev_gate and trainer.global_step % self.__val_interval == 0
+    
+    def on_train_batch_end(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule", outputs: Dict[str, Tensor], batch, batch_idx: int) -> None:
+        if trainer.state.fn != TrainerFn.FITTING:
+            return
+        if self.__dev_gate:
+            return
+        train_loss: float = outputs["loss"].item()
+        if trainer.global_step % self.__sample_gap == 0:
+            self.__step(train_loss, trainer, pl_module)
+        
+    def __step(self, train_loss: float, trainer: "pl.Trainer", pl_module: "pl.LightningModule") -> None:
+        if trainer.current_epoch >= trainer.max_epochs - 3:
+            self.info(f"start to validate every {self.__val_interval} steps")
+            self.__dev_gate = True
+            return
+        self.__train_loss_smooth_list.append(train_loss)
+        if len(self.__train_loss_smooth_list) <= 2:
+            return
+        diff = self.__train_loss_smooth_list[2][-1] - self.__train_loss_smooth_list[2][-2]
+        self.__diff_smooth_list.append(diff)
+        if trainer.global_step > self.__warmup_steps:
+            if self.__trigger(trainer, pl_module):
+                self.__trigger_count += 1
+            else:
+                self.__trigger_count = 0
+            if self.__trigger_count >= self.__trigger_count_thres:
+                self.__dev_gate = True
+                self.info(f"start to validate every {self.__val_interval} steps")
+    
+    def __trigger(self, trainer: "pl.Trainer", pl_module: "pl.LightningModule"):
+        diff_ratio = self.__diff_smooth_list[1][-1] / self.__warm_up_mean_diff
+        self.__diff_ratio_max.step(diff_ratio)
+        normalized_diff_ratio = diff_ratio / self.__diff_ratio_max.max
+        pl_module.log(f"diff_ratio", diff_ratio)
+        pl_module.log(f"normalized_diff_ratio", normalized_diff_ratio)
+        return normalized_diff_ratio < self.__ratio_thres
+    
+    __warm_up_mean_diff_: Optional[float] = None
+    @property
+    def __warm_up_mean_diff(self) -> float:
+        if self.__warm_up_mean_diff_ is None:
+            self.__warm_up_mean_diff_ = interval_mean(self.__diff_smooth_list[1][:self.__warmup_steps])
+        return self.__warm_up_mean_diff_
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/lightning_callbacks/lightning_progress_bar.py b/gts_engine/bagualu/lib/components/lightning_callbacks/lightning_progress_bar.py
new file mode 100644
index 0000000..8468516
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/lightning_callbacks/lightning_progress_bar.py
@@ -0,0 +1,32 @@
+from pytorch_lightning.callbacks.progress.base import ProgressBarBase
+from typing import Union
+from logging import Logger
+
+class LitProgressBar(ProgressBarBase):
+    def __init__(self, logger: Logger):
+        super().__init__()  # don't forget this :)
+        self.enable = True # type: ignore
+        self._logger = logger
+
+    def disable(self):
+        self.enable = False # type: ignore
+
+    def on_train_batch_end(self, trainer, pl_module, outputs, batch, batch_idx):
+        super().on_train_batch_end(trainer, pl_module, outputs, batch, batch_idx)  # don't forget this :)
+        metrics = self.get_metrics(trainer, pl_module)
+        metrics_str_list = [self._get_format(name, val) for name, val in metrics.items()]
+        metrics_str_list.append(f"batch {batch_idx}/{self.total_batches_current_epoch}")
+        metrics_str = " ".join(metrics_str_list)
+        self._logger.info(metrics_str)
+        
+    
+    def _get_format(self, name: str, val: Union[float, int, str]) -> str:
+        if name in ("epoch", "step"):
+            f = f"{name} {int(val)}"
+        elif name in ("lr"):
+            f = f"{name} {val:.4e}"
+        elif isinstance(val, float):
+            f = f"{name} {val:.4f}"
+        else:
+            f = f"{name} {val}"
+        return f
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/losses/__init__.py b/gts_engine/bagualu/lib/components/losses/__init__.py
new file mode 100644
index 0000000..cc7229a
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/losses/__init__.py
@@ -0,0 +1,4 @@
+from .max_multi_logits import MaxMultiLogits
+from .label_smoothing import LabelSmoothing
+from .rdrop_kl_loss import compute_kl_loss
+from .ie_losses import NegativeSampleLoss, DiceLoss, DecoupledBCEloss, DecoupledMSEloss, DistillSelfLoss
diff --git a/gts_engine/bagualu/lib/components/losses/ie_losses.py b/gts_engine/bagualu/lib/components/losses/ie_losses.py
new file mode 100644
index 0000000..4493895
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/losses/ie_losses.py
@@ -0,0 +1,314 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# pylint: disable=no-member
+
+from typing import Optional
+import torch
+from torch import Tensor
+
+
+class NegativeSampleLoss(torch.nn.Module):
+    """ 负例采样损失 """
+    def __init__(self, p=0.4):
+        super().__init__()
+        self.bce_loss = torch.nn.BCELoss(reduction='none')
+        self.dropout = torch.nn.Dropout(p=p)
+
+    def forward(self,
+                logits: Tensor,
+                labels: Tensor,
+                mask: Tensor = None) -> Tensor:
+        """ forward """
+        loss = self.bce_loss(logits, labels)
+        loss_pos = loss * labels.float()
+        loss_neg = loss * (1.0 - labels.float())
+        loss_neg = self.dropout(loss_neg)
+        loss = loss_pos + loss_neg
+        if mask is not None:
+            assert mask.shape == loss.shape
+            loss = loss * mask
+            return torch.sum(loss) / torch.sum(mask)
+        mean_loss = torch.mean(loss)
+        return mean_loss
+
+
+class DiceLoss(torch.nn.Module):
+    """
+    REF https://github.com/ShannonAI/dice_loss_for_NLP/blob/master/loss/dice_loss.py
+    Dice coefficient for short, is an F1-oriented statistic used
+    to gauge the similarity of two sets.
+    Given two sets A and B, the vanilla dice coefficient between them is given as follows:
+        Dice(A, B)  = 2 * True_Positive / (2 * True_Positive + False_Positive + False_Negative)
+                    = 2 * |A and B| / (|A| + |B|)
+    Args:
+        smooth (float, optional): a manual smooth value for numerator and denominator.
+        square_denominator (bool, optional):
+            [True, False], specifies whether to square the denominator in the loss function.
+        with_logits (bool, optional):
+            [True, False], specifies whether the input tensor is normalized
+            by Sigmoid/Softmax funcs.
+        ohem_ratio: max ratio of positive/negative, defautls to 0.0, which means no ohem.
+        alpha: dsc alpha
+    Shape:
+        - input: (*)
+        - target: (*)
+        - mask: (*) 0,1 mask for the input sequence.
+        - Output: Scalar loss
+    """
+    def __init__(self,
+                 smooth: Optional[float] = 1,
+                 square_denominator: Optional[bool] = True,
+                 with_logits: Optional[bool] = False,
+                 ohem_ratio: float = 1.,
+                 alpha: float = 0.01,
+                 reduction: Optional[str] = "mean") -> None:
+        super(DiceLoss, self).__init__()
+
+        self.reduction = reduction
+        self.with_logits = with_logits
+        self.smooth = smooth
+        self.square_denominator = square_denominator
+        self.ohem_ratio = ohem_ratio
+        self.alpha = alpha
+
+    def forward(self, inputs: Tensor, target: Tensor, mask: Optional[Tensor] = None) -> Tensor:
+        """ forward """
+        loss = self._binary_class(inputs, target, mask=mask)
+        if self.reduction == "mean":
+            return loss.mean()
+        if self.reduction == "sum":
+            return loss.sum()
+        return loss
+
+    def _compute_dice_loss(self, flat_input, flat_target):
+        # flat_input = ((1 - flat_input) ** self.alpha) * flat_input
+        interection = torch.sum(flat_input * flat_target, -1)
+        if not self.square_denominator:
+            denominator = flat_input.sum() + flat_target.sum() + self.smooth
+        else:
+            flat_input = torch.square(flat_input)
+            flat_target = torch.square(flat_target)
+            denominator = torch.sum(flat_input, -1) + torch.sum(flat_target, -1) + self.smooth
+
+        loss = 1 - ((2 * interection + self.smooth) / denominator)
+
+        return loss
+
+    def _binary_class(self, inputs, target, mask=None):
+        # flat_input = inputs.view(-1)
+        # flat_target = target.view(-1).float()
+        # mask = mask.view(-1).float()
+        flat_input = inputs.reshape(-1)
+        flat_target = target.reshape(-1).float()
+        mask = mask.reshape(-1).float()
+        flat_input = torch.sigmoid(flat_input) if self.with_logits else flat_input
+
+        if mask is not None:
+            mask = mask.float()
+            flat_input = flat_input * mask
+            flat_target = flat_target * mask
+        else:
+            mask = torch.ones_like(target)
+
+        if self.ohem_ratio > 0:
+            pos_example = target > 0.5
+            neg_example = target <= 0.5
+            mask_neg_num = mask <= 0.5
+
+            pos_num = pos_example.sum() - (pos_example & mask_neg_num).sum()
+            neg_num = neg_example.sum()
+            keep_num = min(int(pos_num * self.ohem_ratio), neg_num) + 1
+
+            neg_scores = torch.masked_select(flat_input, neg_example.bool())
+            neg_scores_sort, _ = torch.sort(neg_scores, )
+            threshold = neg_scores_sort[-keep_num+1]
+            cond = (flat_input >= threshold) | pos_example.view(-1)
+            ohem_mask = torch.where(cond, 1, 0)
+
+            flat_input = flat_input * ohem_mask
+            flat_target = flat_target * ohem_mask
+
+        return self._compute_dice_loss(flat_input, flat_target)
+
+
+class DecoupledBCEloss(torch.nn.Module):
+    """ Decoupled BCE Loss
+
+    Args:
+        threshold (float, optional): threshold. Defaults to 0.1
+        alpha (float, optional): alpha for positive loss. Defaults to 1.
+        beta (float, optional): beta for negative loss. Defaults to 100.
+    """
+    def __init__(self,
+                 threshold: float = 0.1,
+                 alpha: float = 1.,
+                 beta: float = 100.) -> None:
+        super().__init__()
+
+        self.threshold = threshold
+        self.alpha = alpha
+        self.beta = beta
+
+        self._bce_sum = torch.nn.BCELoss(reduction="sum")
+
+    def forward(self,
+                source: Tensor,
+                target: Tensor,
+                mask: Tensor = None) -> Tensor:
+        """ forward
+
+        Args:
+            source (Tensor): source
+            target (Tensor): target
+            mask (Tensor): mask. Defaults to None.
+
+        Returns:
+            Tensor: loss
+        """
+        # mask
+        if mask is None:
+            mask = torch.ones_like(target)
+        else:
+            source = source * mask
+            target = target * mask
+
+        ones = torch.ones_like(source)
+        zeros = torch.zeros_like(source)
+
+        # source和target中超过阈值的位置
+        source_pos = torch.where(source < self.threshold, zeros, ones) * mask
+        target_pos = torch.where(target < self.threshold, zeros, ones) * mask
+
+        # source和target中任一超过阈值(pos)和全部低于阈值(neg)的位置
+        mask_pos = torch.where(source_pos + target_pos > 0, ones, zeros) * mask
+        mask_neg = torch.where(source_pos + target_pos <= 0, ones, zeros) * mask
+
+        # pos和neg的数量
+        num_pos = mask_pos.sum()
+        num_neg = mask_neg.sum()
+
+        # pos部分的loss
+        if num_pos <= 0: # 没有pos
+            loss_pos = 0.
+        else:
+            loss_pos = self._bce_sum(source * mask_pos, target * mask_pos) / num_pos
+
+        # neg部分的loss
+        if num_neg <= 0: # 没有neg
+            loss_neg = 0.
+        else:
+            loss_neg = self._bce_sum(source * mask_neg, target * mask_neg) / num_neg
+
+        loss = self.alpha * loss_pos + self.beta * loss_neg
+
+        return loss
+
+
+class DecoupledMSEloss(torch.nn.Module):
+    """ Decoupled MSE Loss
+
+    Args:
+        threshold (float, optional): threshold. Defaults to 0.1
+        alpha (float, optional): alpha for positive loss. Defaults to 1.
+        beta (float, optional): beta for negative loss. Defaults to 100.
+    """
+    def __init__(self,
+                 threshold: float = 0.1,
+                 alpha: float = 1.,
+                 beta: float = 100.) -> None:
+        super().__init__()
+
+        self.threshold = threshold
+        self.alpha = alpha
+        self.beta = beta
+
+        self._mse_sum = torch.nn.MSELoss(reduction="sum")
+
+    def forward(self,
+                source: Tensor,
+                target: Tensor,
+                mask: Tensor = None) -> Tensor:
+        """ forward
+
+        Args:
+            source (Tensor): source
+            target (Tensor): target
+            mask (Tensor): mask. Defaults to None.
+
+        Returns:
+            Tensor: loss
+        """
+        # mask
+        if mask is not None:
+            source = source * mask
+            target = target * mask
+
+        ones = torch.ones_like(source)
+        zeros = torch.zeros_like(source)
+
+        # source和target中超过阈值的位置
+        source_pos = torch.where(source < self.threshold, zeros, ones) * mask
+        target_pos = torch.where(target < self.threshold, zeros, ones) * mask
+
+        # source和target中任一超过阈值(pos)和全部低于阈值(neg)的位置
+        mask_pos = torch.where(source_pos + target_pos > 0, ones, zeros) * mask
+        mask_neg = torch.where(source_pos + target_pos <= 0, ones, zeros) * mask
+
+        # pos和neg的数量
+        num_pos = mask_pos.sum()
+        num_neg = mask_neg.sum()
+
+        # pos部分的loss
+        if num_pos <= 0: # 没有pos
+            loss_pos = 0.
+        else:
+            loss_pos = self._mse_sum(source * mask_pos, target * mask_pos) / num_pos
+
+        # neg部分的loss
+        if num_neg <= 0: # 没有neg
+            loss_neg = 0.
+        else:
+            loss_neg = self._mse_sum(source * mask_neg, target * mask_neg) / num_neg
+
+        loss = self.alpha * loss_pos + self.beta * loss_neg
+
+        return loss
+
+
+class DistillSelfLoss(torch.nn.Module):
+    """ Distill Self Loss """
+    def __init__(self):
+        super().__init__()
+        self.d_mse = DecoupledMSEloss(threshold=0.1,
+                                    alpha=1.,
+                                    beta=100.)
+
+    def forward(self,
+                s_logits: Tensor,
+                t_logits: Tensor,
+                mask: Tensor) -> Tensor:
+        """ forward
+
+        Args:
+            s_logits (Tensor): source logits, (bsz, seq, seq, label)
+            t_logits (Tensor): target logits, (bsz, seq, seq, label)
+            mask (Tensor): mask, (bsz, seq, seq)
+
+        Returns:
+            Tensor: loss
+        """
+        loss = self.d_mse(s_logits, t_logits, mask)
+        return loss
diff --git a/gts_engine/bagualu/lib/components/losses/label_smoothing.py b/gts_engine/bagualu/lib/components/losses/label_smoothing.py
new file mode 100644
index 0000000..53fd0a7
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/losses/label_smoothing.py
@@ -0,0 +1,25 @@
+import torch
+
+class LabelSmoothing(torch.nn.Module):
+    """
+    NLL loss with label smoothing.
+    """
+
+    def __init__(self, smoothing: float = 0.1):
+        """
+        Constructor for the LabelSmoothing module.
+        :param smoothing: label smoothing factor
+        """
+        super(LabelSmoothing, self).__init__()
+        self.confidence = 1.0 - smoothing
+        self.smoothing = smoothing
+
+    def forward(self, x: torch.Tensor, target: torch.Tensor) -> torch.Tensor:
+        logprobs = torch.nn.functional.log_softmax(x, dim=-1) # type: ignore
+
+        nll_loss = -logprobs.gather(dim=-1, index=target.unsqueeze(1))
+        nll_loss = nll_loss.squeeze(1)
+        smooth_loss = -logprobs.mean(dim=-1)
+        loss = self.confidence * nll_loss + self.smoothing * smooth_loss
+        
+        return loss.mean()
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/losses/max_multi_logits.py b/gts_engine/bagualu/lib/components/losses/max_multi_logits.py
new file mode 100644
index 0000000..7ad872a
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/losses/max_multi_logits.py
@@ -0,0 +1,22 @@
+import torch
+
+class MaxMultiLogits(torch.nn.Module):
+    
+    def __init__(self, class_num: int, hidden_size: int, multiply: int = 1):
+        super().__init__()
+        
+        self.inf = -1e7
+        self.class_num = class_num
+        self.hidden_size = hidden_size
+        self.multiply = multiply
+
+        multi_classes = self.class_num * self.multiply
+        self.linear = torch.nn.Linear(in_features=hidden_size,
+                                      out_features=multi_classes)
+        self.softmax = torch.nn.Softmax(dim=-1)
+
+    def forward(self, input_hidden_states: torch.Tensor) -> torch.Tensor:
+        cls_input = input_hidden_states[:, 0, :]
+        logits: torch.Tensor = self.linear(cls_input).view(-1, self.class_num, self.multiply)
+        max_logits, _ = torch.max(logits, dim=-1)
+        return max_logits
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/losses/rdrop_kl_loss.py b/gts_engine/bagualu/lib/components/losses/rdrop_kl_loss.py
new file mode 100644
index 0000000..caadcd8
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/losses/rdrop_kl_loss.py
@@ -0,0 +1,19 @@
+import torch.nn.functional as F
+ 
+# 定义R-drop损失：根据两次前向计算的logits，计算kl_loss
+def compute_kl_loss(p, q, pad_mask=None):
+     
+    p_loss = F.kl_div(F.log_softmax(p, dim=-1), F.softmax(q, dim=-1), reduction='none')
+    q_loss = F.kl_div(F.log_softmax(q, dim=-1), F.softmax(p, dim=-1), reduction='none')
+     
+    # pad_mask is for seq-level tasks
+    if pad_mask is not None:
+        p_loss.masked_fill_(pad_mask, 0.)
+        q_loss.masked_fill_(pad_mask, 0.)
+ 
+    # You can choose whether to use function "sum" and "mean" depending on your task
+    p_loss = p_loss.sum()
+    q_loss = q_loss.sum()
+ 
+    loss = (p_loss + q_loss) / 2
+    return loss
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/metrics/__init__.py b/gts_engine/bagualu/lib/components/metrics/__init__.py
new file mode 100644
index 0000000..83d0472
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/metrics/__init__.py
@@ -0,0 +1,1 @@
+from .logits_2_acc import Logits2Acc
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/metrics/clf_evaluation.py b/gts_engine/bagualu/lib/components/metrics/clf_evaluation.py
new file mode 100644
index 0000000..5bfea2e
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/metrics/clf_evaluation.py
@@ -0,0 +1,26 @@
+from ctypes import Union
+from dataclasses import dataclass
+from typing import Any, List, Dict, Tuple, Union, Optional
+from sklearn.metrics import classification_report, confusion_matrix
+
+@dataclass
+class GetConfusionMatrixOutput:
+    confusion_matrix: List
+    labels: List[str]
+
+def get_confusion_matrix(y_true: List[int], y_pred: List[int], id2label: Dict[int, str]) -> GetConfusionMatrixOutput:
+    label_ids = sorted(list(set(y_true + y_pred)))
+    label_names = [id2label[i] for i in label_ids]
+    return GetConfusionMatrixOutput(confusion_matrix(y_true=y_true, y_pred=y_pred, labels=label_ids).tolist(), label_names)
+
+def get_classification_report(y_true: List[int], y_pred: List[int], id2label: Dict[int, str]) -> Dict[str, Any]:
+    label_ids = sorted(list(set(y_true + y_pred)))
+    label_names = [id2label[i] for i in label_ids]
+    return classification_report(
+        y_true,
+        y_pred,
+        target_names=label_names,
+        digits=2,
+        zero_division=0,    # type: ignore
+        output_dict=True
+    ) 
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/metrics/logits_2_acc.py b/gts_engine/bagualu/lib/components/metrics/logits_2_acc.py
new file mode 100644
index 0000000..9f13020
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/metrics/logits_2_acc.py
@@ -0,0 +1,21 @@
+import torch
+from torch import nn, Tensor
+
+
+class Logits2Acc(torch.nn.Module):
+    
+    def __init__(self):
+        super().__init__()
+        self.__softmax = torch.nn.Softmax(dim=-1)
+
+    def forward(self, logits: Tensor, labels: Tensor):
+        prob = self.__softmax(logits)
+        y_pred = torch.argmax(prob, dim=-1)
+
+        y_pred = y_pred.view(size=(-1, ))
+        y_true = labels.view(size=(-1, ))
+
+        corr = torch.eq(y_pred, y_true)
+        acc = torch.sum(corr.float())
+
+        return acc, y_pred
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/protocol_args.py b/gts_engine/bagualu/lib/components/protocol_args.py
new file mode 100644
index 0000000..02a5356
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/protocol_args.py
@@ -0,0 +1,70 @@
+from typing import Literal, Optional
+import os
+from pathlib import Path
+from pydantic import FilePath, DirectoryPath
+
+from ..framework import BaseArguments
+from ..framework.consts import TRAIN_MODE
+from ..utils.path import mk_inexist_dir
+
+class ProtocolArgs(BaseArguments):
+    input_dir: DirectoryPath
+    pretrained_model_root: DirectoryPath
+    output_dir: DirectoryPath
+    train_level: TRAIN_MODE
+    
+    def _add_args(self, parser) -> None:
+        parser.add_argument("--gts_input_path", type=Path, dest="input_dir", help="训练数据集路径", required=True)
+        parser.add_argument("--gts_pretrained_model_path", type=Path, dest="pretrained_model_root", help="预训练模型根目录", required=True)
+        parser.add_argument("--gts_output_dir", type=Path, dest="output_dir", help="输出文件路径", required=True)
+        parser.add_argument("--gts_train_level", type=TRAIN_MODE, choices=TRAIN_MODE, dest="train_level", 
+                            help="运行模式: [0 - default | 1 - student | 2 - gts | 3 - 快速模式]", required=True)
+        
+    def _after_parse(self) -> None:
+        mk_inexist_dir(self.output_dir)
+        
+    @property
+    def _arg_name(self) -> Optional[str]:
+        return "调用协议参数"
+    
+    @property
+    def student_output_dir(self) -> DirectoryPath:
+        return self.output_dir / "student_output"
+    
+    @property
+    def log_dir(self) -> DirectoryPath:
+        return self.output_dir / "logs"
+
+class ProtocolArgsMixin:
+    """使当前参数包含ProtocolArgs并隔离"""
+    
+    protocol_args: ProtocolArgs
+    """通用工程协议"""
+    
+    ################## 隔离通用工程协议接口 ################
+    
+    @property
+    def input_dir(self):
+        return self.protocol_args.input_dir
+    
+    @property
+    def pretrained_model_root(self):
+        return self.protocol_args.pretrained_model_root
+    
+    @property
+    def student_output_dir(self):
+        return self.protocol_args.student_output_dir
+    
+    @property
+    def log_dir(self):
+        return self.protocol_args.log_dir
+    
+    @property
+    def train_level(self):
+        return self.protocol_args.train_level
+    
+    @property
+    def output_dir(self):
+        return self.protocol_args.output_dir
+    
+    
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/sample_tools.py b/gts_engine/bagualu/lib/components/sample_tools.py
new file mode 100644
index 0000000..c114146
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/sample_tools.py
@@ -0,0 +1,72 @@
+from typing import Dict, Sequence, Protocol, List, TypeVar, Callable
+import random
+
+class LabeledSampleProto(Protocol):
+    label: str
+    
+LabeledSample = TypeVar("LabeledSample", bound=LabeledSampleProto)
+
+class SoftLabeledSampleProto(Protocol):
+    soft_label: List[float]
+    
+SoftLabeledSample = TypeVar("SoftLabeledSample", bound=SoftLabeledSampleProto)
+
+def balanced_sampling_by_label(labeled_sample_list: Sequence[LabeledSample], sample_num: int, minimum_num_per_label: int) -> Sequence[LabeledSample]:
+    """按标签平衡抽样
+
+    Args:
+        labeled_sample_list (Sequence[LabeledSampleProto]): 抽样的sample列表，sample需有label字段
+        sample_num (int): 总抽样数量
+        minimum_num_per_label (int): 每个类别最小样本量
+    """
+    if len(labeled_sample_list) == 0:
+        return []
+    sample_list_by_label: Dict[str, List[LabeledSample]] = {}
+    for sample in labeled_sample_list:
+        if sample.label not in sample_list_by_label.keys():
+            sample_list_by_label[sample.label] = [sample]
+        else:
+            sample_list_by_label[sample.label].append(sample)
+    num_per_label = int(sample_num / len(sample_list_by_label))
+    num_per_label = max(num_per_label, minimum_num_per_label)
+    res: List[LabeledSample] = []
+    for sample_list in sample_list_by_label.values():
+        random.shuffle(sample_list)
+        res += sample_list[:num_per_label]
+    return res
+
+def get_prob_threshold(label_classes: int, use_unlabel: bool):
+    '''
+    description: 根据类别确定筛选样本的置信度
+                label_classes(int): 样本的个数
+    return {prob_threshold(float): 置信度阈值}
+    '''    
+    if label_classes <= 2:
+        prob_threshold = 0.95
+    elif label_classes <= 10:
+        prob_threshold = 0.9
+    elif label_classes <= 20:
+        prob_threshold = 0.9
+    elif label_classes <= 40:
+        prob_threshold = 0.75
+    else:
+        prob_threshold = 0.5
+    
+    if not use_unlabel:
+        prob_threshold = prob_threshold - 0.1
+
+    return prob_threshold
+
+def filter_data_by_confidence(sample_list: Sequence[SoftLabeledSample], label_classes: int, use_unlabel: bool=True)-> Sequence[SoftLabeledSample]:
+    '''
+    description: 根据不同的类别和置信度过滤数据
+                sample_list: Sequence[SoftLabeledSample]: 样本
+                label_classes: int: 类别数目
+    return {*}
+    '''
+    if len(sample_list) == 0:
+        return []
+    prob_threshold = get_prob_threshold(label_classes, use_unlabel)
+    filter_fx: Callable[[SoftLabeledSample], bool] = lambda sample: len(sample.soft_label) > 0 and max(sample.soft_label) > prob_threshold
+    filtered_sample_list = [sample for sample in sample_list if filter_fx(sample)]
+    return filtered_sample_list
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/samplers.py b/gts_engine/bagualu/lib/components/samplers.py
new file mode 100644
index 0000000..9de3c36
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/samplers.py
@@ -0,0 +1,25 @@
+import random
+from torch.utils.data import DataLoader, Sampler, Dataset, distributed
+
+
+class PairBatchSampler(Sampler):
+    def __init__(self, dataset, batch_size):
+        self.dataset = dataset
+        self.batch_size = batch_size
+
+    def __iter__(self):
+        indices = list(range(len(self.dataset)))
+        random.shuffle(indices)
+        for k in range(len(self)):
+            offset = k*self.batch_size
+            batch_indices = indices[offset:offset+self.batch_size]
+
+            pair_indices = []
+            for idx in batch_indices:
+                y = self.dataset.__getclass__(idx)
+                pair_indices.append(random.choice(self.dataset._classwise_indices[y]))
+
+            yield batch_indices + pair_indices
+
+    def __len__(self):
+        return (len(self.dataset)+self.batch_size-1) // self.batch_size
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/schedulers.py b/gts_engine/bagualu/lib/components/schedulers.py
new file mode 100644
index 0000000..26a5e57
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/schedulers.py
@@ -0,0 +1,45 @@
+from transformers.optimization import AdamW
+from torch.optim import Optimizer
+from abc import ABCMeta, abstractmethod
+from torch.optim.lr_scheduler import LambdaLR
+from typing import Callable, Union, List
+
+Lambda = Union[Callable[[int], float], List[Callable[[int], float]]]
+class LambdaLRWithMinLR(LambdaLR):
+    """重写LambdaLR，支持设置lr下限"""
+    def __init__(self, optimizer: Optimizer, lr_lambda: Lambda, last_epoch: int=-1, min_lr: float=1e-8, verbose: bool=False):
+        self._min_lr = min_lr
+        super().__init__(optimizer, lr_lambda, last_epoch, verbose) # type: ignore # pyi文件和定义冲突
+    
+    def get_lr(self):
+        lr_list: List[float] = super().get_lr() # type: ignore # pyi文件和定义冲突
+        return [max(lr, self._min_lr) for lr in lr_list]
+        
+def warmup_linear_decay_scheduler_factory(
+    optimizer: AdamW, 
+    warm_up_epoch: int, 
+    decay_epoch: int, 
+    epoch: int=4, 
+    min_lr: float=1e-8,
+    train_data_length: int=1000,
+    batch_size: int=8,
+    last_step: int=-1
+) -> LambdaLRWithMinLR:
+    
+    epoch_step = train_data_length / batch_size
+    warm_up_steps = int(epoch_step * warm_up_epoch)
+    decay_steps = int(epoch_step * decay_epoch)
+    all_steps = epoch * (train_data_length / batch_size)
+    
+    def lr_lambda(current_step: int) -> float:
+        if current_step <= warm_up_steps:
+            ratio = current_step / warm_up_steps
+        elif warm_up_steps < current_step <= decay_steps:
+            ratio = 1.0
+        else:
+            ratio = 1.0 - ((current_step - decay_steps) / (all_steps - decay_steps))
+        return ratio   
+     
+    return LambdaLRWithMinLR(optimizer, lr_lambda, last_step, min_lr)
+
+
diff --git a/gts_engine/bagualu/lib/components/text_tools.py b/gts_engine/bagualu/lib/components/text_tools.py
new file mode 100644
index 0000000..dc3e801
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/text_tools.py
@@ -0,0 +1,174 @@
+from typing import List, Optional, Tuple
+import re
+from LAC import LAC
+from transformers.tokenization_utils import PreTrainedTokenizer
+import numpy as np
+
+
+
+
+def cut_sent(para):
+    """分字并处理标点符号"""
+    para = re.sub('([，。！？,\.\?!])([^”’])', r"\1\n\2", para)  # 单字符断句符 
+    para = re.sub('(\.{6})([^”’])', r"\1\n\2", para)  # 英文省略号
+    para = re.sub('(\…{2})([^”’])', r"\1\n\2", para)  # 中文省略号
+    para = re.sub('([。！？\?][”’])([^，。！？\?])', r'\1\n\2', para)
+    # 如果双引号前有终止符，那么双引号才是句子的终点，把分句符\n放到双引号后，注意前面的几句都小心保留了双引号
+    para = para.rstrip()  # 段尾如果有多余的\n就去掉它
+    # 很多规则中会考虑分号;，但是这里我把它忽略不计，破折号、英文双引号等同样忽略，需要的再做些简单调整即可。
+    return para.split("\n")
+
+lac_seg = LAC(mode='lac')
+
+# TODO 未改动，需要优化
+def segment_text(text: str, tokenizer: Optional[PreTrainedTokenizer] = None)-> List[str]: 
+    """
+    输入一句话，返回一句经过处理的分词列表: 为了支持中文全称mask，将被分开的词，将上特殊标记("#")，使得后续处理模块，能够知道哪些字是属于同一个词的。
+    """
+    #text = "".join(cut_sent(text)) # 处理标点符号
+    seq_cws = lac_seg.run(text)[0]
+    seq_cws_dict = {x: 1 for x in seq_cws}  # 分词后的词加入到词典dict
+    
+    segment: List[str] = []
+    i = 0
+    while i < len(text):  # 从句子的第一个字开始处理，知道处理完整个句子
+        '''if len(re.findall('[\u4E00-\u9FA5]', text[i])) == 0:  # 如果找不到中文的，原文加进去即不用特殊处理。
+            new_text.append(text[i])
+            i += 1
+            continue'''
+
+        # 由于LAC对于像2018或者abc这种非中文字符会直接切分成单个字符，所以这里加入连续非中文字符的识别，然后使用bert的分词器进行重新切分
+        non_chinese = []
+        while i < len(text) and len(re.findall('[\u4E00-\u9FA5]', text[i])) == 0:
+            non_chinese.append(text[i])
+            i += 1
+        if non_chinese:
+            tokens = tokenizer.tokenize(
+                ''.join(non_chinese)) if tokenizer else non_chinese
+            segment.extend(tokens)
+        if i >= len(text):
+            break
+
+        has_add = False
+        for length in range(3, 0, -1):
+            if i + length > len(text):
+                continue
+            if ''.join(text[i:i + length]) in seq_cws_dict:
+                segment.append(text[i])
+                for l in range(1, length):
+                    segment.append('##' + text[i + l])
+                i += length
+                has_add = True
+                break
+        if not has_add:
+            segment.append(text[i])
+            i += 1
+    return segment
+
+# TODO 未改动，需要优化
+def text_2_training_instance(text: str, max_length: int, tokenizer: PreTrainedTokenizer, index: Optional[int]=None) -> Tuple[List[str], Optional[bool]]: # type: ignore
+    # char_list = segment_text(text, tokenizer) .   # 用来分句的
+    char_list = [segment_text(sen, tokenizer) for sen in cut_sent(text)]
+    max_num_tokens = max_length - 3
+    target_seq_length = max_num_tokens
+    if index is None:
+        index = np.random.randint(1, 10000)
+    rng = np.random.RandomState(seed=((1234 + index) % 2 ** 32))  # type: ignore
+    current_chunk = []  # 当前处理的文本段，包含多个句子
+    current_length = 0
+    i = 0
+
+    tokens = []
+    is_random_next = None
+
+    while i < len(char_list):  # 从文档的第一个位置开始，按个往下看
+        # segment是列表，代表的是按字分开的一个完整句子，
+        # segment=['我', '是', '一', '爷', '##们', '，', '我', '想', '我', '会', '给', '我',
+        # '媳', '##妇', '最', '##好', '的', '##幸', '##福', '。']
+        segment = char_list[i]
+
+        current_chunk.append(segment)  # 将一个独立的句子加入到当前的文本块中
+        current_length += len(segment)  # 累计到为止位置接触到句子的总长度
+        if i == len(char_list) - 1 or current_length >= target_seq_length:
+            # 如果累计的序列长度达到了目标的长度，或当前走到了文档结尾==>构造并添加到“A[SEP]B“中的A和B中；
+            if current_chunk:  # 如果当前块不为空
+                a_end = 1
+                # 当前块，如果包含超过两个句子，取当前块的一部分作为“A[SEP]B“中的A部分
+                if len(current_chunk) > 2:
+                    a_end = rng.randint(1, len(current_chunk) - 1)
+                elif len(current_chunk) == 2:
+                    a_end = 1
+
+                # 将当前文本段中选取出来的前半部分，赋值给A即tokens_a
+                tokens_a = []
+                for j in range(a_end):
+                    tokens_a.extend(current_chunk[j])
+
+                # 构造“A[SEP]B“中的B部分(有一部分是正常的当前文档中的后半部;在原BERT的实现中一部分是随机的从另一个文档中选取的，）
+                tokens_b = []
+                for j in range(a_end, len(current_chunk)):
+                    tokens_b.extend(current_chunk[j])
+
+                # 有百分之50%的概率交换一下tokens_a和tokens_b的位置
+                # print("tokens_a length1:",len(tokens_a))
+                # print("tokens_b length1:",len(tokens_b)) # len(tokens_b) = 0
+
+                if len(tokens_a) == 0 or len(tokens_b) == 0:
+                    tokens = tokens_a[:max_num_tokens]
+                    tokens.append('[SEP]')
+                    return tokens, None
+
+                if rng.random() < 0.5:  # 交换一下tokens_a和tokens_b
+                    is_random_next = True
+                    temp = tokens_a
+                    tokens_a = tokens_b
+                    tokens_b = temp
+                else:
+                    is_random_next = False
+
+                truncate_seq_pair(tokens_a, tokens_b, max_num_tokens, rng)
+
+                assert len(tokens_a) >= 1
+                assert len(tokens_b) >= 1
+
+                # 把tokens_a & tokens_b加入到按照bert的风格，即以[CLS]tokens_a[SEP]tokens_b[SEP]的形式，结合到一起，
+                # 作为最终的tokens; 也带上segment_ids，前面部分segment_ids的值是0，后面部分的值是1.
+                tokens = []
+                segment_ids = []
+                segment_ids.append(0)
+                for token in tokens_a:
+                    tokens.append(token)
+                    segment_ids.append(0)
+
+                tokens.append("[SEP]")
+                segment_ids.append(0)
+
+                for token in tokens_b:
+                    tokens.append(token)
+                    segment_ids.append(1)
+                segment_ids.append(1)
+
+            return tokens, is_random_next
+
+        i += 1  # 接着文档中的内容往后看
+    
+    
+def truncate_seq_pair(tokens_a, tokens_b, max_num_tokens, rng):
+    """Truncates a pair of sequences to a maximum sequence length."""
+    while True:
+        total_length = len(tokens_a) + len(tokens_b)
+        if total_length <= max_num_tokens:
+            break
+
+        trunc_tokens = tokens_a if len(tokens_a) > len(tokens_b) else tokens_b
+        assert len(trunc_tokens) >= 1
+
+        # We want to sometimes truncate from the front and sometimes from the
+        # back to add more randomness and avoid biases.
+        if rng.random() < 0.5:
+            del trunc_tokens[0]
+        else:
+            trunc_tokens.pop()
+    
+    
+    
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/components/tokenizer_generator.py b/gts_engine/bagualu/lib/components/tokenizer_generator.py
new file mode 100644
index 0000000..3a6cc63
--- /dev/null
+++ b/gts_engine/bagualu/lib/components/tokenizer_generator.py
@@ -0,0 +1,62 @@
+import os
+from typing import Dict, Union
+from transformers.tokenization_utils import PreTrainedTokenizer
+from transformers.models.auto.tokenization_auto import AutoTokenizer
+from types import MethodType
+from pydantic import DirectoryPath
+
+from ..framework.consts import RUN_MODE
+from ..utils.json_processor import load_json
+
+
+class TokenizerGenerator:
+    """Tokenizer生成器
+        
+        通过generate_tokenizer()方法，根据预训练模型路径生成相应的tokenizer
+    """
+    @classmethod
+    def generate_tokenizer(cls, pretrained_model_dir: Union[str, DirectoryPath]) -> PreTrainedTokenizer:
+        if not os.path.exists(pretrained_model_dir):
+            raise Exception("pretrained model directory dose not exists")
+        pretrained_model_config_path = os.path.join(pretrained_model_dir, 'config.json')
+        if not os.path.exists(pretrained_model_config_path):
+            raise Exception(f"model config file {pretrained_model_config_path} does not exist")
+        model_config: Dict = load_json(pretrained_model_config_path)
+        if "vocab_size" not in model_config:
+            raise Exception(f"model config does not have field: 'vocab_size'")
+        return cls.__get_ernie_tokenizer(pretrained_model_dir) if model_config['vocab_size'] == 18000 \
+            else cls.__get_bert_tokenizer(pretrained_model_dir)
+            
+    @classmethod
+    def __get_ernie_tokenizer(cls, pretrained_model_dir: Union[str, DirectoryPath]) -> PreTrainedTokenizer:
+        """
+        load ernie tokenizer
+        """
+        added_token = ['[unused'+str(i+1)+']' for i in range(36)]
+        tokenizer = AutoTokenizer.from_pretrained(pretrained_model_dir,
+                            additional_special_tokens=added_token)
+        return cls.__tokenizer_wraper(tokenizer)
+    
+    @classmethod
+    def __get_bert_tokenizer(cls, pretrained_model_dir: Union[str, DirectoryPath]) -> PreTrainedTokenizer:
+        """
+        load normal tokenizer
+        """
+        added_token = ['[unused'+str(i+1)+']' for i in range(99)]
+        tokenizer = AutoTokenizer.from_pretrained(pretrained_model_dir,
+                                        additional_special_tokens=added_token)
+        return cls.__tokenizer_wraper(tokenizer)
+
+    @classmethod
+    def __tokenizer_wraper(cls, tokenizer: PreTrainedTokenizer) -> PreTrainedTokenizer:
+        """
+            原tokenizer的get_vocab方法运算较慢，覆盖tokenizer实例的get_vocab方法，使之只需访问计算好的缓存
+        """
+        vocab = tokenizer.get_vocab()
+        setattr(tokenizer, "vocab_cache", vocab)
+        def get_vocab(self: PreTrainedTokenizer):
+            return getattr(self, "vocab_cache")
+        tokenizer.get_vocab = MethodType(get_vocab, tokenizer)
+        return tokenizer
+
+
diff --git a/gts_engine/bagualu/lib/framework/__init__.py b/gts_engine/bagualu/lib/framework/__init__.py
new file mode 100644
index 0000000..7cd51e5
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/__init__.py
@@ -0,0 +1,4 @@
+from .base_arguments import BaseArguments
+from .base_training_pipeline import BaseTrainingPipeline
+from .base_inference_manager import BaseInferenceManager
+from .base_gts_engine_interface import GtsEngineArgs, BaseGtsEngineInterface
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/base_arguments.py b/gts_engine/bagualu/lib/framework/base_arguments.py
new file mode 100644
index 0000000..8214071
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/base_arguments.py
@@ -0,0 +1,77 @@
+from abc import ABCMeta, abstractmethod
+from argparse import ArgumentParser, Namespace, _ArgumentGroup
+from typing import Optional, Dict, List, Type, Union, get_type_hints
+from typing_extensions import Self
+import sys
+
+GeneralParser = Union[ArgumentParser, _ArgumentGroup]
+
+class BaseArguments(Namespace, metaclass=ABCMeta):
+    """支持自动解析嵌套结构的参数类
+    
+        通过继承，可以只需定义参数间的嵌套结构，实现形如
+            main_arg
+                - main_arg_param1
+                - main_arg_param2
+                - ...
+                
+                - sub_arg1
+                    - sub_arg_param
+                - sub_arg2
+                    - ...
+                - ...
+        的嵌套参数，通过最外层参数的parse_args()方法自动完成解析
+
+        > 使用方法参见 ./demo/arguments_base_demo.py
+        
+        > todo:
+        > - [ ] 支持从类解析为命令行参数字符串
+        > - [ ] 美化打印
+    """
+    
+    @property
+    def _sub_args_dict(self) -> Dict[str, Type["BaseArguments"]]:
+        hintsDict = get_type_hints(type(self))
+        return {key: val for key, val in hintsDict.items() if isinstance(val, type) and issubclass(val, BaseArguments)}
+    
+    @property
+    def _arg_name(self) -> Optional[str]:
+        return None
+    
+    @property
+    def _arg_description(self) -> Optional[str]:
+        return None
+    
+    def parse_args(self, parse_list: Optional[List[str]] = None) -> Self:
+        # print(self._sub_args_dict)
+        if len(sys.argv) > 1 and sys.argv[1] in {"--help", "-h"}:
+            self._show_help()
+        parser = ArgumentParser(prog=self._arg_name, description=self._arg_description)
+        self._add_args(parser)
+        _, left_params = parser.parse_known_args(parse_list, namespace=self)
+        for arg_name, arg_cls in self._sub_args_dict.items():
+            setattr(self, arg_name, arg_cls().parse_args(left_params))
+        
+        self._after_parse()
+        return self
+    
+    def _add_args(self, parser: GeneralParser) -> None:
+        pass
+    
+    def _show_help(self) -> None:
+        parser = ArgumentParser(prog=self._arg_name, description=self._arg_description)
+        self._add_all_args(parser)
+        parser.parse_args()
+
+    def _add_all_args(self, parser: ArgumentParser) -> ArgumentParser:
+        parser_group = parser.add_argument_group(self._arg_name, self._arg_description)
+        self._add_args(parser_group)
+        for _, arg_cls in self._sub_args_dict.items():
+            arg = arg_cls()
+            arg._add_all_args(parser)
+        return parser
+    
+    def _after_parse(self) -> None:
+        pass
+    
+    
diff --git a/gts_engine/bagualu/lib/framework/base_gts_engine_interface.py b/gts_engine/bagualu/lib/framework/base_gts_engine_interface.py
new file mode 100644
index 0000000..ad28cdb
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/base_gts_engine_interface.py
@@ -0,0 +1,79 @@
+from abc import abstractmethod, ABCMeta, abstractproperty
+from typing import Type, List
+from enum import Enum
+
+from gts_common.arguments import GtsEngineArgs
+
+from .base_training_pipeline import BaseTrainingPipeline
+from .base_inference_manager import BaseInferenceManager
+
+
+class TRAIN_MODE(Enum):
+    DEFAULT = "default"
+    STD = "standard"
+    FAST = "fast"
+    ADV = "advanced"
+
+
+class TASK(Enum):
+    CLS = "classification"
+    NLI = "nli"
+    SIM = "similarity"
+    IE = "ie"
+
+
+class ENGINE_TYPE(Enum):
+    QKD = "qiankunding"
+    BGL = "bagualu"
+
+
+class BaseGtsEngineInterface(metaclass=ABCMeta):
+    """适配GTS-Engine的胶水模块基类，根据GTS-Engine的参数生成对应的bagualu模块"""
+
+    def generate_training_pipeline(self, args: GtsEngineArgs) -> BaseTrainingPipeline:
+        """通过GTS-Engine参数实例化bagualu TrainingPipeline"""
+        parsed_args_list = self._parse_training_args(args)
+        print(
+            f"\n------------------------- parsed args for {self._training_pipeline_type.__name__} -------------------------")
+        print(f"\n{' '.join(parsed_args_list)}\n")
+        print("------------------------------------------------------------------------------------------\n")
+        return self._training_pipeline_type(parsed_args_list)
+
+    def prepare_training(self, args: GtsEngineArgs) -> None:
+        """训练准备处理，如数据格式转换等"""
+        return None
+
+    def generate_inference_manager(self, args: GtsEngineArgs) -> BaseInferenceManager:
+        """通过GTS-Engine参数实例化agualu InferenceManager"""
+        parsed_args_list = self._parse_inference_args(args)
+        print(
+            f"\n------------------------- parsed args for {self._inference_manager_type.__name__} -------------------------")
+        print(f"\n{' '.join(parsed_args_list)}\n")
+        print("------------------------------------------------------------------------------------------\n")
+        inference_manager = self._inference_manager_type(parsed_args_list)
+        inference_manager.prepare_inference()
+        return inference_manager
+
+    def prepare_inference(self, args: GtsEngineArgs) -> None:
+        """推理准备处理，如数据格式转换等"""
+        return None
+
+    ########################### abstract ################################
+
+    @abstractproperty
+    def _training_pipeline_type(self) -> Type[BaseTrainingPipeline]:
+        """对应bagualu TrainingPipeline类"""
+        ...
+
+    @abstractmethod
+    def _parse_training_args(self, args: GtsEngineArgs) -> List[str]:
+        """将GTS-Engine参数解析为bagualu TrainingPipeline启动参数字符串列表"""
+
+    @abstractproperty
+    def _inference_manager_type(self) -> Type[BaseInferenceManager]:
+        """对应bagualu InferenceManager 类"""
+        ...
+
+    @abstractmethod
+    def _parse_inference_args(self, args: GtsEngineArgs) -> List[str]:
+        """将GTS-Engine参数解析为bagualu InferenceManager启动参数字符串列表"""
diff --git a/gts_engine/bagualu/lib/framework/base_inference_manager.py b/gts_engine/bagualu/lib/framework/base_inference_manager.py
new file mode 100644
index 0000000..d3fac30
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/base_inference_manager.py
@@ -0,0 +1,19 @@
+from abc import ABCMeta, abstractmethod
+from typing import Optional, List, Sequence, Dict
+from pydantic import FilePath, BaseModel
+
+from .mixin import ArgsMixin
+
+InfSample = BaseModel
+InfOutput = Dict
+
+class BaseInferenceManager(ArgsMixin, metaclass=ABCMeta):
+    
+    def __init__(self, args_parse_list: Optional[List[str]] = None):
+        ArgsMixin.__init__(self, args_parse_list)
+    
+    @abstractmethod
+    def prepare_inference(self) -> None: ...
+    
+    @abstractmethod
+    def inference(self, sample: Sequence[InfSample]) -> InfOutput: ...
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/base_training_pipeline.py b/gts_engine/bagualu/lib/framework/base_training_pipeline.py
new file mode 100644
index 0000000..d7f185f
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/base_training_pipeline.py
@@ -0,0 +1,26 @@
+from abc import ABCMeta, abstractmethod
+from typing import Optional, List
+
+from .mixin import ArgsMixin
+
+class BaseTrainingPipeline(ArgsMixin, metaclass=ABCMeta):
+    
+    def __init__(self, args_parse_list: Optional[List[str]] = None):
+        ArgsMixin.__init__(self, args_parse_list)
+    
+    @abstractmethod
+    def _before_training(self) -> None:
+        pass
+    
+    @abstractmethod
+    def _train(self) -> None:
+        pass
+    
+    @abstractmethod
+    def _after_training(self) -> None:
+        pass
+    
+    def main(self):
+        self._before_training()
+        self._train()
+        self._after_training()
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/__init__.py b/gts_engine/bagualu/lib/framework/classification_finetune/__init__.py
new file mode 100644
index 0000000..79e5b20
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/__init__.py
@@ -0,0 +1,8 @@
+from .base_training_pipeline_clf import BaseTrainingPipelineClf
+from .base_arguments_clf import BaseTrainingArgumentsClf, BaseInferenceArgumentsClf
+from .base_data_module_clf import BaseDataModuleClf
+from .base_lightnings_clf import BaseTrainingLightningClf, BaseInferenceLightningClf
+from .base_dataset_clf import BaseDatasetClf
+from .base_inference_manager_clf import BaseInferenceManagerClf
+from .prompt import StdPrompt
+from .data_reader_clf import DataReaderClf
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/base_arguments_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/base_arguments_clf.py
new file mode 100644
index 0000000..160593c
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/base_arguments_clf.py
@@ -0,0 +1,144 @@
+from typing import Optional, List, Union
+import os
+import time
+from pathlib import Path
+from pydantic import DirectoryPath, FilePath
+
+from ..base_arguments import BaseArguments, GeneralParser
+from ...components import ProtocolArgsMixin
+from ...utils.path import mk_inexist_dir
+from ...components.lightning_callbacks.adaptive_val_intervals import ADAPTIVE_VAL_INTERVAL_MODE
+from ..consts import RUN_MODE
+
+class BaseTrainingArgumentsClf(BaseArguments, ProtocolArgsMixin): 
+
+    #############################################################################################
+    ######################################## 外部参数 ##########################################
+    
+    dataset: str
+    gpu_num: int
+    load_data_ratio: float
+    selected_pretrained_model_dir: Optional[DirectoryPath]
+    """指定的预训练模型路径"""
+    timestamp: str
+    debug: bool
+    """是否为debug模式"""
+    run_mode: RUN_MODE
+    num_workers: int
+    train_batchsize_per_device: int
+    valid_batchsize_per_device: int
+    test_batchsize_per_device: int
+    max_length: int
+    seed: int
+    learning_rate: float
+    train_data_path: FilePath
+    """训练数据集"""
+    dev_data_path: Optional[FilePath]
+    """验证数据集"""
+    test_data_path: Optional[FilePath]
+    """离线测试数据集"""
+    online_test_data_path: Optional[FilePath]
+    """无标注推理数据集"""
+    label2id_path: FilePath
+    """label2id文件"""
+    _log_dir: Optional[DirectoryPath]
+
+    @property
+    def log_dir(self) -> DirectoryPath:
+        return self.log_dir if self._log_dir is None else self._log_dir
+
+    def _add_args(self, parser: GeneralParser) -> None:
+        parser.add_argument("--train_data_path", dest="train_data_path", type=Path, help="指定训练数据集路径", required=True)
+        parser.add_argument("--dev_data_path", dest="dev_data_path", type=Path, default=None, help="[可选]指定验证数据集路径")
+        parser.add_argument("--test_data_path", dest="test_data_path", type=Path, default=None, help="[可选]指定离线测试数据集路径")
+        parser.add_argument("--online_test_data_path", dest="online_test_data_path", type=Path, default=None, help="[可选]指定在线测试数据集路径")
+        parser.add_argument("--dataset", dest="dataset", type=str, help="数据集名称/路径名", default="default")
+        parser.add_argument("--label2id_path", dest="label2id_path", type=Path, help="指定label2id文件路径", required=True)
+        parser.add_argument("--log_dir", dest="_log_dir", type=Path, default=None, help="[可选]指定日志文件保存路径")
+        parser.add_argument("--run_mode", dest="run_mode", type=RUN_MODE, 
+                            choices=RUN_MODE, default=RUN_MODE.ONLINE, 
+                            help="运行模式 [offline - 离线测试 | online - 上线运行]")
+        parser.add_argument("--load_data_ratio", dest="load_data_ratio", type=float, default=1, help="[可选]指定加载数据比例（0到1之间），用于测试")
+        parser.add_argument("--selected_pretrained_model_dir", dest="selected_pretrained_model_dir", type=str, default=None, help="[可选]指定预训练模型路径")
+        parser.add_argument("--seed", dest="seed", type=int, default=42)
+        parser.add_argument("--debug", action="store_true", help="[可选]debug模式")
+        parser.add_argument("--num_workers", dest="num_workers", type=int, default=6)
+        parser.add_argument("--train_batchsize", dest="train_batchsize_per_device", type=int, default=8)
+        parser.add_argument("--valid_batchsize", dest="valid_batchsize_per_device", type=int, default=8)
+        parser.add_argument("--test_batchsize", dest="test_batchsize_per_device", type=int, default=8)
+        parser.add_argument("--max_length", dest="max_length", type=int, default=512)
+        parser.add_argument("--learning_rate", dest="learning_rate", type=float, default=2e-5)
+        parser.add_argument("--gpu_num", dest="gpu_num", type=int, default=1, help="[可选]指定训练gpu数量，系统自动分配空闲gpu，-1为使用全部可见gpu，默认为1")
+        
+    #############################################################################################
+    ######################################## 固定参数 ##########################################
+    
+    pretrained_model_dir: DirectoryPath
+    logger = "fintune_logger"
+    prefix_prompt = "文本分类任务："
+    training_label_prompt = "[unused1][unused2][MASK][MASK][MASK][unused3][unused4]"
+    inference_label_prompt = "[unused1][MASK][MASK][MASK][MASK][MASK][unused2]"
+    label_guided_rate = 0.5
+    wwm_mask_rate = 0.12
+    @property
+    def train_batch_size(self) -> int:
+        return self.train_batchsize_per_device * self.device_num
+    @property
+    def test_batch_size(self) -> int:
+        return self.test_batchsize_per_device * self.device_num
+    @property
+    def valid_batch_size(self) -> int:
+        return self.valid_batchsize_per_device * self.device_num
+    epoch = 5
+    @property
+    def decay_epoch(self): 
+        return self.epoch - 1
+    warm_up_epoch = 1
+    clip_norm = 0.25
+    validation_mode = ADAPTIVE_VAL_INTERVAL_MODE.ADAPTIVE
+    timestamp = str(int(time.time()))
+    
+    @property
+    def ft_output_dir(self) -> DirectoryPath:
+        if self.run_mode == RUN_MODE.OFFLINE:
+            output_path = self.protocol_args.output_dir / self.dataset
+            output_path = output_path / self.timestamp
+        else:
+            output_path = self.protocol_args.student_output_dir / 'finetune_output'
+        return output_path
+    
+    @property
+    def device_num(self) -> int:
+        return self.gpu_num
+    
+    #############################################################################################
+    ######################################## 其他逻辑 ##########################################
+    
+    def _after_parse(self) -> None:
+        mk_inexist_dir(self.log_dir)
+        mk_inexist_dir(self.ft_output_dir, clean=True)
+     
+   
+    
+class BaseInferenceArgumentsClf(BaseArguments):
+    
+    model_save_dir: DirectoryPath
+    label2id_path: FilePath
+    
+    def _add_args(self, parser) -> None:
+        parser.add_argument("--model_save_dir", dest="model_save_dir", type=Path, help="输出文件路径", required=True)
+        parser.add_argument("--label2id_path", dest="label2id_path", type=Path, help="[可选]指定label2id文件路径", required=True)
+        
+    logger = "fintune_logger"
+    prefix_prompt = "文本分类任务："
+    inference_label_prompt = "[unused1][MASK][MASK][MASK][MASK][MASK][unused2]"
+    max_length = 512
+    batch_size = 8
+    
+    @property
+    def model_state_dict_file_path(self) -> FilePath:
+        return self.model_save_dir / "finetune_pytorch.bin"
+    
+    @property
+    def pretrained_model_dir(self) -> DirectoryPath:
+        return self.model_save_dir
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/base_data_module_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/base_data_module_clf.py
new file mode 100644
index 0000000..f4937c7
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/base_data_module_clf.py
@@ -0,0 +1,208 @@
+from abc import abstractmethod, ABCMeta
+from pytorch_lightning import LightningDataModule
+from transformers.tokenization_utils import PreTrainedTokenizer
+from typing import List, Optional, Literal, Sequence
+from torch.utils.data import DataLoader
+import os
+import math
+from collections import Counter
+from sklearn.model_selection import train_test_split
+import random
+
+from ...components import LoggerManager
+from .prompt import StdPrompt
+from .consts import Label2Token, LabeledSample, UnlabeledSample, InfSampleProto
+from .base_arguments_clf import BaseTrainingArgumentsClf
+from ..consts import TRAINING_STAGE
+from .base_dataset_clf import BaseDatasetClf
+from .data_reader_clf import DataReaderClf
+
+    
+class BaseDataModuleClf(LightningDataModule, metaclass=ABCMeta):
+    
+    def __init__(
+        self,
+        args: BaseTrainingArgumentsClf,
+        prompt: StdPrompt,
+        tokenizer: PreTrainedTokenizer
+    ):
+        super().__init__()
+        self._args = args
+        self._prompt = prompt
+        self._tokenizer = tokenizer
+        self._logger = LoggerManager.get_logger(self._args.logger)
+        
+    def train_dataloader(self, load_ratio: float = 1):
+        sample_list = self.train_sample_list[:int(len(self.train_sample_list) * load_ratio)]
+        self._logger.info(f"number of training sample: {len(sample_list)}")
+        lg_dataset = self._get_train_dataset(sample_list)
+        return DataLoader(dataset=lg_dataset, batch_size=self._args.train_batch_size, num_workers=self._args.num_workers * self._args.device_num, shuffle=True)
+        
+    def val_dataloader(self, stage: Literal[TRAINING_STAGE.VALIDATION, TRAINING_STAGE.INFERENCE], load_ratio: float = 1, resample_thres: Optional[int] = None):
+        if self.dev_sample_num == 0:
+            return None
+        sample_list = self.dev_sample_list[:int(len(self.dev_sample_list) * load_ratio)]
+        if resample_thres:
+            sample_list = self._resample_sample_list(sample_list, resample_thres)
+        if stage == TRAINING_STAGE.VALIDATION:
+            lg_dataset = self._get_test_dataset(sample_list)
+            return DataLoader(dataset=lg_dataset, batch_size=self._args.valid_batch_size, num_workers=self._args.num_workers * self._args.device_num)
+        elif stage == TRAINING_STAGE.INFERENCE:
+            lg_dataset = self._get_inf_dataset(sample_list)
+            return DataLoader(dataset=lg_dataset, batch_size=self._args.valid_batchsize_per_device, num_workers=self._args.num_workers)
+        else:
+            raise Exception("stage is not in [TRAINING_STAGE.VALIDATION, TRAINING_STAGE.INFERENCE]")
+    
+    def test_dataloader(self, stage: Literal[TRAINING_STAGE.TEST, TRAINING_STAGE.INFERENCE], load_ratio: float = 1):
+        sample_list = self.test_sample_list[:int(len(self.test_sample_list) * load_ratio)]
+        if stage == TRAINING_STAGE.TEST:
+            lg_dataset = self._get_test_dataset(sample_list)
+            return DataLoader(dataset=lg_dataset, batch_size=self._args.test_batch_size, num_workers=self._args.num_workers * self._args.device_num)
+        elif stage == TRAINING_STAGE.INFERENCE:
+            lg_dataset = self._get_inf_dataset(sample_list)
+            return DataLoader(dataset=lg_dataset, batch_size=self._args.test_batchsize_per_device, num_workers=self._args.num_workers)
+        else:
+            raise Exception("stage is not in [TRAINING_STAGE.VALIDATION, TRAINING_STAGE.INFERENCE]")
+    
+    def online_test_dataloader(self, load_ratio: float = 1):
+        sample_list = self.online_test_sample_list[:int(len(self.online_test_sample_list) * load_ratio)]
+        lg_dataset = self._get_inf_dataset(sample_list)
+        return DataLoader(dataset=lg_dataset, batch_size=self._args.test_batchsize_per_device, num_workers=self._args.num_workers)
+            
+    @property
+    def train_sample_num(self) -> int:
+        return len(self.train_sample_list)
+    
+    @property
+    def dev_sample_num(self) -> int:
+        return len(self.dev_sample_list)
+    
+    @property
+    def class_num(self) -> int:
+        return len(self._prompt.label2token)
+    
+    _train_sample_list: Optional[List[LabeledSample]] = None
+    @property
+    def train_sample_list(self) -> List[LabeledSample]:
+        if self._train_sample_list is None:
+            self._train_sample_list = self._load_train_sample_list()
+        return self._train_sample_list
+    
+    _dev_sample_list: Optional[List[LabeledSample]] = None
+    @property
+    def dev_sample_list(self) -> List[LabeledSample]:
+        if self._dev_sample_list is None:
+            self._dev_sample_list = self._load_dev_sample_list()
+        return self._dev_sample_list
+    
+    _test_sample_list: Optional[List[LabeledSample]] = None
+    @property
+    def test_sample_list(self) -> List[LabeledSample]:
+        """离线测试数据集"""
+        if self._test_sample_list is None:
+            self._test_sample_list = self._load_test_sample_list()
+        return self._test_sample_list
+    
+    _online_test_sample_list: Optional[List[UnlabeledSample]] = None
+    @property
+    def online_test_sample_list(self) -> List[UnlabeledSample]:
+        """推理数据集"""
+        if self._online_test_sample_list is None:
+            self._online_test_sample_list = self._load_online_test_sample_list()
+        return self._online_test_sample_list
+    
+    def _resample_sample_list(self, sample_list: List[LabeledSample], resample_thres: int = 1000) -> List[LabeledSample]:
+        """验证数据过多时，进行重新抽样"""
+        sample_num = len(sample_list)
+        len_t = min(sample_num, resample_thres)
+
+        split_rate = min(len_t / sample_num, 1.0)
+        if split_rate == 1.0:
+            return sample_list
+
+        labels = [sample.label_id_clf for sample in sample_list]
+        label_lt_five = [
+            ele[0] for ele in Counter(labels).items() if ele[1] <= 5
+        ]
+        sample_lt_five = [
+            sample for sample in sample_list
+            if sample.label_id_clf in label_lt_five
+        ]
+        sample_more_five = [
+            sample for sample in sample_list
+            if sample.label_id_clf not in label_lt_five
+        ]
+        labels_more_five = [
+            sample.label_id_clf for sample in sample_more_five
+        ]
+        _, new_dev, _, _ = train_test_split(sample_more_five,
+                                            labels_more_five,
+                                            test_size=split_rate,
+                                            stratify=labels_more_five)
+        new_dev.extend(sample_lt_five)
+        self._logger.info('Size of new dev dataset: {}'.format(len(new_dev)))
+
+        return new_dev
+    
+    #############################################################################################
+    ## abstract
+    #############################################################################################
+    
+    @abstractmethod
+    def _get_train_dataset(self, sample_list: Sequence[LabeledSample]) -> BaseDatasetClf:
+        """根据需要加载训练Dataset"""
+        
+    @abstractmethod
+    def _get_test_dataset(self, sample_list: Sequence[LabeledSample]) -> BaseDatasetClf:
+        """根据需要加载测试Dataset"""
+        
+    @abstractmethod
+    def _get_inf_dataset(self, sample_list: Sequence[InfSampleProto]) -> BaseDatasetClf:
+        """根据需要加载推理Dataset"""
+    
+    def _load_train_sample_list(self) -> List[LabeledSample]:
+        """加载训练Sample列表"""
+        train_data_path = self._args.train_data_path
+        if train_data_path is not None and os.path.exists(train_data_path):
+            return list(DataReaderClf.read_labeled_sample(self._args.train_data_path, self._prompt.label2token))
+        else:
+            raise Exception("no training data is passed")
+        
+    def _load_dev_sample_list(self) -> List[LabeledSample]:
+        """加载验证Sample列表"""
+        dev_data_path = self._args.dev_data_path
+        if dev_data_path is not None and os.path.exists(dev_data_path):
+            return list(DataReaderClf.read_labeled_sample(dev_data_path, self._prompt.label2token))
+        else:
+            return []
+    
+    def _load_test_sample_list(self) -> List[LabeledSample]:
+        """加载离线测试Sample列表"""
+        test_data_path = self._args.test_data_path
+        if test_data_path is not None and os.path.exists(test_data_path):
+            return list(DataReaderClf.read_labeled_sample(test_data_path, self._prompt.label2token))
+        else:
+            raise Exception("no valid test data path is passed")
+    
+    def _load_online_test_sample_list(self) -> List[UnlabeledSample]:
+        """加载在线推理Sample列表"""
+        online_test_data_path = self._args.online_test_data_path
+        if online_test_data_path is not None and os.path.exists(online_test_data_path):
+            return list(DataReaderClf.read_unlabeled_sample(online_test_data_path))
+        else:
+            raise Exception("no valid online test data path is passed")
+        
+    def reparse_args(self):
+        """根据数据信息更改训练参数"""
+        if len(self._prompt.label_ids) <= 5:
+            self._args.label_guided_rate = 0.3
+        step_per_epoch = self.train_sample_num // self._args.train_batch_size
+        steps = step_per_epoch * self._args.epoch
+        min_steps = min(1400 // self._args.device_num, 40 * step_per_epoch)
+        if steps < min_steps:
+            self._args.epoch = math.ceil(min_steps // step_per_epoch)
+        if self.dev_sample_num == 0:
+            self._args.epoch = int(0.75 * self._args.epoch)
+        self._logger.info(f"reparsing epoch: {self._args.epoch}")
+        self._logger.info(f"batch size per device: {self._args.train_batchsize_per_device}")
+        self._logger.info(f"total batch size: {self._args.train_batch_size}")
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/base_dataset_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/base_dataset_clf.py
new file mode 100644
index 0000000..203a516
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/base_dataset_clf.py
@@ -0,0 +1,57 @@
+from torch.utils.data import Dataset
+from typing import Generic, Protocol, Sequence, Union, TypeVar
+from dataclasses import asdict, is_dataclass
+
+from .base_arguments_clf import BaseTrainingArgumentsClf
+from .prompt import StdPrompt
+from .consts import PreTrainedTokenizer
+from collections import defaultdict
+
+
+class RawSampleProto(Protocol):
+    text: str
+
+class DatasetArgsProto(Protocol):
+    inference_label_prompt: str
+    prefix_prompt: str
+    
+
+
+class BaseDatasetClf(Dataset):
+    def __init__(
+        self,
+        sample_list: Sequence[RawSampleProto],
+        tokenizer: PreTrainedTokenizer,
+        prompt: StdPrompt
+    ):
+        self._tokenizer = tokenizer
+        self._prompt = prompt
+        self._pre_encoded_sample_list = [self._encode_before_iter(sample, idx) for idx, sample in enumerate(sample_list)]
+        self.sample_list = sample_list
+        self._classwise_indices = defaultdict(list)
+        for id in range(len(sample_list)):
+            item = sample_list[id]
+            if hasattr(item, "label_id_clf"):
+                self._classwise_indices[item.label_id_clf].append(id)  # type: ignore
+
+    def __len__(self):
+        return len(self._pre_encoded_sample_list)
+    
+    def __getitem__(self, index):
+        encoded_sample = self._encode_on_iter(self._pre_encoded_sample_list[index], index)
+        return asdict(encoded_sample)
+    
+    def __getclass__(self, index):
+        return self.sample_list[index].label_id_clf # type: ignore
+    
+    def _encode_before_iter(self, sample: RawSampleProto, idx: int): 
+        """迭代前编码"""
+        return sample
+    
+    def _encode_on_iter(self, sample, idx: int):
+        """迭代中编码
+        
+            用于有随机成分的编码，使每次迭代时编码不同
+        """
+        return sample
+        
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/base_inference_manager_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/base_inference_manager_clf.py
new file mode 100644
index 0000000..47bfc11
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/base_inference_manager_clf.py
@@ -0,0 +1,68 @@
+from pydantic import FilePath
+from transformers.tokenization_utils import PreTrainedTokenizer
+import os
+from abc import ABCMeta, abstractmethod
+from pytorch_lightning import Trainer
+import torch
+import shutil
+from torch.utils.data import DataLoader
+
+from ..base_inference_manager import BaseInferenceManager
+from .base_arguments_clf import BaseInferenceArgumentsClf
+from .base_lightnings_clf import BaseInferenceLightningClf
+from .base_dataset_clf import BaseDatasetClf
+from .consts import InferenceManagerInputSampleList, InferenceManagerOutput
+from ..mixin import OptionalLoggerMixin
+from ...components import TokenizerGenerator
+from .prompt import StdPrompt
+
+class BaseInferenceManagerClf(BaseInferenceManager, OptionalLoggerMixin):
+    
+    _args: BaseInferenceArgumentsClf
+    
+    def prepare_inference(self) -> None:
+        self.info(f"loading model...")
+        self.info(f"generate tokenizer...")
+        self._tokenizer = self._generate_tokenizer()
+        self.info(f"loading prompt...")
+        self._prompt = self._load_prompt()
+        self.info(f"loading model...")
+        self._inf_lightning = self._get_inf_lightning()
+        self._inf_lightning.load_model_from_state_dict(torch.load(self._args.model_state_dict_file_path))
+        self.info(f"init prediction lightning trainer")
+        self._trainer =  Trainer(
+            accelerator="gpu",
+            devices=1,
+            default_root_dir=str(self._args.model_save_dir / "tmp"),
+            enable_progress_bar=False,
+            auto_select_gpus=True
+        )
+        
+    def inference(self, sample: InferenceManagerInputSampleList) -> InferenceManagerOutput:
+        self.info(f"processing data...")
+        dataset = self._get_dataset(sample)
+        dataloader = DataLoader(dataset, batch_size=self._args.batch_size, num_workers=6)
+        self.info(f"predicting on data...")
+        inf_output: InferenceManagerOutput = self._trainer.predict( # type: ignore
+            model=self._inf_lightning,
+            dataloaders=dataloader
+        )
+        shutil.rmtree(str(self._args.model_save_dir / "tmp"))
+        return inf_output
+        
+    
+    def _generate_tokenizer(self) -> PreTrainedTokenizer:
+        return TokenizerGenerator.generate_tokenizer(self._args.model_save_dir)
+    
+    def _load_prompt(self):
+        return StdPrompt(self._args.prefix_prompt, self._args.label2id_path)
+    
+    @abstractmethod
+    def _get_inf_lightning(self) -> BaseInferenceLightningClf: ...
+    
+    @abstractmethod
+    def _get_dataset(self, sample: InferenceManagerInputSampleList) -> BaseDatasetClf: ...
+    
+    
+    
+    
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/base_lightnings_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/base_lightnings_clf.py
new file mode 100644
index 0000000..ff46f99
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/base_lightnings_clf.py
@@ -0,0 +1,53 @@
+from pytorch_lightning import LightningModule
+import torch
+from typing import Dict, Any
+from logging import Logger
+from abc import ABCMeta, abstractmethod
+from typing import List
+
+from .consts import InfBatch, InferenceManagerOutput
+
+class BaseTrainingLightningClf(LightningModule):
+    
+    _model: torch.nn.Module
+    _logger: Logger
+    
+    def get_model_state_dict(self):
+        return self._model.state_dict()
+    
+    def save_model(self, path: str):
+        torch.save(self.get_model_state_dict(), f=path)
+        
+    def load_model_from_state_dict(self, state_dict: Dict[str, Any]):
+        self._model.load_state_dict(state_dict, strict=False) # type: ignore
+        
+    @property
+    def model(self):
+        return self._model
+    
+    def on_save_checkpoint(self, checkpoint: Dict[str, Any]) -> None:
+        try:
+            dev_acc: float = self.trainer.logged_metrics["dev_acc"]
+            self._logger.info(f"save checkpoint. dev acc: {dev_acc:.4f}")
+        except:
+            pass
+        return super().on_save_checkpoint(checkpoint)
+    
+
+class BaseInferenceLightningClf(LightningModule):
+    
+    _model: torch.nn.Module
+    
+    def load_model_from_state_dict(self, state_dict: Dict[str, Any]):
+        self._model.load_state_dict(state_dict, strict=False) # type: ignore
+        
+    @abstractmethod
+    def predict_step(self, batch: InfBatch, batch_idx: int, dataloader_idx: int = 0) -> InferenceManagerOutput:
+        ...
+        
+    def on_predict_epoch_end(self, results: List[List[InferenceManagerOutput]]) -> None:
+        res = InferenceManagerOutput(predictions=[], probabilities=[])
+        for inf_output in results[0]:
+            res["predictions"].extend(inf_output["predictions"])
+            res["probabilities"].extend(inf_output["probabilities"])
+        results[0] = res # type: ignore 
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/base_training_pipeline_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/base_training_pipeline_clf.py
new file mode 100644
index 0000000..8d0523a
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/base_training_pipeline_clf.py
@@ -0,0 +1,444 @@
+from abc import abstractmethod, ABCMeta
+from dataclasses import asdict
+import datetime
+from logging import Logger
+import re
+import os
+import shutil
+from transformers import AutoTokenizer, AutoModelForMaskedLM
+from transformers.tokenization_utils import PreTrainedTokenizer
+from pytorch_lightning import Trainer
+from pytorch_lightning.loggers import Logger as PlLogger, TensorBoardLogger
+from typing import List, Literal, Tuple, Dict, Union, Any
+import torch
+from pathlib import Path
+from pydantic import DirectoryPath
+
+from ...utils import LoggerManager
+from ...components import TokenizerGenerator
+from .prompt import StdPrompt
+from ..base_training_pipeline import BaseTrainingPipeline
+from ..consts import TRAINING_STAGE
+from ...utils.json_processor import dump_json_list, load_json_list, dump_json, load_json
+from ...utils.path import get_file_size
+from ...components.metrics.clf_evaluation import get_confusion_matrix, get_classification_report
+from ...utils.statistics import interval_mean, acc
+from .base_arguments_clf import BaseTrainingArgumentsClf
+from .base_data_module_clf import BaseDataModuleClf
+from .base_lightnings_clf import BaseTrainingLightningClf
+from .consts import DevOutput, LabeledSample, InfSampleProto, PredictionResult, TrainingSettings
+from .data_reader_clf import DataReaderClf
+
+class BaseTrainingPipelineClf(BaseTrainingPipeline, metaclass=ABCMeta):
+    
+    _args: BaseTrainingArgumentsClf
+    _mode_name: str = "classification_ft"
+    
+    #############################################################################################
+    ## overwrite
+    #############################################################################################
+    
+    def _before_training(self) -> None:
+        self._logger = self._get_logger()
+        self._logger.info(f"use saving path: {self._output_dir}")
+        self._logger.info(f"log_file: {self._log_file}")
+        self._logger.info("selecting pretrainde model...") 
+        self._select_pretrained_model()
+        self._logger.info("generate tokenizer...")
+        self._tokenizer = self._generate_tokenizer()
+        self._logger.info("loading prompt...")
+        self._prompt = self._load_prompt()
+        
+    def _train(self) -> None:
+        self._data_module = self._get_data_module()
+        self._data_module.reparse_args()
+        self._training_lightning = self._get_training_lightning()
+        self._trainer = self._get_trainer()
+        self._logger.info(f"used gpus: {self._trainer.gpus}")
+        self._fit()
+        
+    def _after_training(self) -> None:
+        self._logger.info("select model from checkpoints...")
+        best_ckpt = self._select_best_model_from_ckpts()
+        if self._args.debug:
+            if self._args.test_data_path is None:
+                self._logger.info(
+                    "test_data_path is not passed, skip testing...")
+            elif not self._args.test_data_path.exists():
+                self._logger.info(f"test_data_path {self._args.test_data_path}"
+                                  f" does not exist, skip testing...")
+            else:
+                self._logger.info("implement test on training model...")
+                self._implement_test(best_ckpt)
+        self._logger.info("generating inference model...")
+        state_dict = self._load_ckpt(best_ckpt).get_model_state_dict()
+        self._inference_lightning = self._get_inf_lightning()
+        self._inference_lightning.load_model_from_state_dict(state_dict) 
+        self._inference_lightning.model.eval()
+        self._logger.info("generate prediction file...")
+        self._generate_prediction_file()
+        if self._args.test_data_path is None:
+            self._logger.info(
+                "test_data_path is not passed, skip testing...")
+        elif not self._args.test_data_path.exists():
+            self._logger.info(f"test_data_path {self._args.test_data_path}"
+                              f" does not exist, skip testing...")
+        else:
+            self._log_test_acc_on_inf_model()
+        self._logger.info("save model files...")
+        torch.save(state_dict, os.path.join(self._output_dir, "finetune_pytorch.bin"))
+        self._logger.info("export model to onnx...")
+        self._export_onnx()
+        self._logger.info("generate and export evaluation results...")
+        self._save_eval_results()
+        self._logger.info("cleaning checkpoints...")
+        for ckpt in self._ckpt_file_list:
+            os.remove(ckpt)
+        if os.path.exists(lightning_logs_path := os.path.join(self._output_dir, "lightning_logs")):
+            shutil.rmtree(lightning_logs_path)
+        self._copy_output_files()
+    
+    #############################################################################################
+    ## private
+    #############################################################################################
+    
+    #############################################################################################
+    ##################################### before training #######################################
+    
+    def _get_logger(self) -> Logger:
+        now_date = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
+        self._task_name = f"{self._mode_name}_{self._args.dataset}_{now_date}"
+        self._log_file = os.path.join(self._args.log_dir, self._task_name + ".log")
+        LoggerManager.set_logger(self._args.logger, self._log_file)
+        return LoggerManager.get_logger(self._args.logger)
+       
+    @property
+    def _output_dir(self) -> DirectoryPath:
+        """输出地址"""
+        return self._args.ft_output_dir
+    
+    def _select_pretrained_model(self): # todo 优化可读性
+        if self._args.selected_pretrained_model_dir is not None and os.path.exists(self._args.selected_pretrained_model_dir):
+            pretrained_model_dir = self._args.selected_pretrained_model_dir
+            self._logger.info(f"using selected pretrained model: {pretrained_model_dir}")
+        else:
+            tapt_output_dir = os.path.join(self._args.student_output_dir, "tapt_output")
+            if os.path.exists(tapt_output_dir) and set(["config.json", "pytorch_model.bin", "vocab.txt"]) <= set(os.listdir(tapt_output_dir)):
+                pretrained_model_dir = tapt_output_dir
+                self._logger.info(f"using tapt output as pretrained model: {pretrained_model_dir}")
+            else:
+                label2id = DataReaderClf.read_label2id(self._args.label2id_path)
+                if len(label2id) == 2:
+                    model = "Erlangshen-MacBERT-110M-BinaryClassification-Chinese"
+                else:
+                    sample_list = list(DataReaderClf.read_unlabeled_sample(self._args.train_data_path))
+                    sentence_len_list = [len(sample.text) for sample in sample_list]
+                    if interval_mean(sentence_len_list) < 100:
+                        model = "chinese-macbert-base"
+                    else:
+                        model = "ernie-1.0-base-zh"
+                pretrained_model_dir = os.path.join(self._args.pretrained_model_root, model)
+                self._logger.info(f"using auto selected pretrained model: {pretrained_model_dir}")
+        self._args.pretrained_model_dir = Path(pretrained_model_dir)
+        
+    def _generate_tokenizer(self) -> PreTrainedTokenizer:
+        if not os.path.exists(self._args.pretrained_model_dir):
+            pretrained_model_dir, model_name = os.path.split(self._args.pretrained_model_dir)
+            if model_name=="chinese-macbert-base":
+                huggingface_model_name = "hfl/chinese-macbert-base"
+            elif model_name=="ernie-1.0-base-zh":
+                huggingface_model_name = "xiaoqin/ernie-1.0-base-zh"
+            else:
+                huggingface_model_name = "IDEA-CCNL/Erlangshen-MacBERT-110M-BinaryClassification-Chinese"
+            cache_path = os.path.join(pretrained_model_dir, model_name, "cache")
+            tokenizer = AutoTokenizer.from_pretrained(huggingface_model_name, cache_dir=cache_path)
+            model = AutoModelForMaskedLM.from_pretrained(huggingface_model_name, cache_dir=cache_path)
+            model.save_pretrained(os.path.join(pretrained_model_dir, model_name))
+            tokenizer.save_pretrained(os.path.join(pretrained_model_dir, model_name))
+            shutil.rmtree(cache_path)
+            self._logger.info("local pretrained model path does not exist, model %s is downloaded from huggingface." % huggingface_model_name)
+        return TokenizerGenerator.generate_tokenizer(self._args.pretrained_model_dir)
+    
+    def _load_prompt(self):
+        try:
+            assert(self._args.label2id_path is not None)
+        except:
+            raise Exception("no label2id path is passed")
+        return StdPrompt(self._args.prefix_prompt, self._args.label2id_path)
+    
+    #############################################################################################
+    ##################################### training ###########################################
+        
+    def _get_lightning_loggers(self) -> List[PlLogger]:
+        tb_logger = TensorBoardLogger(
+            save_dir=str(self._args.log_dir),
+            name="tensor_board_logs",
+            version=self._task_name,
+            max_queue=2, 
+            flush_secs=5
+        )
+        return [tb_logger]
+    
+    #############################################################################################
+    ##################################### after training #############################################
+    
+    def _select_best_model_from_ckpts(self) -> str:
+        """从保存的多个checkpoints中选择在全样本验证数据中表现最好的，返回checkpoint路径"""
+        ckpt_file_list = self._ckpt_file_list
+        if len(ckpt_file_list) == 0:
+            raise Exception("no checkpoint is saved")
+        if len(ckpt_file_list) == 1:
+            return ckpt_file_list[0]
+        acc_ckpt_tuple_list: List[Tuple[float, str]] = []
+        for ckpt in ckpt_file_list:
+            dev_output = self._dev_from_ckpt(ckpt, TRAINING_STAGE.VALIDATION)
+            acc_ckpt_tuple_list.append((dev_output.dev_acc, ckpt))
+        best_ckpt_tuple = max(acc_ckpt_tuple_list, key=lambda tp: tp[0])
+        self._logger.info(f"best checkpoint dev acc: {best_ckpt_tuple[0]:.4f}")
+        return best_ckpt_tuple[1]
+    
+    @property
+    def _ckpt_file_list(self) -> List[str]:
+        file_list = os.listdir(self._output_dir)
+        ckpt_file_list = [os.path.join(self._output_dir, file) for file in file_list if re.match(r"^.*\.ckpt$", file) is not None]
+        return ckpt_file_list
+    
+    def _dev_from_ckpt(self, checkpoint_path: str, stage: Literal[TRAINING_STAGE.TEST, TRAINING_STAGE.VALIDATION]) -> DevOutput:
+        """输出对应checkpoint的验证/测试结果"""
+        model = self._load_ckpt(checkpoint_path)
+        data_loader = self._data_module.test_dataloader(stage=TRAINING_STAGE.TEST) if stage == TRAINING_STAGE.TEST else self._data_module.val_dataloader(stage=TRAINING_STAGE.VALIDATION, load_ratio=self._args.load_data_ratio)
+        eval_output = self._trainer.validate(
+            model=model, 
+            dataloaders = data_loader,
+            verbose=False
+        )
+        dev_acc = eval_output[0]["dev_acc"]
+        dev_loss = eval_output[0]["dev_loss"]
+        return DevOutput(dev_loss=dev_loss, dev_acc=dev_acc)
+    
+    def _load_ckpt(self, checkpoint_path: str) -> BaseTrainingLightningClf:
+        return type(self._training_lightning).load_from_checkpoint(
+                checkpoint_path,
+                args=self._args,
+                class_num=len(self._prompt.label2token),
+                sample_num=self._data_module.train_sample_num
+            )
+    
+    def _implement_test(self, ckpt: str) -> None:
+        dev_output = self._dev_from_ckpt(ckpt, TRAINING_STAGE.TEST)
+        self._logger.info(f"training model test acc: {dev_output.dev_acc:.4f}")
+
+    def _generate_prediction_file(self):
+        prediction_trainer = Trainer(
+            accelerator="gpu",
+            devices=1,
+            default_root_dir=str(self._output_dir),
+            enable_progress_bar=False,
+            auto_select_gpus=True
+        )
+        if self._args.dev_data_path is None:
+            self._logger.info(
+                "dev_data_path is not passed, skip predicting on dev data...")
+        elif not self._args.dev_data_path.exists():
+            self._logger.info(f"dev_data_path {self._args.dev_data_path} does "
+                              f"not exist, skip predicting on dev data...")
+        else:
+            self._logger.info("predicting on dev data..")
+            dev_prediction_output = prediction_trainer.predict(
+                model=self._inference_lightning,
+                dataloaders=self._data_module.val_dataloader(stage=TRAINING_STAGE.INFERENCE)
+            )
+            dev_results = self._generate_prediction_results(dev_prediction_output, self._data_module.dev_sample_list) # type: ignore
+            dump_json_list(dev_results, os.path.join(self._output_dir, "test_prediction_results.json"))
+        if self._args.test_data_path is None:
+            self._logger.info(
+                "test_data_path is not passed, skip predicting on test data...")
+        elif not self._args.test_data_path.exists():
+            self._logger.info(f"test_data_path {self._args.test_data_path} does "
+                              f"not exist, skip predicting on test data...")
+        else:
+            self._logger.info("predicting on test data..")
+            test_prediction_output = prediction_trainer.predict(
+                model=self._inference_lightning,
+                dataloaders=self._data_module.test_dataloader(stage=TRAINING_STAGE.INFERENCE)
+            )
+            test_results = self._generate_prediction_results(test_prediction_output, self._data_module.test_sample_list) # type: ignore
+            dump_json_list(test_results, os.path.join(self._output_dir, "offline_test_prediction_results.json"))
+        if self._args.online_test_data_path is None:
+            self._logger.info(
+                "online_test_data_path is not passed, skip predicting on online test data...")
+        elif not self._args.online_test_data_path.exists():
+            self._logger.info(f"online_test_data_path {self._args.online_test_data_path} does "
+                              f"not exist, skip predicting on online test data...")
+        else:
+            self._logger.info("predicting on online test data..")
+            online_test_prediction_output = prediction_trainer.predict(
+                model=self._inference_lightning,
+                dataloaders=self._data_module.online_test_dataloader()
+            )
+            online_test_results = self._generate_prediction_results(online_test_prediction_output, self._data_module.online_test_sample_list) # type: ignore
+            dump_json_list(online_test_results, os.path.join(self._output_dir, "online_test_prediction_results.json"))
+
+    def _generate_prediction_results(self, prediction_output: Dict[int, str], sample_list: Union[List[InfSampleProto], List[LabeledSample]]) -> List[PredictionResult]:
+        assert len(prediction_output) == len(sample_list)
+        y_true = []
+        y_pred = []
+        result_list = []
+        id2label = {key: val.label for key, val in self._prompt.id2label.items()}
+        label2id = {val: key for key, val in id2label.items()}
+        for idx, sample in enumerate(sample_list):
+            prediction = prediction_output[idx]
+            content = sample.text
+            true_label = self._prompt.id2label[sample.label_id_clf].label if hasattr(sample, "label_id_clf") else None # type: ignore
+            result_list.append(PredictionResult(
+                id=sample.id,
+                content=content,
+                label=true_label,
+                predict=prediction
+            ))
+            if hasattr(sample, "label_id_clf"):
+                y_true.append(sample.label_id_clf)  # type: ignore
+            y_pred.append(label2id[prediction])
+        if len(y_true)>0:
+            clf_report = get_classification_report(y_true, y_pred, id2label)
+            accuracy = clf_report.pop('accuracy')
+            self._logger.info(f"acc is: {accuracy:.4f}")
+        return result_list
+            
+    def _export_onnx(self) -> None:
+        batch_size = 2
+        seq_len = 64
+        dummy_input_ids = torch.ones([batch_size, seq_len],
+                                     dtype=torch.long).cuda(0)
+        dummy_input_mask = torch.ones([batch_size, seq_len],
+                                      dtype=torch.long).cuda(0)
+        dummy_input_seg = torch.ones([batch_size, seq_len],
+                                     dtype=torch.long).cuda(0)
+        input_names = ['input_ids', 'input_mask', 'input_seg']
+        output_names = ['pred_labels', 'probs']
+        dynamic_axes = {
+            'input_ids': {
+                0: 'batch_size',
+                1: 'seq_length'
+            },
+            'input_mask': {
+                0: 'batch_size',
+                1: 'seq_length'
+            },
+            'input_seg': {
+                0: 'batch_size',
+                1: 'seq_length'
+            },
+            'pred_labels': {
+                0: 'batch_size'
+            },
+            'probs': {
+                0: 'batch_size',
+            }
+        }
+        self._inference_lightning.model.eval()
+        torch.onnx.export(
+            self._inference_lightning.model.cuda(0), 
+            (dummy_input_ids, dummy_input_mask, dummy_input_seg),
+            os.path.join(self._output_dir, "model.onnx"),
+            verbose=False,
+            opset_version=12,
+            input_names=input_names,
+            output_names=output_names,
+            dynamic_axes=dynamic_axes)
+        self._logger.info('Save ONNX model to {} successfully!'.format(os.path.join(self._output_dir, "model.onnx")))
+
+    def _save_eval_results(self) -> None:
+        settings = TrainingSettings(
+            learning_rate=self._args.learning_rate,
+            label_guided_rate=self._args.label_guided_rate,
+            prefix_prompt=self._args.prefix_prompt,
+            training_label_prompt=self._args.training_label_prompt,
+            inference_label_prompt=self._args.inference_label_prompt,
+            batch_size=self._args.train_batch_size,
+            epoch=self._args.epoch,
+            warmup_epoch=self._args.warm_up_epoch,
+            decay_epoch=self._args.decay_epoch,
+            dropout_rate=0.1
+        )
+        model_size = get_file_size(os.path.join(self._output_dir, "model.onnx"), "mb")
+        if os.path.exists(os.path.join(self._output_dir, "test_prediction_results.json")):
+            y_true, y_pred, id2label = self._load_test_results(os.path.join(self._output_dir, "test_prediction_results.json"))
+            confusion_matrix = get_confusion_matrix(y_true, y_pred, id2label)
+            clf_report = get_classification_report(y_true, y_pred, id2label)
+            accuracy = clf_report.pop('accuracy')
+            macro_avg = clf_report.pop("macro avg")
+            weighted_avg = clf_report.pop("weighted avg")
+            eval_results = {
+                "global_indicator": {
+                    "accuracy": accuracy,
+                    "macro_avg": macro_avg,
+                    "weighted_avg": weighted_avg
+                },
+                "settings": asdict(settings),
+                "confusion_matrix": asdict(confusion_matrix),
+                "label_result": clf_report,
+                "model_size": model_size
+            }
+        else:
+            eval_results = {
+                "global_indicator": {
+                    "accuracy": 0,
+                    "macro_avg": 0,
+                    "weighted_avg": 0
+                },
+                "settings": asdict(settings),
+                "confusion_matrix": None,
+                "label_result": None,
+                "model_size": model_size
+            }
+        dump_json(eval_results, os.path.join(self._output_dir, "result.json"), indent=2)
+        
+    def _load_test_results(self, path: str):
+        test_results_list = list(load_json_list(path, PredictionResult))
+        id2label = {key: val.label for key, val in self._prompt.id2label.items()}
+        label2id = {val: key for key, val in id2label.items()}
+        y_true = [label2id[result.label] for result in test_results_list] # type: ignore
+        y_pred = [label2id[result.predict] for result in test_results_list]
+        return y_true, y_pred, id2label
+    
+    def _log_test_acc_on_inf_model(self) -> None:
+        y_true, y_pred, _ = self._load_test_results(os.path.join(self._output_dir, "offline_test_prediction_results.json"))
+        acc_ = acc(y_true, y_pred)
+        self._logger.info(f"inference model test acc: {acc_:.4f}")
+    
+    def _copy_output_files(self) -> None:
+        shutil.copy(os.path.join(self._args.pretrained_model_dir, "vocab.txt"), os.path.join(self._output_dir, "vocab.txt"))
+        shutil.copy(os.path.join(self._args.pretrained_model_dir, "config.json"), os.path.join(self._output_dir, "config.json"))
+        shutil.copy(self._log_file, os.path.join(self._output_dir, "training_log.log"))
+        shutil.copy(os.path.join(self._output_dir, "result.json"), os.path.join(self._args.student_output_dir, "result.json"))
+        shutil.copy(os.path.join(self._output_dir, "model.onnx"), os.path.join(self._args.student_output_dir, "model.onnx"))
+        shutil.copy(os.path.join(self._output_dir, "vocab.txt"), os.path.join(self._args.student_output_dir, "vocab.txt"))
+        shutil.copy(os.path.join(self._output_dir, "config.json"), os.path.join(self._args.student_output_dir, "config.json"))
+        if os.path.exists(os.path.join(self._output_dir, "test_prediction_results.json")):
+            shutil.copy(os.path.join(self._output_dir, "test_prediction_results.json"), os.path.join(self._args.student_output_dir, "test_prediction_results.json"))
+    
+    
+    #############################################################################################
+    ## abstract 
+    #############################################################################################
+    
+    @abstractmethod
+    def _get_data_module(self) -> BaseDataModuleClf: 
+        """根据模式需要处理和加载数据"""
+        
+    @abstractmethod
+    def _get_training_lightning(self) -> BaseTrainingLightningClf: 
+        """根据模式需要加载训练模型"""
+        
+    @abstractmethod
+    def _get_trainer(self) -> Trainer:
+        """根据模式需要定义和加载Lightning Trainer"""
+        
+    @abstractmethod
+    def _fit(self) -> None:
+        """根据模式需要定义和实施训练"""
+        
+    @abstractmethod
+    def _get_inf_lightning(self) -> BaseTrainingLightningClf:
+        """根据模式需要加载推理模型"""
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/consts.py b/gts_engine/bagualu/lib/framework/classification_finetune/consts.py
new file mode 100644
index 0000000..497cd5b
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/consts.py
@@ -0,0 +1,178 @@
+from dataclasses import dataclass, field
+from torch import Tensor
+from typing import List, Optional, Protocol, TypedDict, Dict, NamedTuple
+from torch import Tensor
+from transformers.tokenization_utils import PreTrainedTokenizer
+from pydantic import BaseModel, Field
+
+#############################################################################################
+## Sample & Data
+#############################################################################################
+
+class RawSample(BaseModel):
+    """数据在json文件中储存的字段"""
+    content: str
+    id: int
+    label: Optional[str] = None
+    probs: List[float] = Field(default_factory=list) # 默认为空列表，dataclass不允许默认值为动态数据结构[]，故采用这种方式
+    
+@dataclass
+class LabeledSample:
+    """数据被加载并初步处理后的字段"""
+    text: str
+    id: int
+    label: str
+    label_id: int
+    label_id_clf: int
+    soft_label: List[float] = field(default_factory=list)
+
+@dataclass
+class UnlabeledSample:
+    """无标注数据"""
+    text: str
+    id: int
+    
+@dataclass
+class PreEncodedTrainSample:
+    text: str
+    id: int
+    label: str
+    label_id: int
+    label_id_clf: int
+    words: List[str]
+    inference_prompt: List[str]
+    training_prompt: List[str]
+    soft_label: List[float] = field(default_factory=list)
+
+@dataclass
+class EncodedTrainSample:
+    input_ids: Tensor
+    input_seg: Tensor
+    input_mask: Tensor
+    mask_positions: Tensor
+    labels: Tensor
+    label_id: Tensor
+    label_id_clf: Tensor
+    id: int
+    """数据集自带id"""
+    weight: Tensor
+    soft_label: Tensor
+    seq_len: int
+    my_id: int
+    """自定义辅助id"""
+
+@dataclass
+class EncodedInfSample:
+    input_ids: Tensor
+    input_seg: Tensor
+    input_mask: Tensor
+    my_id: int
+    """自定义辅助id"""
+    
+class InfSampleProto(Protocol):
+    """用于推理的数据格式"""
+    id: int
+    text: str
+
+class InferenceManagerInputSample(BaseModel):
+    text: str
+
+InferenceManagerInputSampleList = List[InferenceManagerInputSample]
+
+#############################################################################################
+## Prompt & Label
+#############################################################################################
+
+class PromptToken(NamedTuple):
+    label: str
+    label_id: int
+    label_id_clf: int
+    key: str
+
+class PromptLabel(NamedTuple):
+    label: str
+    key: str
+
+Label2Token = Dict[str, PromptToken]
+
+class PromptLabelParseOutput(NamedTuple):
+    label2token: Label2Token
+    id2label: Dict[int, PromptLabel]
+    label_ids: List[PromptToken]
+
+class Label2IdValue(BaseModel):
+    id: int
+    label_desc_en: Optional[str] = None
+    label_desc_zh: Optional[str] = None
+    
+Label2Id = Dict[str, Label2IdValue]
+
+#############################################################################################
+## In / Out
+#############################################################################################
+class TrainingModelOutput(TypedDict):
+    loss_total: Tensor
+    loss_ce: Tensor
+    loss_mlm: Tensor
+    kl_loss: Tensor
+    logits: Tensor
+    loss_rd: Tensor
+    loss_ctr: Tensor
+    loss_ner: Tensor
+    
+class InferenceModelOutput(TypedDict):
+    positions: Tensor
+    probs: Tensor
+    embeds: Tensor
+
+@dataclass
+class DevOutput:
+    dev_loss: float
+    dev_acc: float
+    
+@dataclass
+class PredictionResult:
+    id: int
+    content: str
+    predict: str
+    label: Optional[str] = None
+
+@dataclass
+class TrainingSettings:
+    learning_rate: float
+    label_guided_rate: float
+    prefix_prompt: str
+    training_label_prompt: str
+    inference_label_prompt: str
+    batch_size: int
+    epoch: int
+    warmup_epoch: int
+    decay_epoch: int
+    dropout_rate: float
+
+class InferenceManagerOutput(TypedDict):
+    predictions: List[str]
+    probabilities: List[List[float]]
+    
+class InfBatch(TypedDict):
+    input_ids: Tensor
+    input_seg: Tensor
+    input_mask: Tensor
+    my_id: List[int]
+    """自定义辅助id"""
+    
+class TrainBatch(TypedDict):
+    input_ids: Tensor
+    input_seg: Tensor
+    input_mask: Tensor
+    mask_positions: Tensor
+    labels: Tensor
+    label_id: Tensor
+    label_id_clf: Tensor
+    id: List[int]
+    """数据集自带id"""
+    weight: Tensor
+    soft_label: Tensor
+    seq_len: Tensor
+    my_id: List[int]
+    """自定义辅助id"""
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/data_reader_clf.py b/gts_engine/bagualu/lib/framework/classification_finetune/data_reader_clf.py
new file mode 100644
index 0000000..d0c9134
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/data_reader_clf.py
@@ -0,0 +1,46 @@
+from pathlib import Path
+from typing import Union, Iterable
+
+from ...utils.json_processor import load_json_list, load_json
+from .consts import (
+    Label2Token, 
+    LabeledSample, 
+    UnlabeledSample, 
+    Label2Id, 
+    Label2IdValue, 
+    RawSample
+)
+
+Path_ = Union[str, Path]
+
+class DataReaderClf:
+    
+    @classmethod
+    def read_labeled_sample(cls, path: Path_, label2token: Label2Token) -> Iterable[LabeledSample]:
+        for raw_sample in cls.__read_raw_clf_sample(path):
+            label_key = str(raw_sample.label)
+            label = label2token[label_key].label
+            label_id = label2token[label_key].label_id
+            label_id_clf = label2token[label_key].label_id_clf
+            labeled_sample = LabeledSample(
+                text=raw_sample.content,
+                id=raw_sample.id,
+                label=label,
+                label_id=label_id,
+                label_id_clf=label_id_clf,
+                soft_label=raw_sample.probs
+            )
+            yield labeled_sample
+    
+    @classmethod
+    def read_unlabeled_sample(cls, path: Path_) -> Iterable[UnlabeledSample]:
+        for raw_sample in cls.__read_raw_clf_sample(path):
+            yield UnlabeledSample(raw_sample.content, raw_sample.id)
+            
+    @classmethod
+    def read_label2id(cls, path: Path_) -> Label2Id:
+        return load_json(path, Label2Id)
+        
+    @classmethod
+    def __read_raw_clf_sample(cls, path: Path_) -> Iterable[RawSample]:
+        return load_json_list(path, RawSample)
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/mask_tools.py b/gts_engine/bagualu/lib/framework/classification_finetune/mask_tools.py
new file mode 100644
index 0000000..e0fa54c
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/mask_tools.py
@@ -0,0 +1,179 @@
+import numpy as np
+from typing import List, NamedTuple
+from transformers.tokenization_utils import PreTrainedTokenizer
+import re
+
+from .prompt import PromptBase
+
+class MaskedLmInstance(NamedTuple):
+    index: int
+    label: str
+
+class CreateMaskedLmPredictionsOutput(NamedTuple):
+    output_tokens: List[str]
+    masked_lm_positions: List[int]
+    masked_lm_labels: List[str]
+    
+
+def create_masked_lm_predictions(tokens: List[str],
+                                 masked_lm_prob: float,
+                                 max_predictions_per_seq: int,
+                                 vocab_words: List[str],
+                                 index: int,
+                                 masked_label: str='[MASK]',
+                                 do_wwm: bool=True):
+    """Creates the predictions for the masked LM objective."""
+
+    rng = np.random.RandomState(seed=((1234 + index) % 2 ** 32))
+
+    cand_indexes = []
+    for (i, token) in enumerate(tokens):
+        if token == "[CLS]" or token == "[SEP]" or token == '[MASK]':
+            continue
+        # Whole Word Masking means that if we mask all of the wordpieces
+        # corresponding to an original word. When a word has been split into
+        # WordPieces, the first token does not have any marker and any subsequence
+        # tokens are prefixed with ##. So whenever we see the ## token, we
+        # append it to the previous set of word indexes.
+        #
+        # Note that Whole Word Masking does *not* change the training code
+        # at all -- we still predict each WordPiece independently, softmaxed
+        # over the entire vocabulary.
+        if (do_wwm and len(cand_indexes) >= 1 and
+                token.startswith("##")):
+            cand_indexes[-1].append(i)
+        else:
+            cand_indexes.append([i])
+
+    rng.shuffle(cand_indexes)
+
+    # if non chinese is False, that means it is chinese, then try to remove "##" which is added previously
+    output_tokens = [t[2:] if len(re.findall(
+        '##[\u4E00-\u9FA5]', t)) > 0 else t for t in tokens]  # 去掉"##"
+    num_to_predict = min(max_predictions_per_seq,
+                         max(1, int(round(len(tokens) * masked_lm_prob))))
+
+    masked_lms: List[MaskedLmInstance] = []
+    covered_indexes = set()
+    for index_set in cand_indexes:
+        if len(masked_lms) >= num_to_predict:
+            break
+        # If adding a whole-word mask would exceed the maximum number of
+        # predictions, then just skip this candidate.
+        if len(masked_lms) + len(index_set) > num_to_predict:
+            continue
+        is_any_index_covered = False
+        for index in index_set:
+            if index in covered_indexes:
+                is_any_index_covered = True
+                break
+        if is_any_index_covered:
+            continue
+        for index in index_set:
+            covered_indexes.add(index)
+
+            masked_token = None
+            # 80% of the time, replace with [MASK]
+            if rng.random() < 0.8:
+                masked_token = masked_label
+            else:
+                # 10% of the time, keep original
+                if rng.random() < 0.5:
+                    masked_token = tokens[index][2:] if len(re.findall('##[\u4E00-\u9FA5]', tokens[index])) > 0 \
+                        else tokens[index]  # 去掉"##"
+                # 10% of the time, replace with random word
+                else:
+                    masked_token = vocab_words[rng.randint(
+                        0, len(vocab_words) - 1)]
+
+            output_tokens[index] = masked_token
+
+            masked_lms.append(MaskedLmInstance(
+                index=index, label=tokens[index]))
+    assert len(masked_lms) <= num_to_predict
+    masked_lms = sorted(masked_lms, key=lambda x: x.index)
+
+    masked_lm_positions: List[int] = []
+    masked_lm_labels: List[str] = []
+    for p in masked_lms:
+        masked_lm_positions.append(p.index)
+        masked_lm_labels.append(p.label)
+
+    return CreateMaskedLmPredictionsOutput(output_tokens, masked_lm_positions, masked_lm_labels)
+
+class WWMMaskingOutput(NamedTuple):
+    input_ids: List[int]
+    mlm_labels: List[int]
+    input_mask: List[int]
+    masked_lm_positions: List[int]
+
+def wwm_masking(new_tokens: List[str], index: int, tokenizer: PreTrainedTokenizer, prompt_mode: PromptBase, max_length: int, mask_rate: float, offset: int=0):
+    output_tokens, masked_lm_positions, masked_lm_labels = create_masked_lm_predictions(new_tokens[offset:],
+                                                                                        masked_lm_prob=mask_rate,
+                                                                                        max_predictions_per_seq=30,
+                                                                                        masked_label=prompt_mode.mask_token,
+                                                                                        vocab_words=list(tokenizer.get_vocab().keys()),
+                                                                                        index=index)
+
+    masked_lm_positions = [ele + offset for ele in masked_lm_positions]
+    input_ids = tokenizer.convert_tokens_to_ids(
+        new_tokens[:offset] + output_tokens)
+    assert isinstance(input_ids, list) and len(input_ids) <= max_length
+    input_mask = [1] * len(input_ids)
+    assert len(input_mask) <= max_length
+    while len(input_ids) < max_length:
+        input_ids.append(0)
+        input_mask.append(0)
+    assert len(input_ids) == max_length
+    assert len(input_mask) == max_length
+
+    masked_lm_ids = tokenizer.convert_tokens_to_ids(masked_lm_labels)
+    mlm_labels = [-100] * max_length
+    assert isinstance(masked_lm_ids, list)
+    for pos, masked_id in zip(masked_lm_positions, masked_lm_ids):
+        mlm_labels[pos] = masked_id
+
+    return WWMMaskingOutput(input_ids, mlm_labels, input_mask, masked_lm_positions)
+
+
+# TODO: 存在问题，之后再来改
+# def random_masking(token_ids, max_length, mask_rate, prompt_mode, tokenizer=None):
+#     """
+#     对输入进行随机mask
+#     """
+#     rands = np.random.random(len(token_ids))
+#     source, target, mask_pos = [], [], []
+#     #mask_token = [prompt_mode.mask_token]
+#     #mask_id = tokenizer.convert_tokens_to_ids(mask_token)[0]
+
+#     # 删除-CLS SEP id
+#     mask_id = tokenizer.convert_tokens_to_ids(['[MASK]'])[0]
+#     cls_id = tokenizer.convert_tokens_to_ids(["[CLS]"])[0]
+#     sep_id = tokenizer.convert_tokens_to_ids(["[SEP]"])[0]
+
+#     for i, (r, t) in enumerate(zip(rands, token_ids)):
+#         if t == cls_id or t == sep_id:
+#             source.append(t)
+#             target.append(-100)
+#             continue
+
+#         if r < mask_rate * 0.8:
+#             source.append(mask_id)
+#             target.append(t)
+#             mask_pos.append(i)
+#         elif r < mask_rate * 0.9:
+#             source.append(t)
+#             target.append(t)
+#             mask_pos.append(i)
+#         elif r < mask_rate:
+#             source.append(np.random.choice(len(tokenizer.get_vocab().keys()) - 2) + 1)
+#             target.append(t)
+#             mask_pos.append(i)
+#         else:
+#             source.append(t)
+#             target.append(-100)
+    
+#     while len(source) < max_length:
+#         source.append(0)
+#         target.append(-100)
+#     return source[:max_length], target[:max_length], mask_pos
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/classification_finetune/prompt.py b/gts_engine/bagualu/lib/framework/classification_finetune/prompt.py
new file mode 100644
index 0000000..f1194e3
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/classification_finetune/prompt.py
@@ -0,0 +1,49 @@
+from typing import Any, Dict, NamedTuple, Union, List
+from abc import abstractmethod, ABCMeta
+from pydantic import FilePath
+
+from .consts import PromptLabelParseOutput, PromptToken, PromptLabel
+from .data_reader_clf import DataReaderClf
+
+class PromptBase(metaclass=ABCMeta):
+    
+    def __init__(self, prompt: str, label2id_path: Union[str, FilePath]) -> None:
+        self._mask_id = 103
+        self._mask_token = '[MASK]'
+        self._prompt = prompt
+        self._label2id_path = label2id_path
+        self._label_des2tag = DataReaderClf.read_label2id(label2id_path)
+        self.parse_label()
+    
+    @abstractmethod
+    def parse_label(self) -> PromptLabelParseOutput:
+        pass
+    
+    @property
+    def prompt(self):
+        return self._prompt
+    
+    @property
+    def mask_token(self):
+        return self._mask_token
+
+class StdPrompt(PromptBase):
+    """通用Prompt"""
+    def parse_label(self) -> PromptLabelParseOutput:
+        label2token: Dict[str, PromptToken] = {}
+        id2label: Dict[int, PromptLabel] = {}
+        for i, item in enumerate(self._label_des2tag.items()):
+            key, val = item
+            if val.label_desc_zh is None:
+                raise Exception("label2id has no field \"label_desc_zh\"")
+            label = val.label_desc_zh
+            id = val.id
+            label2token[key] = PromptToken(label, i, id, key)
+            id2label[id] = PromptLabel(label, key)
+            
+        label_ids = [ele for ele in label2token.values()]
+        label_ids.sort()
+        self.label2token = label2token
+        self.id2label = id2label
+        self.label_ids = label_ids
+        return PromptLabelParseOutput(label2token, id2label, label_ids)
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/consts.py b/gts_engine/bagualu/lib/framework/consts.py
new file mode 100644
index 0000000..407bef9
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/consts.py
@@ -0,0 +1,39 @@
+from typing import NewType, NamedTuple, Dict, List, Optional, Protocol, Union
+from typing_extensions import TypedDict
+from enum import Enum
+from torch import Tensor
+
+#############################################################################################
+## Types
+#############################################################################################
+
+class BertInput(TypedDict):
+    input_ids: Tensor
+    attention_mask: Tensor
+    token_type_ids: Tensor
+    labels: Optional[Tensor]
+
+#############################################################################################
+## Enums
+#############################################################################################
+
+class TRAIN_MODE(Enum):
+    DEFAULT = "0"
+    STUDENT = "1"
+    GTS = "2"
+    FAST = "3"
+
+class RUN_MODE(Enum):
+    OFFLINE = "offline"
+    ONLINE = "online"
+    
+class TASK(Enum):
+    SINGLE_SENTENCE_CLF = 1
+    SENTENCE_CLF = 2
+    MULTI_TASK = 3
+    
+class TRAINING_STAGE(Enum):
+    TRAINING = 1
+    TEST = 2
+    VALIDATION = 3
+    INFERENCE = 4
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/mixin/__init__.py b/gts_engine/bagualu/lib/framework/mixin/__init__.py
new file mode 100644
index 0000000..00238fb
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/mixin/__init__.py
@@ -0,0 +1,2 @@
+from .optional_logger_mixin import OptionalLoggerMixin
+from .args_mixin import ArgsMixin
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/mixin/args_mixin.py b/gts_engine/bagualu/lib/framework/mixin/args_mixin.py
new file mode 100644
index 0000000..2e5665d
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/mixin/args_mixin.py
@@ -0,0 +1,13 @@
+from typing import Type, Optional, List, get_type_hints
+
+from ..base_arguments import BaseArguments
+
+class ArgsMixin:
+    
+    def __init__(self, args_parse_list: Optional[List[str]] = None):
+        try:
+            arg_cls: Type[BaseArguments] = get_type_hints(type(self))["_args"]
+        except:
+            raise Exception("BaseTrainingPipeline should have property typing hint \"_args\" to indicate the BaseArguments class")
+        arg_obj = arg_cls()
+        self._args = arg_obj.parse_args(args_parse_list)
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/framework/mixin/optional_logger_mixin.py b/gts_engine/bagualu/lib/framework/mixin/optional_logger_mixin.py
new file mode 100644
index 0000000..8a09ec2
--- /dev/null
+++ b/gts_engine/bagualu/lib/framework/mixin/optional_logger_mixin.py
@@ -0,0 +1,13 @@
+from typing import Optional
+import logging
+
+class OptionalLoggerMixin:
+    """支持选择性输出到logger或print的mixin"""
+    def __init__(self, logger_name: Optional[str] = None):
+        self._logger = logging.getLogger(logger_name) if logger_name else None
+    
+    def info(self, info: str):
+        if hasattr(self, "_logger") and self._logger:
+            self._logger.info(info)
+        else:
+            print(info)
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/utils/__init__.py b/gts_engine/bagualu/lib/utils/__init__.py
new file mode 100644
index 0000000..0c6d777
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/__init__.py
@@ -0,0 +1,1 @@
+from .logger_manager import LoggerManager
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/utils/detect_gpu_memory.py b/gts_engine/bagualu/lib/utils/detect_gpu_memory.py
new file mode 100644
index 0000000..017c0ee
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/detect_gpu_memory.py
@@ -0,0 +1,43 @@
+from pynvml import *
+
+
+def print_gpu_utilization():
+    nvmlInit()
+    handle = nvmlDeviceGetHandleByIndex(0)
+    info = nvmlDeviceGetMemoryInfo(handle)
+    print(f"GPU memory occupied: {info.used//1024**2} MB.")
+
+
+def print_summary(result):
+    print(f"Time: {result.metrics['train_runtime']:.2f}")
+    print(f"Samples/second: {result.metrics['train_samples_per_second']:.2f}")
+    print_gpu_utilization()
+
+
+def detect_gpu_memory():
+    nvmlInit()
+    handle = nvmlDeviceGetHandleByIndex(0)
+    info = nvmlDeviceGetMemoryInfo(handle)
+    # print(info)
+
+    return info.total//1024**2, info.used//1024**2
+
+def decide_gpu(gpu_memory):
+    lower_gpu = 13000
+    low_gpu = 25000
+    mid_gpu = 33000
+    high_gpu = 41000
+
+    if gpu_memory<= lower_gpu:
+        return "lower_gpu" #eg：3060
+    elif gpu_memory> lower_gpu and gpu_memory <= low_gpu:
+        return "low_gpu"  #eg：3090
+    elif gpu_memory > low_gpu and gpu_memory < mid_gpu: 
+        return "mid_gpu"  #eg：v100
+    elif gpu_memory >= mid_gpu:
+        return "high_gpu" #eg：a100
+
+if __name__=="__main__":
+    print_gpu_utilization()
+    gpu_memory, gpu_used_memory = detect_gpu_memory()
+    print(gpu_memory, gpu_used_memory)
diff --git a/gts_engine/bagualu/lib/utils/json_processor.py b/gts_engine/bagualu/lib/utils/json_processor.py
new file mode 100644
index 0000000..9e169e8
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/json_processor.py
@@ -0,0 +1,45 @@
+import json
+from dataclasses import asdict
+from pathlib import Path
+from pydantic import BaseModel, parse_obj_as
+from pydantic.json import pydantic_encoder
+from typing import List, Union, TypeVar, Type, Any, Optional, Tuple, Dict, Iterable, overload
+
+LoadType = TypeVar("LoadType", bound=Type)
+
+@overload
+def load_json(file: Union[str, Path]) -> Any: ...
+
+@overload
+def load_json(file: Union[str, Path], type_: Type[LoadType]) -> LoadType: ...
+
+@overload
+def load_json_list(file: Union[str, Path]) -> Iterable[Any]: ...
+
+@overload
+def load_json_list(file: Union[str, Path], type_: Type[LoadType]) -> Iterable[LoadType]: ...
+
+def dump_json(obj: Any, file: Union[str, Path], indent: Optional[int] = None) -> None:
+    with open(file, 'w', encoding='utf-8') as f:
+        json.dump(obj, f, indent=indent, ensure_ascii=False, default=pydantic_encoder)
+
+def dump_json_list(obj: List[Any], file: Union[str, Path]) -> None:
+    with open(file, 'w', encoding='utf-8') as f:
+        f.writelines([json.dumps(content, ensure_ascii=False, default=pydantic_encoder) + "\n" for content in obj])
+
+def load_json(file: Union[str, Path], type_: Optional[Type[LoadType]] = None) -> Union[Any, LoadType]:
+    with open(file, "r", encoding="utf-8") as f:
+        obj = json.load(f)
+    if type_ is None:
+        return obj
+    else:
+        return parse_obj_as(type_, obj)
+
+def load_json_list(file: Union[str, Path], type_: Optional[Type[LoadType]] = None) -> Union[Iterable[Any], Iterable[LoadType]]:
+    with open(file, 'r', encoding="utf-8") as f:
+        for line in f:
+            if type_ is None:
+                yield json.loads(line)
+            else:
+                yield parse_obj_as(type_, json.loads(line))
+
diff --git a/gts_engine/bagualu/lib/utils/log_parser.py b/gts_engine/bagualu/lib/utils/log_parser.py
new file mode 100644
index 0000000..079d8e5
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/log_parser.py
@@ -0,0 +1,91 @@
+from abc import ABCMeta, abstractmethod
+from typing import Callable, Dict, List, Optional, Type
+import re
+import os
+
+
+Matcher = Callable[[str], Optional[str]]
+
+#############################################################################################
+## Base
+#############################################################################################
+class LogParser(metaclass=ABCMeta):
+    
+    def parse_log(self, file: str) -> Dict[str, Optional[str]]:
+        if not os.path.exists(file):
+            raise Exception("file not exist")
+        log = self._load_log(file)
+        return {name: self._support_error(matcher)(log) for name, matcher in self._target_info_dict.items()} 
+    
+    def _load_log(self, file: str) -> str:
+        with open(file, "r", encoding="utf-8") as f:
+            return f.read()
+        
+    @property
+    @abstractmethod
+    def _target_info_dict(self) -> Dict[str, Matcher]:
+        return {}
+    
+    def _support_error(self, matcher: Matcher) -> Matcher:
+        def wraper(log: str):
+            try:
+                res = matcher(log)
+            except:
+                res = None
+            return res
+        return wraper
+    
+    
+    
+#############################################################################################
+## Derived
+#############################################################################################
+    
+class StudentLogParser(LogParser):
+    
+    @property
+    def _target_info_dict(self) -> Dict[str, Matcher]:
+        return {
+            "dev_acc": self.__dev_acc_matcher,
+            "train_model_test_acc": self.__train_test_acc_matcher,
+            "inf_model_test_acc": self.__inf_test_acc_matcher
+        }
+
+    def __dev_acc_matcher(self, log: str) -> Optional[str]:
+        multi_model_dev_acc_list = re.findall(r"(?<=best checkpoint dev acc: )0\.\d+", log)
+        if len(multi_model_dev_acc_list) > 0:
+            return multi_model_dev_acc_list[0]
+        else:
+            return re.findall(r"(?<=save checkpoint. dev acc: )0\.\d+", log)[-1]
+        
+    def __train_test_acc_matcher(self, log: str) -> Optional[str]:
+        return re.findall(r"(?<=INFO: training model test acc: )0\.\d+", log)[0]
+    
+    def __inf_test_acc_matcher(self, log: str) -> Optional[str]:
+        return re.findall(r"(?<=INFO: inference model test acc: )0\.\d+", log)[0]
+
+class TeacherLogParser(LogParser):
+    
+    @property
+    def _target_info_dict(self) -> Dict[str, Matcher]:
+        return {
+            "test_acc": self.__test_acc_matcher,
+        }
+    
+    def __test_acc_matcher(self, log: str) -> Optional[str]:
+        return re.findall(r"(?<=global_acc: )0\.\d+", log)[0]
+
+
+#############################################################################################
+## Test
+#############################################################################################
+
+if __name__ == "__main__":
+    file = "/raid/jiangyuzhen/Documents/gts_student_test/my_tests/unified_test/logs/tnews_student_2shots_1.log"
+    file="/raid/jiangyuzhen/Documents/gts-student-finetune-std/out/student_output/logs/ft_std_csldcp_2022-10-31_16-50-19.log"
+    res = StudentLogParser().parse_log(file)
+    print(res)
+    
+        
+    
+    
diff --git a/gts_engine/bagualu/lib/utils/logger_manager.py b/gts_engine/bagualu/lib/utils/logger_manager.py
new file mode 100644
index 0000000..73d78ad
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/logger_manager.py
@@ -0,0 +1,31 @@
+import os
+import logging
+import datetime
+
+class LoggerManager:
+    @staticmethod
+    def set_logger(logger_name: str, log_file_path: str):
+        logger = logging.getLogger(logger_name)
+        logger.setLevel(logging.DEBUG)
+        file_handler = logging.FileHandler(log_file_path, mode='w')
+        file_handler.setLevel(logging.INFO)
+        file_handler.setFormatter(
+            logging.Formatter(
+                fmt='%(asctime)s - %(filename)s[line:%(lineno)d] - %(levelname)s: %(message)s',
+                datefmt='%Y-%m-%d %H:%M:%S')
+        )
+
+        logger.addHandler(file_handler)
+        
+        console_handler = logging.StreamHandler()
+        console_handler.setLevel(logging.INFO)
+        console_handler.setFormatter(
+            logging.Formatter(
+                fmt='%(asctime)s - %(filename)s - %(levelname)s: %(message)s',
+                datefmt='%Y-%m-%d %H:%M:%S')
+        )
+        logger.addHandler(console_handler)
+
+    @staticmethod
+    def get_logger(logger_name: str) -> logging.Logger:
+        return logging.getLogger(logger_name)
\ No newline at end of file
diff --git a/gts_engine/bagualu/lib/utils/path.py b/gts_engine/bagualu/lib/utils/path.py
new file mode 100644
index 0000000..01ccf73
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/path.py
@@ -0,0 +1,36 @@
+import os
+from typing import Literal
+import shutil
+from typing import Union
+from pathlib import Path
+
+def mk_inexist_dir(path: Union[str, Path], r: bool = True, clean: bool = False):
+    """创建不存在的路径 
+
+    Args:
+        path: 路径
+        r: 是否递归地创建父路径. Defaults to True.
+        clean: 如果路径不为空，是否清空
+    """
+    if not os.path.exists(path):
+        if r:
+            os.makedirs(path)
+        else:
+            os.mkdir(path)
+    if clean:
+        shutil.rmtree(path)
+        os.mkdir(path)
+        
+def get_file_size(path: Union[str, Path], unit: Literal["kb", "mb", "gb"]) -> float:
+    """获取文件在指定单位下的大小"""
+    if unit == "kb":
+        scale = 1024
+    elif unit == "mb":
+        scale = 1024 ** 2
+    elif unit == "gb":
+        scale = 1024 ** 3
+    else:
+        raise Exception("unit is not supported")
+    size = os.path.getsize(path)
+    return round(size / scale, 4)
+    
diff --git a/gts_engine/bagualu/lib/utils/statistics.py b/gts_engine/bagualu/lib/utils/statistics.py
new file mode 100644
index 0000000..9bb4c13
--- /dev/null
+++ b/gts_engine/bagualu/lib/utils/statistics.py
@@ -0,0 +1,137 @@
+from typing import Hashable, List, TypeVar, Sequence, Literal, Union, Iterable, Callable, Generic, Optional
+import numpy as np
+from functools import cmp_to_key
+
+
+def interval_mean(list_: List, low_quantile=0.25, high_quantile=0.75) -> float:
+    """计算上下分位数之间元素的均值"""
+    length = len(list_)
+    low_idx = int(length * low_quantile)
+    high_idx = int(length * high_quantile)
+    res = np.mean(np.sort(list_)[low_idx:high_idx])
+    return res # type: ignore
+
+T = TypeVar(name="T", bound=Hashable)
+def acc(predict_list: List[T], true_list: List[T]) -> float:
+    assert len(predict_list) == len(true_list)
+    return np.mean(np.array(predict_list) == np.array(true_list)) # type: ignore
+
+
+
+class ExponentialSmoothingList:
+    
+    number = Union[float, int]
+    
+    def __init__(self, level: Literal[1, 2, 3], alpha: Union[float, List[float]] ,warmup_steps: int = 3) -> None:
+        self.__check_condition(level, alpha)
+        if isinstance(alpha, list):
+            self.__alpha_list = alpha
+        else:
+            self.__alpha_list = [alpha] * level
+        self.__values = [[] for _ in range(level + 1)]
+        self.__level = level
+        self.__warmup_steps = warmup_steps
+        
+    def append(self, value: Union[number, Sequence[number]]):
+        if isinstance(value, Iterable):
+            for v in value:
+                self.__step_value(v)
+        else:
+            self.__step_value(value)
+            
+    @property
+    def values(self):
+        return self.__values
+    
+    def __repr__(self) -> str:
+        return str(self.__values)
+    
+    def __getitem__(self, level: int):
+        return self.__values[level]
+    
+    def __len__(self):
+        return len(self.__values[0])
+    
+    #############################################################################################
+    ######################################## private ##########################################
+        
+    def __step_value(self, value: number) -> None:
+        self.__values[0].append(value)
+        for level in range(1, self.__level + 1):
+            self.__append_level_value_list(level)
+    
+    def __append_level_value_list(self, level: int):
+        if len(self.__values[level - 1]) <= self.__warmup_steps:
+            self.__values[level].append(self.__values[level - 1][-1])
+        else:
+            alpha = self.__alpha_list[level - 1]
+            if len(self.__values[level - 1]) == self.__warmup_steps + 1:
+                self.__re_calculate_warmup_steps(level, alpha)
+            smoothed_value = alpha * self.__values[level][-1] + (1 - alpha) * self.__values[level - 1][-1]
+            self.__values[level].append(smoothed_value)
+    
+    def __re_calculate_warmup_steps(self, level: int, alpha: float):
+        self.__values[level][0] = sum(self.__values[level - 1][:self.__warmup_steps]) / self.__warmup_steps
+        for step in range(1, self.__warmup_steps):
+            smoothed_value = alpha * self.__values[level][step - 1] + (1 - alpha) * self.__values[level - 1][step]
+            self.__values[level][step] = smoothed_value
+    
+    def __check_condition(self, level, alpha) -> None:
+        if isinstance(alpha, list):
+            try:
+                assert len(alpha) == level
+            except:
+                raise Exception("the alpha list should be as long as level")
+            for a in alpha:
+                try:
+                    assert a >= 0 and a <= 1
+                except:
+                    raise Exception("alpha should be in [0, 1]")
+        else:
+            try:
+                assert alpha >= 0 and alpha <= 1
+            except:
+                raise Exception("alpha should be in [0, 1]")
+
+
+_MaxVar = TypeVar("_MaxVar")         
+class DynamicMax(Generic[_MaxVar]):
+    """带缓存、可自定义比较符的、可指定top n的动态max类"""
+    
+    def __init__(
+        self, 
+        top_n: int = 1, 
+        allow_repeat: bool = True,
+        cmp: Callable[[_MaxVar, _MaxVar], Union[int, float]] = lambda x1, x2: x1 - x2, # type: ignore
+    ):
+        self.__cmp = cmp
+        self.__top_n = top_n
+        self.__allow_repeat = allow_repeat
+        self.__cache: List[_MaxVar] = []
+    
+    def step(self, value: Union[_MaxVar, Sequence[_MaxVar]]) -> None:
+        if isinstance(value, Iterable):
+            for v in value:
+                self.__step_value(v)
+        else:
+            self.__step_value(value)
+        
+    def __step_value(self, value: _MaxVar) -> None:
+        self.__cache.append(value)
+        if not self.__allow_repeat:
+            self.__cache = list(set(self.__cache))
+        self.__cache.sort(key=cmp_to_key(self.__cmp), reverse=True) # type: ignore
+        if len(self.__cache) > self.__top_n:
+            self.__cache.pop()
+            
+    @property
+    def max_list(self) -> List[_MaxVar]:
+        return self.__cache
+        
+    @property
+    def max(self) -> _MaxVar:
+        return self.__cache[-1]
+    
+
+    
+    
\ No newline at end of file
diff --git a/gts_engine/bagualu/models/__init__.py b/gts_engine/bagualu/models/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/gts_engine/bagualu/models/ie/__init__.py b/gts_engine/bagualu/models/ie/__init__.py
new file mode 100644
index 0000000..fc88477
--- /dev/null
+++ b/gts_engine/bagualu/models/ie/__init__.py
@@ -0,0 +1,4 @@
+from .model import BagualuIEModel
+from .lit_model import BagualuIELitModel
+from .extract_model import BagualuIEExtractModel
+from .onnx_conf import BagualuIEOnnxConfig
diff --git a/gts_engine/bagualu/models/ie/extract_model.py b/gts_engine/bagualu/models/ie/extract_model.py
new file mode 100644
index 0000000..6fcaafe
--- /dev/null
+++ b/gts_engine/bagualu/models/ie/extract_model.py
@@ -0,0 +1,72 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import List
+import copy
+
+from transformers import PreTrainedTokenizer
+
+from ...dataloaders.ie.item_encoder import ItemEncoder
+from ...dataloaders.ie.item_decoder import ItemDecoder
+from ...arguments.ie import TrainingArgumentsIEStd
+from .model import BagualuIEModel
+
+
+class BagualuIEExtractModel(object):
+    """ BagualuIEExtractModel
+
+    Args:
+        tokenizer (PreTrainedTokenizer): tokenizer
+        args (TrainingArgumentsIEStd): arguments
+    """
+    def __init__(self,
+                 tokenizer: PreTrainedTokenizer,
+                 args: TrainingArgumentsIEStd) -> None:
+        self.encoder = ItemEncoder(tokenizer, args.max_length)
+        self.decoder = ItemDecoder(tokenizer, args)
+
+    def extract(self, batch_data: List[dict], model: BagualuIEModel) -> List[dict]:
+        """ extract
+
+        Args:
+            batch_data (List[dict]): batch of data
+            model (BagualuIEModel): model
+
+        Returns:
+            List[dict]: batch of data
+        """
+
+        model = model.cuda()
+        model.eval()
+
+        batch_data = copy.deepcopy(batch_data)
+        batch = [self.encoder.encode_item(item, with_label=False) for item in batch_data]
+        batch = self.encoder.collate(batch)
+
+        batch = {k: v.cuda() for k, v in batch.items()}
+
+        span_logits = model(**batch).cpu().detach().numpy()
+        label_mask = batch["label_mask"].cpu().detach().numpy()
+
+        for i, item in enumerate(batch_data):
+
+            entity_list, spo_list = self.decoder.decode(item,
+                                                        span_logits[i],
+                                                        label_mask[i])
+
+            item["spo_list"] = spo_list
+            item["entity_list"] = entity_list
+
+        return batch_data
diff --git a/gts_engine/bagualu/models/ie/lit_model.py b/gts_engine/bagualu/models/ie/lit_model.py
new file mode 100644
index 0000000..d128f15
--- /dev/null
+++ b/gts_engine/bagualu/models/ie/lit_model.py
@@ -0,0 +1,332 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# pylint: disable=no-member
+
+# import os
+from typing import List, Tuple
+from logging import Logger
+
+import torch
+from torch import Tensor
+from transformers.optimization import get_linear_schedule_with_warmup, get_constant_schedule
+from pytorch_lightning import LightningModule
+
+from .model import BagualuIEModel
+from ...arguments.ie import TrainingArgumentsIEStd
+from ...lib.components.losses import NegativeSampleLoss, DiceLoss, DistillSelfLoss, DecoupledBCEloss
+
+
+class BagualuIELitModel(LightningModule):
+    """ BagualuIELitModel
+
+    Args:
+        args (TrainingArgumentsIEStd): arguments
+        num_data (int, optional): num of input data. Defaults to 1.
+    """
+
+    _model: BagualuIEModel
+    _logger: Logger
+
+    def __init__(self, args: TrainingArgumentsIEStd, logger: Logger, num_data: int = 1) -> None:
+        super().__init__()
+
+        self.args = args
+        self.num_data = num_data
+
+        self._model = BagualuIEModel.from_pretrained(args.pretrained_model_root)
+        self._logger = logger
+        if args.loss == "dc":
+            self.criterion = DiceLoss(smooth=args.dc_smooth,
+                                      square_denominator=True,
+                                      with_logits=False,
+                                      ohem_ratio=0,
+                                      alpha=0.,
+                                      reduction="mean")
+        elif args.loss == "bce":
+            self.criterion = NegativeSampleLoss(args.neg_drop)
+        elif args.loss == "d_bce":
+            self.criterion = DecoupledBCEloss()
+        else:
+            raise NotImplementedError()
+
+        self.num_step = -1
+        self.num_step_each_epoch = -1
+        self.distill_self = args.distill_self
+        if self.distill_self:
+            self.distill_self_loss = DistillSelfLoss()
+
+    def setup(self, stage: str) -> None:  # pylint: disable=signature-differs
+        """ set up
+
+        Args:
+            stage (str): stage name
+        """
+
+        batch_size = self.args.batch_size
+        num_devices = self.trainer.num_devices
+        max_epochs = self.trainer.max_epochs
+        accumulate_grad_batches = self.trainer.accumulate_grad_batches
+
+        if stage == "fit":
+            if self.trainer.max_steps and self.trainer.max_steps > 0:
+                num_parallel = max(1, num_devices if num_devices is not None else 0)
+                num_step_each_epoch = int(self.num_data / num_parallel / accumulate_grad_batches / batch_size)
+                self.num_step_each_epoch = num_step_each_epoch
+                self.num_step = self.trainer.max_steps
+                self._logger.info("----- set max steps -----")
+                self._logger.info("num_data: %d", self.num_data)
+                self._logger.info("num_parallel: %d", num_parallel)
+                self._logger.info("one epoch training step: %d", self.num_step_each_epoch)
+                self._logger.info("Total training step: %d", self.num_step)
+            elif max_epochs and max_epochs > 0:
+                num_data = max_epochs * self.num_data
+                num_parallel = max(1, num_devices if num_devices is not None else 0)
+                num_step = int(num_data / num_parallel / accumulate_grad_batches / batch_size)
+                self.num_step = num_step
+                self.num_step_each_epoch = num_step / max_epochs
+                self._logger.info("----- set max epochs -----")
+                self._logger.info("num_data: %d", self.num_data)
+                self._logger.info("num_parallel: %d", num_parallel)
+                self._logger.info("one epoch training step: %d", self.num_step_each_epoch)
+                self._logger.info("Total training step: %d", self.num_step)
+            else:
+                raise ValueError()
+
+    def training_step(self, batch: dict, batch_idx: int):  # pylint: disable=signature-differs,arguments-differ,unused-argument
+        """ training
+
+        Args:
+            batch (dict): batch of data
+            batch_idx (int): batch index
+
+        Returns:
+            float: loss
+        """
+        span_logits = self._model(**batch)
+        span_labels = batch["span_labels"]
+        span_mask = batch["label_mask"]
+
+        loss = self.criterion(span_logits.reshape(-1),
+                              span_labels.reshape(-1),
+                              span_mask.reshape(-1)) * self.args.loss_boost
+
+        if self.distill_self:
+            mask_atten_logits, full_atten_logits = torch.chunk(span_logits, 2, dim=0)
+            mask, _ = torch.chunk(span_mask, 2, dim=0)
+            # label, label_1 = torch.chunk(span_labels, 2, dim=0)
+            assert full_atten_logits.shape == mask_atten_logits.shape
+            distill_loss1 = self.distill_self_loss(
+                mask_atten_logits,
+                full_atten_logits,
+                mask)
+            distill_loss2 = self.distill_self_loss(
+                full_atten_logits,
+                mask_atten_logits,
+                mask)
+            distill_loss = distill_loss1 + distill_loss2
+            loss += distill_loss
+
+        f1, recall, precise, sum_corr, sum_y_true, sum_y_pred = \
+            self.compute_metrics(span_logits, span_labels, span_mask)
+
+        self.log("train_step",  self.global_step)
+        self.log("lr", self.lr_schedulers().get_last_lr()[0])
+        self.log("train_loss", loss)
+        self.log("train_f1", f1)
+        self.log("train_recall", recall)
+        self.log("train_precise", precise)
+        # self.log("train_precise", precise, rank_zero_only=True)
+
+        if self.global_rank == 0:
+            if self.global_step % self.args.log_step == 0:
+                if not self.distill_self:
+                    self._logger.info("[TRAIN-%s] step:%s, epoch:%s, batch_idx:%s/%s, lr:%.2e, "
+                                      "loss:%.6f, f1/p/r=%.4f/%.4f/%.4f, corr/pred/true=%d/%d/%d",
+                                      self.global_rank,
+                                      self.global_step,
+                                      self.current_epoch,
+                                      batch_idx,
+                                      self.num_step_each_epoch,
+                                      self.lr_schedulers().get_last_lr()[0],
+                                      loss,
+                                      f1,
+                                      precise,
+                                      recall,
+                                      sum_corr,
+                                      sum_y_pred,
+                                      sum_y_true)
+                else:
+                    self._logger.info("[TRAIN-%s] step:%s, epoch:%s, batch_idx:%s/%s, lr:%.2e, "
+                                      "loss:%.6f, mse loss:%.6f, f1/p/r=%.4f/%.4f/%.4f, "
+                                      "corr/pred/true=%d/%d/%d",
+                                      self.global_rank,
+                                      self.global_step,
+                                      self.current_epoch,
+                                      batch_idx,
+                                      self.num_step_each_epoch,
+                                      self.lr_schedulers().get_last_lr()[0],
+                                      loss,
+                                      distill_loss,
+                                      f1,
+                                      precise,
+                                      recall,
+                                      sum_corr,
+                                      sum_y_pred,
+                                      sum_y_true)
+
+        return loss
+
+    def validation_step(self, batch: dict, batch_idx: int) -> None:  # pylint: disable=signature-differs,arguments-differ,unused-argument
+        """ validation
+
+        Args:
+            batch (dict): batch of data
+            batch_idx (int): batch index
+        """
+        span_logits = self._model(**batch)
+        span_labels = batch["span_labels"]
+        span_mask = batch["label_mask"]
+
+        loss = self.criterion(span_logits.reshape(-1),
+                              span_labels.reshape(-1),
+                              span_mask.reshape(-1)) * self.args.loss_boost
+
+        _, _, _, sum_corr, sum_y_true, sum_y_pred = self.compute_metrics(span_logits, span_labels, span_mask)
+
+        return [loss.cpu().detach().numpy(), sum_corr, sum_y_true, sum_y_pred]
+
+    def validation_epoch_end(self, outputs):
+        """ Called at the end of the validation epoch with the outputs of all validation steps.
+        Args:
+            outputs(list): outputs of all val_step
+        """
+        all_loss, all_acc, all_sum_y_pred, all_sum_y_true = [], 0., 0., 0.
+        for loss, sum_corr, sum_y_true, sum_y_pred in outputs:
+            all_loss.append(loss)
+            all_acc += sum_corr
+            all_sum_y_pred += sum_y_pred
+            all_sum_y_true += sum_y_true
+
+        recall = 0. if all_sum_y_true <= 0 else all_acc / all_sum_y_true
+        precise = 0. if all_sum_y_pred <= 0 else all_acc / all_sum_y_pred
+        f1 = 0. if recall + precise <= 0 else 2 * recall * precise / (recall + precise)
+
+        loss = sum(all_loss) / len(all_loss)
+
+        self._logger.info("\t[VALIDATION-%s] step:%s, epoch:%s, loss:%.6f, "
+                          "f1/p/r=%.4f/%.4f/%.4f, corr/pred/true=%d/%d/%d",
+                          self.global_rank,
+                          self.global_step, self.current_epoch, loss,
+                          f1,
+                          precise,
+                          recall,
+                          all_acc,
+                          all_sum_y_pred,
+                          all_sum_y_true)
+
+        self.log("val_loss", loss)
+        self.log("val_f1", f1)
+
+    def predict_step(self, batch: dict, batch_idx: int):  # pylint: disable=signature-differs,arguments-differ,unused-argument
+        """ predict
+
+        Args:
+            batch (dict): batch of data
+            batch_idx (int): batch index
+        """
+        span_logits = self._model(**batch)
+        span_mask = batch["label_mask"]
+        return span_logits, span_mask
+
+    def configure_optimizers(self) -> List[dict]:
+        """ configure optimizers
+
+        Returns:
+            List[dict]: optimizers
+        """
+
+        no_decay = ["bias", "LayerNorm.bias", "LayerNorm.weight"]
+        paras = list(filter(lambda p: p[1].requires_grad, self.named_parameters()))
+
+        paras = [
+            {
+                "params": [p for n, p in paras if not any(nd in n for nd in no_decay)],
+                "weight_decay": self.args.weight_decay
+            },
+            {
+                "params": [p for n, p in paras if any(nd in n for nd in no_decay)],
+                "weight_decay": 0.0
+            }
+        ]
+
+        optimizer = torch.optim.AdamW(paras, lr=self.args.learning_rate)
+
+        if self.args.warmup > 0:
+            num_warmup_steps = int(self.num_step * self.args.warmup)
+            scheduler = get_linear_schedule_with_warmup(optimizer,
+                                                        num_warmup_steps,
+                                                        self.num_step)
+        else:
+            scheduler = get_constant_schedule(optimizer)
+
+        return [{
+            "optimizer": optimizer,
+            "lr_scheduler": {
+                "scheduler": scheduler,
+                "interval": "step",
+                "frequency": 1
+            }
+        }]
+
+    def compute_metrics(self,
+                        logits: Tensor,
+                        labels: Tensor,
+                        mask: Tensor) -> Tuple[float, float, float]:
+        """ compute metrics
+
+        Args:
+            logits (Tensor): logits
+            labels (Tensor): labels
+            mask (Tensor): mask
+
+        Returns:
+            Tuple[float, float, float]: (f1, recall, precise)
+        """
+        ones = torch.ones_like(logits)
+        zero = torch.zeros_like(logits)
+        logits = torch.where(logits < 0.5, zero, ones)
+
+        y_pred = logits.reshape(shape=(-1, ))
+        y_true = labels.reshape(shape=(-1, ))
+        mask = mask.reshape(shape=(-1, ))
+        assert mask.shape == y_pred.shape == y_true.shape
+        y_pred = y_pred * mask
+        y_true = y_true * mask
+
+        corr = torch.eq(y_pred, y_true).float()
+        corr = torch.multiply(y_true, corr)
+
+        sum_y_pred = torch.sum(y_pred.float())
+        sum_y_true = torch.sum(y_true.float())
+        sum_corr = torch.sum(corr.float())
+
+        recall = 0. if sum_y_true <= 0 else sum_corr / sum_y_true
+        precise = 0. if sum_y_pred <= 0 else sum_corr / sum_y_pred
+        f1 = 0. if recall + precise <= 0 else 2 * recall * precise / (recall + precise)
+        if sum_y_true <= 0 and sum_y_pred <= 0:
+            f1, recall, precise = 1., 1., 1.
+
+        return f1, recall, precise, sum_corr, sum_y_true, sum_y_pred
diff --git a/gts_engine/bagualu/models/ie/model.py b/gts_engine/bagualu/models/ie/model.py
new file mode 100644
index 0000000..ed610f4
--- /dev/null
+++ b/gts_engine/bagualu/models/ie/model.py
@@ -0,0 +1,156 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# pylint: disable=no-member
+
+import torch
+from torch import nn, Tensor
+from transformers import BertPreTrainedModel, BertModel, BertConfig
+
+
+class Triaffine(nn.Module):
+    """ Triaffine module
+
+    Args:
+        triaffine_hidden_size (int): Triaffine module hidden size
+    """
+    def __init__(self, triaffine_hidden_size: int) -> None:
+        super().__init__()
+
+        self.triaffine_hidden_size = triaffine_hidden_size
+
+        self.weight_start_end = nn.Parameter(
+            torch.zeros(triaffine_hidden_size,
+                        triaffine_hidden_size,
+                        triaffine_hidden_size))
+
+        nn.init.normal_(self.weight_start_end, mean=0, std=0.1)
+
+    def forward(self,
+                start_logits: Tensor,
+                end_logits: Tensor,
+                cls_logits: Tensor) -> Tensor:
+        """forward
+
+        Args:
+            start_logits (Tensor): start logits
+            end_logits (Tensor): end logits
+            cls_logits (Tensor): cls logits
+
+        Returns:
+            Tensor: span_logits
+        """
+        start_end_logits = torch.einsum("bxi,ioj,byj->bxyo",
+                                        start_logits,
+                                        self.weight_start_end,
+                                        end_logits)
+
+        span_logits = torch.einsum("bxyo,bzo->bxyz",
+                                   start_end_logits,
+                                   cls_logits)
+
+        return span_logits
+
+
+class MLPLayer(nn.Module):
+    """MLP layer
+
+    Args:
+        input_size (int): input size
+        output_size (int): output size
+    """
+    def __init__(self, input_size: int, output_size: int) -> None:
+        super().__init__()
+        self.linear = nn.Linear(in_features=input_size, out_features=output_size)
+        self.act = nn.GELU()
+
+    def forward(self, x: Tensor) -> Tensor: # pylint: disable=invalid-name
+        """ forward
+
+        Args:
+            x (Tensor): input
+
+        Returns:
+            Tensor: output
+        """
+        x = self.linear(x)
+        x = self.act(x)
+        return x
+
+
+class BagualuIEModel(BertPreTrainedModel):
+    """ BagualuIEModel
+
+    Args:
+        config (BertConfig): config
+    """
+    def __init__(self, config: BertConfig) -> None:
+        super().__init__(config)
+        self.bert = BertModel(config)
+        self.config = config
+
+        self.triaffine_hidden_size = 128
+
+        self.mlp_start = MLPLayer(self.config.hidden_size,
+                                  self.triaffine_hidden_size)
+        self.mlp_end = MLPLayer(self.config.hidden_size,
+                                self.triaffine_hidden_size)
+        self.mlp_cls = MLPLayer(self.config.hidden_size,
+                                self.triaffine_hidden_size)
+
+        self.triaffine = Triaffine(self.triaffine_hidden_size)
+
+    def forward(self,  # pylint: disable=unused-argument
+                input_ids: Tensor,
+                attention_mask: Tensor,
+                position_ids: Tensor,
+                token_type_ids: Tensor,
+                text_len: Tensor,
+                label_token_idx: Tensor,
+                **kwargs) -> Tensor:
+        """ forward
+
+        Args:
+            input_ids (Tensor): input_ids
+            attention_mask (Tensor): attention_mask
+            position_ids (Tensor): position_ids
+            token_type_ids (Tensor): token_type_ids
+            text_len (Tensor): query length
+            label_token_idx (Tensor, optional): label_token_idx
+
+        Returns:
+            Tensor: span logits
+        """
+
+        # bert forward
+        hidden_states = self.bert(input_ids=input_ids,
+                                  attention_mask=attention_mask,
+                                  position_ids=position_ids,
+                                  token_type_ids=token_type_ids,
+                                  output_hidden_states=True)[0]  # (bsz, seq, dim)
+
+        max_text_len = text_len.max()
+
+        # 获取start、end、cls的hidden_states
+        hidden_start_end = hidden_states[:, :max_text_len, :] # text部分表示
+        hidden_cls = hidden_states.gather(1, label_token_idx.unsqueeze(-1)\
+            .repeat(1, 1, self.config.hidden_size)) # (bsz, task, dim)
+
+        # Triaffine
+        span_logits = self.triaffine(self.mlp_start(hidden_start_end),
+                                     self.mlp_end(hidden_start_end),
+                                     self.mlp_cls(hidden_cls)).sigmoid()
+
+        return span_logits
diff --git a/gts_engine/bagualu/models/ie/onnx_conf.py b/gts_engine/bagualu/models/ie/onnx_conf.py
new file mode 100644
index 0000000..d03be7d
--- /dev/null
+++ b/gts_engine/bagualu/models/ie/onnx_conf.py
@@ -0,0 +1,66 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+class BagualuIEOnnxConfig(object):
+    """ config """
+    def __init__(self) -> None:
+        self.input_names = [
+            "input_ids",
+            "attention_mask",
+            "position_ids",
+            "token_type_ids",
+            "text_len",
+            "label_token_idx",
+        ]
+        self.output_names = [
+            "span_logits"
+        ]
+        self.dynamic_axes = {
+
+            # input
+            "input_ids": {
+                0: "batch_size",
+                1: "input_len",
+            },
+            "attention_mask": {
+                0: "batch_size",
+                1: "input_len",
+                2: "input_len",
+            },
+            "position_ids": {
+                0: "batch_size",
+                1: "input_len",
+            },
+            "token_type_ids": {
+                0: "batch_size",
+                1: "input_len",
+            },
+            "text_len": {
+                0: "batch_size",
+            },
+            "label_token_idx": {
+                0: "batch_size",
+                1: "num_labels",
+            },
+
+            # output
+            "span_logits": {
+                0: "batch_size",
+                1: "seq_len",
+                2: "seq_len",
+                3: "num_labels",
+            },
+        }
diff --git a/gts_engine/bagualu/models/text_classification/__init__.py b/gts_engine/bagualu/models/text_classification/__init__.py
new file mode 100644
index 0000000..d65adbd
--- /dev/null
+++ b/gts_engine/bagualu/models/text_classification/__init__.py
@@ -0,0 +1,1 @@
+from .lightnings_std import TrainLightningClfStd, PredictLightningClfStd, InferenceLightningClfStd
\ No newline at end of file
diff --git a/gts_engine/bagualu/models/text_classification/lightnings_std.py b/gts_engine/bagualu/models/text_classification/lightnings_std.py
new file mode 100644
index 0000000..cec5bb8
--- /dev/null
+++ b/gts_engine/bagualu/models/text_classification/lightnings_std.py
@@ -0,0 +1,246 @@
+from dataclasses import asdict
+from typing import Any, Dict, List, Tuple, Union
+from torch import Tensor
+from transformers.optimization import AdamW
+import torch
+from transformers.tokenization_utils import PreTrainedTokenizer
+
+from ...lib.framework.consts import BertInput
+from ...lib.framework.classification_finetune.consts import InfBatch, InferenceManagerOutput, InferenceModelOutput, TrainBatch
+from ...lib.framework.classification_finetune import StdPrompt, BaseTrainingLightningClf, BaseInferenceLightningClf
+from ...lib.components.schedulers import warmup_linear_decay_scheduler_factory
+from ...lib.components.metrics import Logits2Acc
+from ...lib.components.losses import compute_kl_loss
+from ...lib.utils import LoggerManager
+
+from ...models.text_classification.models_std import TrainingModelClfStd, InferenceModelClfStd
+from ...arguments.text_classification.arguments_std import TrainingArgumentsClfStd, InferenceArgumentsClfStd
+
+class TrainLightningClfStd(BaseTrainingLightningClf):
+    
+    #############################################################################################
+    ## setup
+    #############################################################################################
+    
+    def __init__(
+        self, 
+        args: TrainingArgumentsClfStd, 
+        class_num: int, 
+        sample_num: int,
+    ):
+        super().__init__()
+        self._args = args
+        self._model: TrainingModelClfStd = TrainingModelClfStd(self._args,
+            self._args.pretrained_model_dir,
+            class_num
+        )
+        self._sample_num = sample_num
+        self._logits_2_acc = Logits2Acc()
+        self._logger = LoggerManager.get_logger(self._args.logger)
+        
+    def configure_optimizers(self):
+        optimizer = AdamW(
+            self._model.parameters(),
+            lr = self._args.learning_rate,
+            eps=1e-8,
+            correct_bias=True
+        )
+        scheduler = warmup_linear_decay_scheduler_factory(
+            optimizer=optimizer,
+            warm_up_epoch=self._args.warm_up_epoch,
+            decay_epoch=self._args.decay_epoch,
+            epoch=self._args.epoch,
+            train_data_length=self._sample_num,
+            batch_size=self._args.train_batch_size
+        )
+        scheduler_config = {
+            "scheduler": scheduler,
+            "interval": "step",
+            "frequency": 1
+        }
+        return {
+            "optimizer": optimizer,
+            "lr_scheduler": scheduler_config
+        }
+    
+    #############################################################################################
+    ## training 
+    #############################################################################################
+    
+    def training_step(self, batch: TrainBatch, batch_idx: int):
+        bert_input = BertInput(
+            input_ids=batch["input_ids"],
+            attention_mask=batch["input_mask"],
+            token_type_ids=batch["input_seg"],
+            labels=batch["labels"]
+        )
+        batch_size = batch["input_ids"].shape[0]
+        model_output = self._model.forward(
+            bert_input,
+            sample_weight=batch["weight"],
+            label_id_clf=batch["label_id_clf"]
+        )
+        loss = model_output["loss_total"].float()
+        
+        if self._args.use_rdrop:
+            # rdrop
+            logits = model_output["logits"]
+            model_output = self._model.forward(
+                bert_input,
+                sample_weight=batch["weight"],
+                label_id_clf=batch["label_id_clf"]
+            )
+            loss2 = model_output["loss_total"].float()
+            logits2 = model_output["logits"]
+            kl_loss = compute_kl_loss(logits, logits2)
+            loss  = (loss+loss2)/2 + self._args.rdrop_alpha * kl_loss
+        lr = self.lr_schedulers().get_last_lr()[-1] # type: ignore
+        loss_ce = model_output["loss_ce"]
+        loss_mlm = model_output["loss_mlm"]
+        return {
+            "loss": loss,
+            "lr": lr,
+            "batch_idx": batch_idx,
+            "batch_size": batch_size,
+            "loss_ce": loss_ce,
+            "loss_mlm": loss_mlm
+        }
+    
+    def training_step_end(self, step_output: Dict[str, Tensor]):
+        """聚合training_step多卡输出并打印"""
+        lr = step_output["lr"][0]
+        batch_idx = step_output["batch_idx"][0]
+        batch_size_list = step_output["batch_size"]
+        avg = lambda loss_list: (loss_list * (batch_size_list / batch_size_list.sum())).sum()
+        loss = avg(step_output["loss"])
+        loss_mlm = avg(step_output["loss_mlm"])
+        loss_ce = avg(step_output["loss_ce"])
+        self._logger.info(f"loss: {loss:.4f} lr: {lr:.2e} batch: {batch_idx}/{self.trainer.num_training_batches} epoch: {self.current_epoch} step: {self.global_step}")
+        metrics = {
+            "train_loss": loss,
+            "lr": lr,
+            "train_loss_ce": loss_ce,
+            "train_loss_mlm": loss_mlm
+        }
+        self.log_dict(metrics)
+        return loss
+    
+    #############################################################################################
+    ## validation
+    #############################################################################################
+    
+    def on_validation_epoch_start(self) -> None:
+        self._logger.info("validating...")
+    
+    def validation_step(self, batch: TrainBatch, batch_idx: int):
+        bert_input = BertInput(
+            input_ids=batch["input_ids"],
+            attention_mask=batch["input_mask"],
+            token_type_ids=batch["input_seg"],
+            labels=batch["labels"]
+        )
+        model_output = self._model.forward(
+            bert_input, 
+            sample_weight=batch["weight"],
+            label_id_clf=batch["label_id_clf"],
+            is_training=False
+        )
+        loss = model_output["loss_total"].float()
+        logits = model_output["logits"]
+        acc, _ = self._logits_2_acc.forward(logits, batch["label_id_clf"])
+        batch_size = len(batch["input_ids"])
+        return {"loss": loss, "acc": acc, "batch_size": batch_size} 
+
+    def validation_step_end(self, validation_step_outputs: Dict[str, Tensor]): 
+        """聚合validation_step多卡输出"""
+        loss_list = validation_step_outputs["loss"]
+        batch_size_list = validation_step_outputs["batch_size"]
+        step_loss = (loss_list * (batch_size_list / batch_size_list.sum())).sum() # loss按batch_size加权平均
+        step_acc = validation_step_outputs["acc"].sum()
+        step_batch_size = batch_size_list.sum()
+        return {"loss": step_loss, "acc": step_acc, "batch_size": step_batch_size}
+    
+    def validation_epoch_end(self, validation_step_outputs: List[Dict[str, Tensor]]) -> None:
+        """聚合所有验证结果并打印"""
+        loss_list = torch.stack([step["loss"] for step in validation_step_outputs])
+        acc_list = torch.stack([step["acc"] for step in validation_step_outputs])
+        batch_size_list = torch.stack([step["batch_size"] for step in validation_step_outputs])
+        dev_loss = loss_list.mean()
+        dev_acc = acc_list.sum() / batch_size_list.sum()
+        metrics = {
+            "dev_loss": dev_loss,
+            "dev_acc": dev_acc
+        }
+        self.log_dict(metrics, prog_bar=False, logger=True, rank_zero_only=True)
+        self._logger.info(f"validation - loss: {dev_loss:.4f} acc: {dev_acc:.4f} epoch: {self.current_epoch}")
+
+      
+class PredictLightningClfStd(BaseTrainingLightningClf):
+    def __init__(
+        self, 
+        prompt: StdPrompt,
+        args: TrainingArgumentsClfStd,
+        tokenizer: PreTrainedTokenizer,
+        datastore=None,
+        best_hyper=None,
+    ):
+        super().__init__()
+        self._model: InferenceModelClfStd = InferenceModelClfStd(prompt, args, tokenizer, datastore=datastore, best_hyper=best_hyper)
+        self._prompt = prompt
+        
+    def forward(self, input_ids: Tensor, input_mask: Tensor, input_seg: Tensor):
+        return InferenceModelOutput(**self._model.forward(input_ids, input_mask, input_seg))
+    
+    def predict_step(self, batch: InfBatch, batch_idx: int, dataloader_idx: int = 0) -> List[Tuple[int, str]]:
+        inference_output = self.forward(
+            input_ids=batch["input_ids"],
+            input_mask=batch["input_mask"],
+            input_seg=batch["input_seg"]
+        )
+        id_list: List[int] = batch["my_id"]
+        prediction_id_list: List[int] = inference_output["positions"].squeeze().tolist()
+        if not isinstance(prediction_id_list, list): # 可能出现最后一个batch是单个值的情况
+            prediction_id_list = [prediction_id_list]
+        prediction_label_list = [self._prompt.id2label[prediction_id].label for prediction_id in prediction_id_list]
+        prediction_output_list = [(int(id), label) for id, label in zip(id_list, prediction_label_list)]
+        return prediction_output_list
+    
+    def on_predict_epoch_end(self, results) -> None:
+        """聚合每个batch的预测结果"""
+        concated_list: List[Tuple[int, str]] = sum(results[0], [])
+        result_dict = {id: label for id, label in concated_list}
+        results[0] = result_dict
+        
+
+class InferenceLightningClfStd(BaseInferenceLightningClf):
+    
+    def __init__(
+        self, 
+        prompt: StdPrompt,
+        args: InferenceArgumentsClfStd,
+        tokenizer: PreTrainedTokenizer,
+        datastore=None,
+        best_hyper=None,
+    ):
+        super().__init__()
+        self._model: InferenceModelClfStd = InferenceModelClfStd(prompt, args, tokenizer, datastore=datastore, best_hyper=best_hyper) # type: ignore
+        self._prompt = prompt
+        
+    def forward(self, input_ids: Tensor, input_mask: Tensor, input_seg: Tensor):
+        return InferenceModelOutput(**self._model.forward(input_ids, input_mask, input_seg))
+        
+    def predict_step(self, batch: InfBatch, batch_idx: int, dataloader_idx: int = 0) -> InferenceManagerOutput:
+        inference_output = self.forward(
+            batch["input_ids"],
+            batch["input_mask"],
+            batch["input_seg"]
+        )
+        prediction_id_list: List[int] = inference_output["positions"].squeeze().tolist()
+        if not isinstance(prediction_id_list, list): # 可能出现最后一个batch是单个值的情况
+            prediction_id_list = [prediction_id_list]
+        prediction_label_list = [self._prompt.id2label[prediction_id].label for prediction_id in prediction_id_list]
+        probabilities_list = inference_output["probs"].tolist()
+        return InferenceManagerOutput(
+            predictions=prediction_label_list,
+            probabilities=probabilities_list
+        )
\ No newline at end of file
diff --git a/gts_engine/bagualu/models/text_classification/models_std.py b/gts_engine/bagualu/models/text_classification/models_std.py
new file mode 100644
index 0000000..86229e2
--- /dev/null
+++ b/gts_engine/bagualu/models/text_classification/models_std.py
@@ -0,0 +1,165 @@
+import torch
+from torch import Tensor, nn
+from transformers.models.bert.configuration_bert import BertConfig
+from transformers.models.bert.modeling_bert import BertForMaskedLM
+from typing import Optional, List, Protocol, Union
+from transformers.modeling_outputs import MaskedLMOutput
+from dataclasses import asdict
+from transformers.tokenization_utils import PreTrainedTokenizer
+from pathlib import Path
+import numpy as np
+from sklearn.metrics import pairwise_distances
+
+from ...lib.framework.consts import BertInput
+from ...lib.framework.classification_finetune.consts import PromptToken, TrainingModelOutput, InferenceModelOutput
+from ...lib.framework.classification_finetune import StdPrompt
+from ...lib.components.losses import MaxMultiLogits, LabelSmoothing
+from ...lib.components.knn_tools import inference_with_knn
+
+from ...arguments.text_classification.arguments_std import TrainingArgumentsClfStd
+
+
+class TrainingModelClfStd(nn.Module):
+    
+    def __init__(self, args, pretrained_model_dir: Union[str, Path], class_num: int, last_layers: int = 1):
+        super().__init__()
+        self._config = BertConfig.from_pretrained(pretrained_model_dir)
+        args.memory_optimization_setting
+        if args.use_gradient_checkpointing=="True":
+            self._config.gradient_checkpointing=True
+            print("使用gradient_checkpointing！")
+        self._bert_encoder: BertForMaskedLM = BertForMaskedLM.from_pretrained(pretrained_model_dir, config=self._config) # type: ignore
+        self._last_layers = last_layers
+        self._multiply = 1 if last_layers == 4 else 3
+        
+        self._max_logits = MaxMultiLogits(
+            class_num=class_num,
+            hidden_size=last_layers * self._config.hidden_size, 
+            multiply=self._multiply
+        )
+        self._label_smoothing = LabelSmoothing()
+        self._softmax = nn.Softmax(dim=-1)
+        self._KL_criterion = nn.KLDivLoss(reduction='batchmean')
+    
+    def forward(
+        self,
+        bert_input: BertInput,
+        sample_weight: Optional[Tensor] = None,
+        label_id_clf: Optional[Tensor] = None,
+        is_training: bool = True
+    ) -> TrainingModelOutput:
+        bert_output: MaskedLMOutput = self._bert_encoder.forward(**bert_input, output_hidden_states=True) # type: ignore
+        loss_mlm = bert_output.loss
+        logits = bert_output.logits
+        hidden_states = bert_output.hidden_states
+        assert loss_mlm is not None and logits is not None and hidden_states is not None
+        x_layer_cls: List[Tensor] = [hidden_states[-x] for x in range(1, self._last_layers + 1)]
+        input_hidden_states = torch.cat(x_layer_cls, dim=-1)
+        loss_ce = torch.tensor(0.)
+        kl_loss = torch.tensor(0.)
+        if label_id_clf is not None:
+            cls_logits = self._max_logits.forward(input_hidden_states)
+            logits = cls_logits
+            loss_cls = self._label_smoothing.forward(cls_logits, label_id_clf.view(-1))
+            if sample_weight is not None:
+                proced_weight = sample_weight.view([-1, 1])
+                loss_cls = loss_cls * proced_weight
+            loss_ce = loss_cls.float().mean()
+        loss = loss_ce + loss_mlm if is_training else loss_ce
+        return TrainingModelOutput(
+            loss_total=loss,
+            loss_ce=loss_ce,
+            loss_mlm=loss_mlm,
+            kl_loss=kl_loss,
+            logits=logits,
+            loss_rd=torch.tensor(0.0),
+            loss_ctr=torch.tensor(0.0),
+            loss_ner=torch.tensor(0.0)
+        )
+
+
+class InfModelArgsProto(Protocol):
+    pretrained_model_dir: Path
+    inference_label_prompt: str
+    max_length: int
+
+class InferenceModelClfStd(nn.Module):
+
+    def __init__(
+        self,
+        prompt: StdPrompt,
+        args: InfModelArgsProto,
+        tokenizer: PreTrainedTokenizer,
+        last_layers: int = 1,
+        datastore=None,
+        best_hyper=None,
+    ):
+
+        super().__init__()
+        self._config = BertConfig.from_pretrained(args.pretrained_model_dir)
+        self._bert_encoder: BertForMaskedLM = BertForMaskedLM(self._config)
+        self._last_layers = last_layers
+        self._multiply = 1 if last_layers == 4 else 3
+        
+        self._max_logits = MaxMultiLogits(
+            class_num=len(prompt.label2token),
+            hidden_size=last_layers * self._config.hidden_size, 
+            multiply=self._multiply
+        )
+        self._softmax = torch.nn.Softmax(dim=-1)
+        
+        predict_prompt = args.inference_label_prompt
+        prompt_ = prompt.prompt + predict_prompt
+        prompt_tokens = tokenizer.tokenize(prompt_)
+        encode_dict = tokenizer.encode_plus(prompt_)
+        self._prompt_ids = torch.tensor(
+            encode_dict['input_ids']).long().unsqueeze(0)
+        self._prompt_mask = torch.tensor(encode_dict['attention_mask']).long().unsqueeze(0)
+        self._prompt_seg = torch.tensor(encode_dict['token_type_ids']).long().unsqueeze(0)
+        self._max_length = args.max_length
+
+        self.datastore = datastore
+        self.best_hyper = best_hyper
+        self.sep_id = tokenizer.convert_tokens_to_ids(["[SEP]"])[0]  # type: ignore
+
+    def forward(self, input_ids: Tensor, input_mask: Tensor, input_seg: Tensor, prompt_gate=True):
+        batch_len = input_seg.shape[0]
+        device = self._bert_encoder.device
+        if prompt_gate:
+            # 无标注数据/onnx推理时，需拼接prompt
+            prompt_ids = self._prompt_ids.repeat(batch_len, 1).to(device)
+            prompt_mask = self._prompt_mask.repeat(batch_len, 1).to(device)
+            prompt_seg = self._prompt_seg.repeat(batch_len, 1).to(device)
+
+            input_ids = torch.cat((prompt_ids, input_ids.to(device)[:, 1:]), 1)[:, :self._max_length]
+            input_ids[:,-1][(input_ids[:,-1]>0).nonzero()] = self.sep_id #拼接后的input_ids特殊处理，保证最末有效token是[SEP]
+            input_seg = input_mask #句对模式，因此需修改句子的input_seg为1，此时等于input_mask
+            input_mask = torch.cat((prompt_mask, input_mask.to(device)[:, 1:]), 1)[:, :self._max_length]
+            input_seg = torch.cat((prompt_seg, input_seg.to(device)[:, 1:]), 1)[:, :self._max_length]
+        with torch.no_grad():
+            bert_output = self._bert_encoder.forward(
+                input_ids=input_ids.to(device),
+                attention_mask=input_mask.to(device),
+                token_type_ids=input_seg.to(device),
+                output_hidden_states=True
+            )
+            x_layer_cls = [bert_output.hidden_states[-x] for x in range(1, self._last_layers + 1)] # type: ignore
+            input_hidden_states: Tensor = torch.cat(x_layer_cls, dim=-1) # type: ignore
+            max_logits = self._max_logits.forward(input_hidden_states)
+            probs = self._softmax.forward(max_logits)
+            positions = torch.argmax(probs, dim=-1)
+
+            # 最后一层的平均作为句向量表征
+            embeds = bert_output.hidden_states[-1].mean(1)  # type: ignore
+            if self.datastore is not None:
+                # knn_inference 
+                knn_prob = inference_with_knn(self.datastore, probs, embeds, self.best_hyper)  # type: ignore
+                knn_prob = self._softmax.forward(knn_prob)
+                probs = knn_prob
+                positions = torch.argmax(probs, dim=-1)
+            
+        return InferenceModelOutput(
+            positions=positions,
+            probs=probs,
+            embeds=embeds
+        )
\ No newline at end of file
diff --git a/gts_engine/gts_common/arguments.py b/gts_engine/gts_common/arguments.py
index df15b2c..3ad248e 100644
--- a/gts_engine/gts_common/arguments.py
+++ b/gts_engine/gts_common/arguments.py
@@ -11,7 +11,7 @@ class GtsEngineArgs(Namespace):
     engine_type: Literal["qiankunding", "bagualu"]
     train_mode: Literal["fast", "standard", "advanced"]
     task_dir: PathStr
-    task_type: Literal["classification", "similarity", "nli"]
+    task_type: Literal["classification", "similarity", "nli", "ie"]
     num_workers: int
     train_batchsize: int
     valid_batchsize: int
@@ -27,24 +27,22 @@ class GtsEngineArgs(Namespace):
     seed: int
     lr: float
     gpus: int
-    num_sanity_val_steps: int
-    accumulate_grad_batches: int
     val_check_interval: float
     max_epochs: Optional[int]
     min_epochs: Optional[int]
-    
+    gradient_checkpointing_gate: str
     @property
     def train_data_path(self) -> Path:
         return Path(self.data_dir) / self.train_data
-    
+
     @property
     def valid_data_path(self) -> Path:
         return Path(self.data_dir) / self.valid_data
-    
+
     @property
     def test_data_path(self) -> Optional[Path]:
         return Path(self.data_dir) / self.test_data if hasattr(self, "test_data") else None
-    
+
     @property
     def label_data_path(self) -> Optional[Path]:
-        return Path(self.data_dir) / self.label_data if hasattr(self, "label_data") else None
\ No newline at end of file
+        return Path(self.data_dir) / self.label_data if hasattr(self, "label_data") else None
diff --git a/gts_engine/gts_common/pipeline_utils.py b/gts_engine/gts_common/pipeline_utils.py
index c655f38..7b46a0e 100644
--- a/gts_engine/gts_common/pipeline_utils.py
+++ b/gts_engine/gts_common/pipeline_utils.py
@@ -6,12 +6,10 @@
 from pytorch_lightning import Trainer, seed_everything, loggers
 from pytorch_lightning.callbacks import ModelCheckpoint, EarlyStopping
 from transformers import AutoModel, AutoTokenizer
-from .arguments import GtsEngineArgs
+from gts_common.arguments import GtsEngineArgs
 from gts_common.logs_utils import Logger
-
 logger = Logger().get_log()
 
-
 def download_model_from_huggingface(pretrained_model_dir, model_name, model_class=AutoModel, tokenizer_class=AutoTokenizer):
     if os.path.exists(os.path.join(pretrained_model_dir, model_name)):
         logger.info("model already exists.")
@@ -24,7 +22,7 @@ def download_model_from_huggingface(pretrained_model_dir, model_name, model_clas
     shutil.rmtree(cache_path)
     logger.info("model %s is downloaded from huggingface." % model_name)
 
-def generate_common_trainer(args, save_path):
+def generate_common_trainer(args: GtsEngineArgs, save_path):
     # Prepare Trainer
     checkpoint = ModelCheckpoint(dirpath=save_path,
                                     save_top_k=1,
@@ -40,9 +38,13 @@ def generate_common_trainer(args, save_path):
                                 ) 
 
     logger = loggers.TensorBoardLogger(save_dir=os.path.join(save_path, 'logs/'))
-
-    trainer = Trainer.from_argparse_args(
-        args,
+    trainer = Trainer(
+        max_epochs=args.max_epochs,
+        min_epochs=args.min_epochs,
+        accumulate_grad_batches=8,
+        val_check_interval=args.val_check_interval,
+        num_sanity_val_steps=1000,
+        gpus=args.gpus,
         logger=logger,
         callbacks=[checkpoint, early_stop]
     )
@@ -60,10 +62,8 @@ def __getattr__(self,name):
     def __setattr__(self,name,value):
         self[name]=value
 
-def save_args(args):
-    args.num_sanity_val_steps = 1000 
-    args.accumulate_grad_batches = 8 
-    args.val_check_interval = 0.5 
+
+def save_args(args: GtsEngineArgs):
     args_path = os.path.join(args.save_path, "args.json")
     with open(args_path, 'w') as f:
         json.dump(vars(args), f, indent=4)
diff --git a/gts_engine/gts_engine_inference.py b/gts_engine/gts_engine_inference.py
index 6991abd..1f62c9d 100644
--- a/gts_engine/gts_engine_inference.py
+++ b/gts_engine/gts_engine_inference.py
@@ -32,7 +32,7 @@ def main():
                             type=str, help="specific task directory")
     total_parser.add_argument("--engine_type", required=True, choices=["qiankunding", "bagualu"],
                             type=str, help="engine type")
-    total_parser.add_argument("--task_type", required=True, choices=["classification", "similarity", "nli"],
+    total_parser.add_argument("--task_type", required=True, choices=["classification", "similarity", "nli", "ie"],
                             type=str, help="task type for training")
     total_parser.add_argument("--input_path", required=True,
                             type=str, help="input path of data which will be inferenced")
diff --git a/gts_engine/gts_engine_service.py b/gts_engine/gts_engine_service.py
index 1a7844d..12823e0 100644
--- a/gts_engine/gts_engine_service.py
+++ b/gts_engine/gts_engine_service.py
@@ -24,7 +24,6 @@
 from gts_common.logs_utils import Logger
 
 logger = Logger().get_log()
-format_checker = service_utils.DataFormatChecker()
 
 app = FastAPI()
 
@@ -183,6 +182,8 @@ def start_train(train_input: TrainInput):
         return {"ret_code": -102, "message": "任务信息文件不存在"}
     task_info = json.load(open(task_info_path))
 
+    format_checker = service_utils.DataFormatChecker()
+
     valid, msg = format_checker.check_from_path(task_type=task_info["task_type"],
                                                 data_type="train",
                                                 data_path=os.path.join(task_data_dir, train_input.train_data))
@@ -338,6 +339,7 @@ def predict(inputs: PredictInput):
     task_type = task_info["task_type"]
     engine_type = task_info["engine_type"]
 
+    format_checker = service_utils.DataFormatChecker()
     valid, msg = format_checker.check_data(task_type=task_type,
                                            data_type="unlabeled",
                                            data=sentences)
diff --git a/gts_engine/gts_engine_train.py b/gts_engine/gts_engine_train.py
index cdde61a..c47a900 100644
--- a/gts_engine/gts_engine_train.py
+++ b/gts_engine/gts_engine_train.py
@@ -60,7 +60,7 @@ def main():
                             type=str, help="training mode")
     total_parser.add_argument("--task_dir", required=True, 
                             type=str, help="specific task directory")
-    total_parser.add_argument("--task_type", required=True, choices=["classification", "similarity", "nli"],
+    total_parser.add_argument("--task_type", required=True, choices=["classification", "similarity", "nli", "ie"],
                             type=str, help="task type for training")
     total_parser.add_argument('--num_workers', default=8,
                             type=int, help="number of workers for data preprocessing.")
@@ -99,9 +99,15 @@ def main():
                             type=float, help="learning rate")
     
     # * Args for Trainer
-    total_parser = Trainer.add_argparse_args(total_parser)
-    logger.info("total_parser:{}".format(total_parser))
-    args = total_parser.parse_args()
+    total_parser.add_argument('--max_epochs', default=None,
+                              type=int, help="upper limit of training epochs")
+    total_parser.add_argument('--min_epochs', default=None,
+                              type=int, help="lower limit of training epochs")
+    total_parser.add_argument('--val_check_interval', default=0.5,
+                              type=float, help="perform a validation loop every after every `N` training epochs")
+
+    print("total_parser:",total_parser)
+    args = total_parser.parse_args(namespace=GtsEngineArgs())
 
     logger.info("pretrained_model_dir {}".format(args.pretrained_model_dir))
     args.gpus = 1
diff --git a/gts_engine/pipelines/bagualu_classification.py b/gts_engine/pipelines/bagualu_classification.py
new file mode 100644
index 0000000..ec703fa
--- /dev/null
+++ b/gts_engine/pipelines/bagualu_classification.py
@@ -0,0 +1,38 @@
+from typing import List, Literal, Tuple, Type, Callable, Dict, Any
+from argparse import Namespace
+from enum import Enum
+
+from gts_common.registry import PIPELINE_REGISTRY
+from gts_common.pipeline_utils import load_args, save_args
+
+from bagualu.lib.framework.base_gts_engine_interface import GtsEngineArgs, BaseGtsEngineInterface, TRAIN_MODE
+from bagualu.lib.framework.classification_finetune.consts import InferenceManagerInputSample
+from bagualu.entrances.text_classification import GtsEngineInterfaceClfStd
+from bagualu.lib.framework.classification_finetune import BaseInferenceManagerClf
+
+
+mode_to_interface: Dict[TRAIN_MODE, BaseGtsEngineInterface] = {
+    TRAIN_MODE.STD: GtsEngineInterfaceClfStd()
+}
+
+@PIPELINE_REGISTRY.register(suffix=__name__) # type: ignore
+def train_pipeline(args: GtsEngineArgs):
+    # save args
+    args = save_args(args)
+    train_mode = TRAIN_MODE(args.train_mode)
+    module_factory = mode_to_interface[train_mode]
+    module_factory.prepare_training(args)
+    module_factory.generate_training_pipeline(args).main()
+    
+@PIPELINE_REGISTRY.register(suffix=__name__) # type: ignore
+def prepare_inference(save_path):
+    args: GtsEngineArgs = load_args(save_path) # type: ignore
+    train_mode = TRAIN_MODE(args.train_mode)
+    module_factory = mode_to_interface[train_mode]
+    return module_factory.generate_inference_manager(args)
+
+@PIPELINE_REGISTRY.register(suffix=__name__) # type: ignore
+def inference(samples: List[Dict[str, Any]], inference_manager: BaseInferenceManagerClf):
+    inf_sample_list = [InferenceManagerInputSample(text=sample["content"]) for sample in samples]
+    results = inference_manager.inference(inf_sample_list)
+    return results
\ No newline at end of file
diff --git a/gts_engine/pipelines/bagualu_ie.py b/gts_engine/pipelines/bagualu_ie.py
new file mode 100644
index 0000000..568e9c1
--- /dev/null
+++ b/gts_engine/pipelines/bagualu_ie.py
@@ -0,0 +1,81 @@
+# coding=utf-8
+# Copyright 2021 The IDEA Authors. All rights reserved.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import os
+from typing import List, Dict, Any
+
+from gts_common.registry import PIPELINE_REGISTRY
+from gts_common.pipeline_utils import load_args, save_args, download_model_from_huggingface
+from gts_common.arguments import GtsEngineArgs
+
+from bagualu.lib.framework.base_gts_engine_interface import BaseGtsEngineInterface, TRAIN_MODE
+from bagualu.entrances.ie import GtsEngineInterfaceIEStd
+from bagualu.entrances.ie.inference_manager_std import InferenceManagerIEStd
+
+mode_to_interface: Dict[TRAIN_MODE, BaseGtsEngineInterface] = {
+    TRAIN_MODE.STD: GtsEngineInterfaceIEStd()
+}
+
+
+@PIPELINE_REGISTRY.register(suffix=__name__)
+def train_pipeline(args: GtsEngineArgs) -> None:
+    """ Bagualu IE training pipeline
+
+    Args:
+        args (GtsEngineArgs): user arguments
+    """
+    # save args
+    args = save_args(args)
+    model_name = "Erlangshen-BERT-120M-IE-Chinese"
+    download_model_from_huggingface(args.pretrained_model_dir, model_name)
+    args.pretrained_model_dir = os.path.join(args.pretrained_model_dir, model_name)
+    train_mode = TRAIN_MODE(args.train_mode)
+    module_factory: GtsEngineInterfaceIEStd = mode_to_interface[train_mode]
+    module_factory.prepare_training(args)
+    module_factory.generate_training_pipeline(args).main()
+
+
+@PIPELINE_REGISTRY.register(suffix=__name__)
+def prepare_inference(save_path: str) -> InferenceManagerIEStd:
+    """ prepare inference
+
+    Args:
+        save_path (str): saved path from training.
+
+    Returns:
+        InferenceManagerIEStd: inference manager.
+    """
+    model_name = "Erlangshen-BERT-120M-IE-Chinese"
+    args: GtsEngineArgs = load_args(save_path)
+    download_model_from_huggingface(args.pretrained_model_dir, model_name)
+    args.pretrained_model_dir = os.path.join(args.pretrained_model_dir, model_name)
+    train_mode = TRAIN_MODE(args.train_mode)
+    module_factory: GtsEngineInterfaceIEStd = mode_to_interface[train_mode]
+    return module_factory.generate_inference_manager(args)
+
+
+@PIPELINE_REGISTRY.register(suffix=__name__)
+def inference(samples: List[Dict[str, Any]],
+              inference_manager: InferenceManagerIEStd) -> List[dict]:
+    """ inference
+
+    Args:
+        samples (List[Dict[str, Any]]): input samples for inference.
+        inference_manager (InferenceManagerIEStd): inference manager.
+
+    Returns:
+        List[dict]: inference results
+    """
+    results = inference_manager.inference(samples)
+    return results
diff --git a/requirements.txt b/requirements.txt
index 5ae3365..4609456 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -3,7 +3,7 @@ numpy==1.22.3
 psutil==5.8.0
 pydantic==1.10.2
 pynvml==11.0.0
-pytorch_lightning==1.6.2
+pytorch_lightning==1.7.6
 scikit_learn==1.1.3
 setuptools==58.0.4
 starlette==0.20.4
@@ -11,3 +11,7 @@ torch==1.11.0+cu113
 tqdm==4.62.3
 transformers==4.18.0
 uvicorn==0.19.0
+python-multipart==0.0.5
+sentence-transformers==2.2.2
+LAC==2.1.2
+textda==0.1.0.6


Test patch:
diff --git a/examples/ner/weibo_test.json b/examples/ner/weibo_test.json
new file mode 100644
index 0000000..a9575b2
--- /dev/null
+++ b/examples/ner/weibo_test.json
@@ -0,0 +1,20 @@
+{"task": "实体识别", "text": "一节课的时间真心感动了李开复感动", "entity_list": [{"entity_text": "李开复", "entity_type": "人名", "entity_index": [11, 14]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "回复支持，赞成，哈哈米八吴够历史要的陈小奥丁丁我爱小肥肥一族大头仔大家团结一致，誓要去台湾饮喜酒，由包机，团结的力量大", "entity_list": [{"entity_text": "陈小奥", "entity_type": "人名", "entity_index": [18, 21]}, {"entity_text": "台湾", "entity_type": "地理位置", "entity_index": [43, 45]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "剑网乱世长安公测盛典今日开启，海量豪礼火爆开送精美挂件、听雨·汉服娃娃、诙谐双骑独轮车等你来拿，更有千台红米手机、等十四重惊喜转发即抽活动地址：已有人参与剑网官方微博剑网客户服务", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "在街上听见音乐我舞动起来很丢人？真的很丢人吗？", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "三毛说我唯一锲而不舍，愿意以自己的生命去努力的，只不过是保守我个人的心怀意念，在我有生之日，做一个真诚的人，不放弃对生活的热爱和执着，在有限的时空里，过无限广", "entity_list": [{"entity_text": "三毛", "entity_type": "人名", "entity_index": [0, 2]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "星期天的早晨七点学车，驾校太给力。我的头发都没有洗", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "发表了博文小学美术新课标的反思学习新学期开学有两个多月了，在这段时间的教学计划、教案的撰写工作中，让我对小学美术新课程标准有了进一步的认识。我们现在所用的教材，根据儿童心理特点以及", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "哈哈哈哈哈哈镰刀刮腋毛哈哈哈哈哈哈我的朋友是个呆哈哈哈哈哈哈哈哈哈哈哈哈哈十万个冷笑话官网你这个不孝子哈哈哈哈哈萬萬哈哈哈哈哈哈哈杰克波比哈哈哈狼心狗肺猪肾哈哈哈哈哈哈哈胡唠哈哈哈哈哈哈哈哈哈柯震倲哈哈哈哈哈哈哈", "entity_list": [{"entity_text": "杰克", "entity_type": "人名", "entity_index": [65, 67]}, {"entity_text": "波比", "entity_type": "人名", "entity_index": [67, 69]}, {"entity_text": "柯震倲", "entity_type": "人名", "entity_index": [96, 99]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "��就爱这个牌子的糖果", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "延参法师品味人生如同走进一片山水，静静的呼吸，安静的欣赏，这就是生活。", "entity_list": [{"entity_text": "延参", "entity_type": "人名", "entity_index": [0, 2]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "就感恩吧越来越没战斗力了。", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "叶赟葆：全球时尚财运滚滚而来钱", "entity_list": [{"entity_text": "叶赟葆", "entity_type": "人名", "entity_index": [0, 3]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "我要去我要去花心花心花心耶分手大师贵仔邓超四大名捕围观话筒转发邓超贴吧微博号外话筒望周知。邓超四大名捕", "entity_list": [{"entity_text": "邓超", "entity_type": "人名", "entity_index": [19, 21]}, {"entity_text": "邓超", "entity_type": "人名", "entity_index": [31, 33]}, {"entity_text": "邓超", "entity_type": "人名", "entity_index": [45, 47]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "曾小贤终于跟一菲表白了哈哈鼓掌", "entity_list": [{"entity_text": "曾小贤", "entity_type": "人名", "entity_index": [0, 3]}, {"entity_text": "一菲", "entity_type": "人名", "entity_index": [6, 8]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "太看重感情的人就是这样容易满足更容易受伤；总有一种被忽视的感觉；付出的远远超过得到的；很固执，习惯冷战；在别人面前笑得很开心，一个人的时候却很漠落；陌生人前很安静，朋友面前胡闹；坐在电脑前，不知道做什么，却又不想关掉它；不喜欢等待，却总是等待；经常不经意的发呆。", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "霸气当时我就泪奔了笑哈哈冲着这霸气果断转了笑死了", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "我叫赵新民我们也这样照吧啊啊啊哈哈哈哈真好玩", "entity_list": [{"entity_text": "赵新民", "entity_type": "人名", "entity_index": [2, 5]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "日中午，宋同志抵达汉口站转动车回家。哈哈", "entity_list": [{"entity_text": "汉口站", "entity_type": "地点", "entity_index": [9, 12]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "我討厭現在的生活，覺得日子過得好苦，我每一天都哭過，痛過才去睡覺。不知道這樣做是否值得，但是就是不忍心這樣就離開，我不想用自己的快樂去換取別人的痛苦，女人大概是心軟的，這絕對是真的", "entity_list": [], "choice": ["地理位置", "组织机构", "地点", "人名"]}
+{"task": "实体识别", "text": "我叫赵新民是不是很有爱啊疯狂世界我爱你你会说话？", "entity_list": [{"entity_text": "赵新民", "entity_type": "人名", "entity_index": [2, 5]}], "choice": ["地理位置", "组织机构", "地点", "人名"]}