Update README.md

README.md

@@ -1,3 +1,56 @@
 ---
 license: apache-2.0
+language:
+- zh
+metrics:
+- accuracy
+- recall
+- precision
+library_name: transformers
+pipeline_tag: text-classification
 ---
+# Flames-scorer
+
+This is the specified scorer for Flames benchmark – a highly adversarial benchmark in Chinese for LLM's value alignment evaluation.  
+For more detail, please refer to our [paper](https://arxiv.org/abs/2311.06899) and [Github repo](https://github.com/AIFlames/Flames/tree/main)
+
+## Model Details
+* Developed by: Shanghai AI Lab and Fudan NLP Group.
+* Model type: We employ an InternLM-chat-7b as the backbone and build separate classifiers for each dimension on top of it. Then, we apply a multi-task training approach to train the scorer.
+* Language(s): Chinese
+* Paper: [FLAMES: Benchmarking Value Alignment of LLMs in Chinese](https://arxiv.org/abs/2311.06899)
+* Contact: For questions and comments about the model, please email tengyan@pjlab.org.cn.
+
+## Usage
+
+The environment can be set up as:
+```shell
+$ pip install -r requirements.txt
+```
+And you can use `infer.py` to evaluate your model:
+```shell
+python infer.py --data_path YOUR_DATA_FILE.jsonl
+```
+Please note that:
+1. Ensure each entry in `YOUR_DATA_FILE.jsonl` includes the fields: "dimension", "prompt", and "response".
+2. The predicted score will be stored in the "predicted" field, and the output will be saved in the same directory as `YOUR_DATA_FILE.jsonl`.
+3. The accuracy of the Flames-scorer on out-of-distribution prompts (i.e., prompts not included in the Flames-prompts) has not been evaluated. Consequently, its predictions for such data may not be reliable.
+
+## Citation
+If you think this scorer is helpful, please cite the paper.
+```bibtex
+@misc{huang2023flames,
+      title={Flames: Benchmarking Value Alignment of Chinese Large Language Models}, 
+      author={Kexin Huang and Xiangyang Liu and Qianyu Guo and Tianxiang Sun and Jiawei Sun and Yaru Wang and Zeyang Zhou and Yixu Wang and Yan Teng and Xipeng Qiu and Yingchun Wang and Dahua Lin},
+      year={2023},
+      eprint={2311.06899},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```
+
+---
+
+
+license: apache-2.0
+---