Create README.md (#1)

- Create README.md (829cbec2b255f60ad1dcec4289b8df99ef0ce141)

README.md

@@ -0,0 +1,99 @@
+---
+license: apache-2.0
+datasets:
+- tatsu-lab/alpaca
+- news_commentary
+language:
+- ar
+- el
+- hi
+- tr
+- vi
+- zh
+- en
+metrics:
+- bleu
+- bleurt
+- comet
+pipeline_tag: text-generation
+---
+# Extrapolating Large Language Models to Non-English by Aligning Languages
+
+This repository contains the code implementation for the project that aims to empower pre-trained Large Language Models (LLMs) on non-English languages by building semantic alignment across languages. The project explores cross-lingual instruction-tuning and multilingual instruction-tuning techniques. The code implementation is based on [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca).
+
+![](./xllama.jpg)
+
+## Requirements and Installation
+To install this repository, follow these steps:
+```
+git clone git@github.com:NJUNLP/x-LLM.git
+cd x-LLM
+pip install --editable ./
+```
+
+For detailed information about the conda environment, refer to the environment.yml file.
+
+## Usage
+### Download Pre-trained LLM
+Start by downloading the pre-trained LLM into the ./model directory.
+
+### Download Dataset
+You can download all the datasets used in this project from this [link](https://drive.google.com/file/d/1bkejieKDJFDJ45UmQYiY4eeqpGBwj-r-/view?usp=drive_link). Once downloaded, place the datasets in the ./data directory. The datasets include:
+
+* Training dataset
+  * Alpaca
+  * Wikimatrix
+  * Newscommentary
+* Evaluation dataset
+  * XQUAD
+  * MLQA
+  * Flores-101
+  * MI-Eval
+
+### Load Raw Data Along with Instruction 
+You can load raw data along with instruction using the provided scripts (./data/<dataset>/<dataset.py>). If you want to use a new dataset, you need to implement the corresponding script. The loaded data will have the following structure:
+``` python
+datasets.Features(
+    {
+        "id": datasets.Value("string"),
+        "instruction": datasets.Value("string"),
+        "input": datasets.Value("string"),
+        "output": datasets.Value("string")
+    }
+)
+```
+
+## Instruction-tune Pre-trained LLM
+To instruction-tune the pre-trained LLM, run the train.sh script. For example, you can instruction-tune LLaMA-7B to x-LLaMA-7B (Chinese) with the following command:
+``` bash
+bash script/train.sh llama-7b-hf alpaca_en+alpaca_zh+translation_ncwm_en-zh
+```
+In this command, the first argument denotes the pre-trained LLM to use, and the second argument represents the training data to use. You can use + to concatenate multiple datasets, and the training data will be shuffled by the Huggingface Trainer.
+
+Once the training is complete, the finetuned LLM will be saved in ./model/llama-7b-hf.alpaca_en+alpaca_zh+translation_ncwm_en-zh.finetune. You can use aliases to define shorter names, and more details can be found in ./data/alias/alias.json.
+
+## Test Finetuned LLM
+To test the finetuned LLM, run the inference.sh script. For example, you can test the tuned LLM on the Flores dataset with the following command:
+``` bash
+bash script/inference.sh llama-7b-hf.alpaca_en+alpaca_zh+translation_ncwm_en-zh.finetune translation_flores_en-zh
+```
+The output results will be saved in model/llama-7b-hf.alpaca_en+alpaca_zh+translation_ncwm_en-zh.finetune/test/translation_flores_en-zh.inference.jsonl. The prediction field represents the generated content of the LLM.
+
+## Interact with LLM Through Web UI
+To interact with the LLM through a web UI, run app.py with the following command:
+``` bash
+bash app.py model/llama-7b-hf.alpaca_en+alpaca_zh+translation_ncwm_en-zh.finetune
+```
+
+## Citation
+If you find this repository helpful, please consider citing our paper:
+```
+@misc{zhu2023extrapolating,
+      title={Extrapolating Large Language Models to Non-English by Aligning Languages}, 
+      author={Wenhao Zhu and Yunzhe Lv and Qingxiu Dong and Fei Yuan and Jingjing Xu and Shujian Huang and Lingpeng Kong and Jiajun Chen and Lei Li},
+      year={2023},
+      eprint={2308.04948},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL}
+}
+```