Update README.md

README.md

@@ -5,7 +5,6 @@ license_link: >-
   https://github.com/SkyworkAI/Skywork/blob/main/Skywork%20Community%20License.pdf
 ---
 
-
 <!-- <div align="center">
 <h1>
   ✨Skywork
@@ -14,7 +13,7 @@ license_link: >-
 <div align="center"><img src="misc/skywork_logo.jpeg" width="550"/></div>
 
 <p align="center">
-🤗 <a href="https://huggingface.co/Skywork" target="_blank">Hugging Face</a> • 🤖 <a href="https://modelscope.cn/organization/Skywork" target="_blank">ModelScope</a> • 👾 <a href="https://wisemodel.cn/organization/Skywork" target="_blank">Wisemodel</a> • 💬 <a href="https://github.com/SkyworkAI/Skywork/blob/main/misc/wechat.png?raw=true" target="_blank">WeChat</a>• 📜<a href="https://github.com/SkyworkAI/Skywork-MoE/blob/main/skywork-moe-tech-report.pdf" target="_blank">Tech Report</a>
+🤗 <a href="https://huggingface.co/Skywork" target="_blank">Hugging Face</a> • 🤖 <a href="https://modelscope.cn/organization/Skywork" target="_blank">ModelScope</a> • 👾 <a href="https://wisemodel.cn/organization/Skywork" target="_blank">Wisemodel</a> • 💬 <a href="https://github.com/SkyworkAI/Skywork/blob/main/misc/wechat.png?raw=true" target="_blank">WeChat</a>• 📜<a href="https://arxiv.org/pdf/2406.06563" target="_blank">Tech Report</a>
 </p>
 
 <div align="center">
@@ -41,6 +40,7 @@ Skywork-MoE demonstrates comparable or superior performance to models with more
 
 # Table of contents
 
+- [☁️Download URL](#Download-URL)
 - [👨‍💻Benchmark Results](#Benchmark-Results)
 - [🏆Demonstration of Hugging Face Model Inference](#Demonstration-of-HuggingFace-Model-Inference)
 - [📕Demonstration of vLLM Model Inference](#Demonstration-of-vLLM-Model-Inference)
@@ -48,7 +48,16 @@ Skywork-MoE demonstrates comparable or superior performance to models with more
 - [🤝Contact Us and Citation](#Contact-Us-and-Citation)
 
 
+# Download URL
+
+|         |                               HuggingFace Model                                |  ModelScope Model   |  Wisemodel Model  |
+|:-------:|:------------------------------------------------------------------------------:|:-----------------------------:|:-----------------------------:|
+| **Skywork-MoE-Base**     |     🤗 [Skywork-MoE-Base](https://huggingface.co/Skywork/Skywork-MoE-Base)     | 🤖[Skywork-MoE-Base](https://www.modelscope.cn/models/skywork/Skywork-MoE-base) | 👾[Skywork-MoE-Base](https://wisemodel.cn/models/Skywork/Skywork-MoE-base) |
+| **Skywork-MoE-Base-FP8**  | 🤗 [Skywork-MoE-Base-FP8](https://huggingface.co/Skywork/Skywork-MoE-Base-FP8) | 🤖[Skywork-MoE-Base-FP8](https://www.modelscope.cn/models/skywork/Skywork-MoE-Base-FP8) | 👾[Skywork-MoE-Base-FP8](https://wisemodel.cn/models/Skywork/Skywork-MoE-Base-FP8) |
+| **Skywork-MoE-Chat** |                               😊 [Coming Soon]()                               | 🤖 | 👾 |
+
 # Benchmark Results
+
 We evaluated Skywork-MoE-Base model on various popular benchmarks, including C-Eval, MMLU, CMMLU, GSM8K, MATH and HumanEval.
 <img src="misc/skywork_moe_base_evaluation.png" alt="Image" width="600" height="280">
 
@@ -94,15 +103,28 @@ coming soon...
 
 We provide a method to quickly deploy the Skywork-MoE-Base model based on vllm.
 
+Under fp8 precision you can run Skywork-MoE-Base with just only 8*4090.
+
 You can get the source code in [`vllm`](https://github.com/SkyworkAI/vllm)
 
+You can get the fp8 model in [`Skywork-MoE-Base-FP8`](https://huggingface.co/Skywork/Skywork-MoE-Base-FP8)
 
 ### Based on local environment
 
-Some dependencies need to be installed:
+Since pytorch only supports 4090 using fp8 precision in the nightly version, you need to install the corresponding or newer version of pytorch.
+
+``` shell
+# for cuda12.1
+pip3 install --pre torch pytorch-triton --index-url https://download.pytorch.org/whl/nightly/cu121
+# for cuda12.4
+pip3 install --pre torch pytorch-triton --index-url https://download.pytorch.org/whl/nightly/cu124
+```
+
+Some other dependencies also need to be installed:
 
 ```shell
-pip3 install xformers vllm-flash-attn 
+MAX_JOBS=8 pip3 install git+https://github.com/facebookresearch/xformers.git # need to wait for a long time
+pip3 install vllm-flash-attn --no-deps
 ```
 
 Then clone the [`vllm`](https://github.com/SkyworkAI/vllm) provided by skywork:
@@ -115,10 +137,12 @@ cd vllm
 Then compile and install vllm:
 
 ``` shell
+pip3 install -r requirements-build.txt
+pip3 install -r requirements-cuda.txt
 MAX_JOBS=8 python3 setup.py install
 ```
 
-### Based on docker
+### Base on docker
 
 You can use the docker image provided by skywork to run vllm directly:
 
@@ -129,7 +153,7 @@ docker pull registry.cn-wulanchabu.aliyuncs.com/triple-mu/skywork-moe-vllm:v1
 Then start the container and set the model path and working directory.
 
 ```shell
-model_path="Skywork/Skywork-MoE-Base"
+model_path="Skywork/Skywork-MoE-Base-FP8"
 workspace=${PWD}
 
 docker run \
@@ -142,7 +166,7 @@ docker run \
     --privileged=true \
     --ulimit stack=67108864 \
     --ipc=host \
-    -v ${model_path}:/Skywork-MoE-Base \
+    -v ${model_path}:/Skywork-MoE-Base-FP8 \
     -v ${workspace}:/workspace \
     registry.cn-wulanchabu.aliyuncs.com/triple-mu/skywork-moe-vllm:v1
 ```
@@ -154,7 +178,7 @@ Now, you can run the Skywork MoE model for fun!
 ``` python
 from vllm import LLM, SamplingParams
 
-model_path = 'Skywork/Skywork-MoE-Base'
+model_path = 'Skywork/Skywork-MoE-Base-FP8'
 prompts = [
     "The president of the United States is",
     "The capital of France is",
@@ -205,10 +229,21 @@ If you find our work helpful, please feel free to cite our paper~
 ```
 @misc{wei2024skywork,
       title={Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models}, 
-      author={Tianwen Wei, Bo Zhu, Liang Zhao, Cheng Cheng, Biye Li, Weiwei Lü, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Liang Zeng, Xiaokun Wang, Yutuan Ma, Rui Hu, Shuicheng Yan, Han Fang, Yahui Zhou},
+      author={Tianwen Wei, Bo Zhu, Liang Zhao, Cheng Cheng, Biye Li, Weiwei Lü, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Liang Zeng, Xiaokun Wang, Yutuan Ma, Rui Hu, Shuicheng Yan, Han Fang, Yahui Zhou},
+      url={https://arxiv.org/pdf/2406.06563},
       year={2024},
       archivePrefix={arXiv},
       primaryClass={cs.CL}
 }
 ```
 
+```
+@article{zhao2024longskywork,
+  title={LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models},
+  author={Zhao, Liang and Wei, Tianwen and Zeng, Liang and Cheng, Cheng and Yang, Liu and Cheng, Peng and Wang, Lijie and Li, Chenxia and Wu, Xuejie and Zhu, Bo and others},
+  journal={arXiv preprint arXiv:2406.00605},
+  url={https://arxiv.org/abs/2406.00605},
+  year={2024}
+}
+```
+