Instructions to use internlm/internlm2-wqx-20b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use internlm/internlm2-wqx-20b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="internlm/internlm2-wqx-20b", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("internlm/internlm2-wqx-20b", trust_remote_code=True, dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use internlm/internlm2-wqx-20b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "internlm/internlm2-wqx-20b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/internlm2-wqx-20b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/internlm/internlm2-wqx-20b
- SGLang
How to use internlm/internlm2-wqx-20b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "internlm/internlm2-wqx-20b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/internlm2-wqx-20b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "internlm/internlm2-wqx-20b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "internlm/internlm2-wqx-20b", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use internlm/internlm2-wqx-20b with Docker Model Runner:
docker model run hf.co/internlm/internlm2-wqx-20b
File size: 2,387 Bytes
7f8a007 f2a49cc 7f8a007 f2a49cc 7f8a007 f2a49cc 7f8a007 f2a49cc 7f8a007 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | # InternLM2-WQX-20B
<div align="center">
<img src="https://raw.githubusercontent.com/InternLM/InternLM/main/assets/logo.svg" width="200"/>
<div> </div>
<div align="center">
<b><font size="5">InternLM2-WQX</font></b>
<sup>
<a href="https://internlm.intern-ai.org.cn/">
<i><font size="4">HOT</font></i>
</a>
</sup>
<div> </div>
</div>
[](./LICENSE)
InternLM2-WQX-20B <a href="https://huggingface.co/internlm/internlm2-wqx-20b">🤗</a> <a href="https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-wqx-20b/summary"><img src="https://raw.githubusercontent.com/InternLM/InternLM/main/assets/modelscope_logo.png" width="20px"></a> | InternLM2-WQX-VL-20B <a href="https://huggingface.co/internlm/internlm2-wqx-vl-20b">🤗</a> <a href="https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm2-wqx-vl-20b/summary"><img src="https://raw.githubusercontent.com/InternLM/InternLM/main/assets/modelscope_logo.png" width="20px"></a>
</div>
# Introduction
InternLM2-WQX与InternLM2-WQX-VL是InternLM团队于2024年高考前夕最新推出的文曲星系列模型。
高考覆盖各类学科及题型,同时因其开考前的“绝密性”,被视作中国最具权威的考试之一,成为评估考生综合能力的“试金石”。这一面向人类设计的高难度综合性测试,目前普遍被研究者用于考察大模型的智能水平。InternLM2-WQX系列模型在2024年高考评测集[GAOKAO-Eval](https://github.com/open-compass/GAOKAO-Eval)上取得了优异的成绩,综合表现与GPT-4o相当,且超越了国内外一系列开源大模型,体现了InternLM2-WQX系列模型优秀的性能。
我们即将更新关于文曲星系列模型数据准备的相关说明,敬请期待。
# MD5 Check
```
md5sum ./*
5209adfd6ef7d1724848ff0372362568 ./model-00001-of-00004.safetensors
e37ee2eafecfed543d10dca75998204e ./model-00002-of-00004.safetensors
ea3da8035b0c2a31c369dd463adf9b52 ./model-00003-of-00004.safetensors
f1ff218f801c69fd4c12c534b64e1b60 ./model-00004-of-00004.safetensors
```
# Citation
```bibtex
@misc{2024internlm2wqx,
title={https://github.com/InternLM/InternLM-WQX},
author={InternLM Team},
howpublished = {\url{https://github.com/InternLM/InternLM-WQX}},
year={2024}
}
``` |