Update README.md

README.md

@@ -15,7 +15,9 @@ Qwen2 is the new series of Qwen large language models. For Qwen2, we release a n
 
 Compared with the state-of-the-art opensource language models, including the previous released Qwen1.5, Qwen2 has generally surpassed most opensource models and demonstrated competitiveness against proprietary models across a series of benchmarks targeting for language understanding, language generation, multilingual capability, coding, mathematics, reasoning, etc.
 
-Qwen2-57B-A14B-Instruct-GPTQ-Int4 supports a context length of up to 65,536 tokens, enabling the processing of extensive inputs. Please refer to [this section](#processing-long-texts) for detailed instructions on how to deploy Qwen2 for handling long texts.
+**Note: vLLM does not support the GPTQ version of Qwen2MoeForCausalLM currently.**
+
+Qwen2-57B-A14B-Instruct supports a context length of up to 65,536 tokens, enabling the processing of extensive inputs. However, since vLLM currently does not support this model (Qwen2-57B-A14B-Instruct-GPTQ-Int4), please refer to [Qwen2-57B-A14B-Instruct](https://huggingface.co/Qwen/Qwen2-57B-A14B-Instruct).
 
 For more details, please refer to our [blog](https://qwenlm.github.io/blog/qwen2/), [GitHub](https://github.com/QwenLM/Qwen2), and [Documentation](https://qwen.readthedocs.io/en/latest/).
 <br>
@@ -71,63 +73,6 @@ generated_ids = [
 response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
 ```
 
-### Processing Long Texts
-
-To handle extensive inputs exceeding 32,768 tokens, we utilize [YARN](https://arxiv.org/abs/2309.00071), a technique for enhancing model length extrapolation, ensuring optimal performance on lengthy texts.
-
-For deployment, we recommend using vLLM. You can enable the long-context capabilities by following these steps:
-
-1. **Install vLLM**: You can install vLLM by running the following command.
-
-```bash
-pip install "vllm>=0.4.3"
-```
-
-Or you can install vLLM from [source](https://github.com/vllm-project/vllm/).
-
-2. **Configure Model Settings**: After downloading the model weights, modify the `config.json` file by including the below snippet:
-    ```json
-        {
-            "architectures": [
-                "Qwen2MoeForCausalLM"
-            ],
-            // ...
-            "vocab_size": 152064,
-
-            // adding the following snippets
-            "rope_scaling": {
-                "factor": 2.0,
-                "original_max_position_embeddings": 32768,
-                "type": "yarn"
-            }
-        }
-    ```
-    This snippet enable YARN to support longer contexts.
-
-3. **Model Deployment**: Utilize vLLM to deploy your model. For instance, you can set up an openAI-like server using the command:
-
-    ```bash
-    python -m vllm.entrypoints.openai.api_server --served-model-name Qwen2-57B-A14B-Instruct-GPTQ-Int4 --model path/to/weights
-    ```
-
-    Then you can access the Chat API by:
-
-    ```bash
-    curl http://localhost:8000/v1/chat/completions \
-        -H "Content-Type: application/json" \
-        -d '{
-        "model": "Qwen2-57B-A14B-Instruct-GPTQ-Int4",
-        "messages": [
-            {"role": "system", "content": "You are a helpful assistant."},
-            {"role": "user", "content": "Your Long Input Here."}
-        ]
-        }'
-    ```
-
-    For further usage instructions of vLLM, please refer to our [Github](https://github.com/QwenLM/Qwen2).
-
-**Note**: Presently, vLLM only supports static YARN, which means the scaling factor remains constant regardless of input length, **potentially impacting performance on shorter texts**. We advise adding the `rope_scaling` configuration only when processing long contexts is required.
-
 ## Benchmark and Speed
 
 To compare the generation performance between bfloat16 (bf16) and quantized models such as GPTQ-Int8, GPTQ-Int4, and AWQ, please consult our [Benchmark of Quantized Models](https://qwen.readthedocs.io/en/latest/benchmark/quantization_benchmark.html). This benchmark provides insights into how different quantization techniques affect model performance.