qualcomm_Qwen2.5-7B-Instruct
- Model creator: Qwen
- Original model: Qwen2.5-7B-Instruct
This repository contains the Qwen2.5-7B-Instruct model optimized for Qualcomm hardware using Qualcomm® AI Engine Direct (QNN). It is designed for high-performance inference on edge devices powered by Qualcomm Snapdragon platforms, enabling efficient on-device AI capabilities with low latency and reduced power consumption.
Model Details
- Developed by: Advantech-EIOT / Alibaba Cloud
- Architecture: Qwen2.5
- Task: Text Generation (Chat/Instruction)
- Precision: Quantized (w4a16) for NPU optimization
- Target Device: Quantized and optimized specifically for Snapdragon® X Elite
- Optimization: Qualcomm® AI Stack / QNN SDK
Hardware Compatibility
This model is highly optimized for Advantech Edge AI platforms powered by Qualcomm processors:
- Windows: Snapdragon® X Elite (e.g. Snapdragon® based Microsoft Surface Pro)
- Linux: Dragonwing® Platforms (e.g. Dragonwing® IQ-9075)
Limitations and Disclaimer
Qwen2.5 is a powerful language model but may exhibit hallucinations or generate inaccurate information.
- Accuracy: Users should validate outputs for critical applications.
- Usage: Please refer to the Apache 2.0 License for usage restrictions and acceptable use policies.
- Edge Optimization: Inference performance may vary depending on the specific hardware configuration and thermal constraints of the edge device.