Post
640
🤗 I did some test of AI performance of
@Intel
Ultra 9 185h GPU and NPU!
(My RAM: LPDDR5x 7467 16GB * 2)
🗺 GPU:
💻 @OpenVINO :
rajatkrishna/Meta-Llama-3-8B-OpenVINO-INT4 : 12 tok/s
💻IPEX-LLM (Llama.cpp backend):
Qwen/Qwen2-7B-Instruct-GGUF Q4_K_M: 15.5 tok/s
Qwen/Qwen2-0.5B-Instruct-GGUF Q8_0: 65 tok/s
🧠🤖 NPU:
microsoft/Phi-3-mini-128k-instruct int4: 7 tok/s
🤓It looks like it have really reached a usable level!
(My RAM: LPDDR5x 7467 16GB * 2)
🗺 GPU:
💻 @OpenVINO :
rajatkrishna/Meta-Llama-3-8B-OpenVINO-INT4 : 12 tok/s
💻IPEX-LLM (Llama.cpp backend):
Qwen/Qwen2-7B-Instruct-GGUF Q4_K_M: 15.5 tok/s
Qwen/Qwen2-0.5B-Instruct-GGUF Q8_0: 65 tok/s
🧠🤖 NPU:
microsoft/Phi-3-mini-128k-instruct int4: 7 tok/s
🤓It looks like it have really reached a usable level!