Hugging Face's logo
Xuan Son NGUYEN's picture
40 26 26

Xuan Son NGUYEN

ngxson
shtefcs's profile picture fgdrfgrgrdgdr's profile picture nezubn's profile picture
·
https://blog.ngxson.com

AI & ML interests

Doing AI for fun, not for profit

Recent Activity

reacted to mitkox's post with 🚀 about 7 hours ago
llama.cpp is 26.8% faster than ollama. I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison. Total duration: llama.cpp 6.85 sec <- 26.8% faster ollama 8.69 sec Breakdown by phase: Model loading llama.cpp 241 ms <- 2x faster ollama 553 ms Prompt processing llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster ollama 42.17 tokens/s with an eval time of 498 ms Token generation llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster ollama 122.07 tokens/s with an eval time 7.64 sec llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing. Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
reacted to mitkox's post with 👍 about 7 hours ago
llama.cpp is 26.8% faster than ollama. I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison. Total duration: llama.cpp 6.85 sec <- 26.8% faster ollama 8.69 sec Breakdown by phase: Model loading llama.cpp 241 ms <- 2x faster ollama 553 ms Prompt processing llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster ollama 42.17 tokens/s with an eval time of 498 ms Token generation llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster ollama 122.07 tokens/s with an eval time 7.64 sec llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing. Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
updated a Space 1 day ago
ngxson/fastapi_hackathon_template
View all activity

Articles

Organizations

Hugging Face's profile picture Blog-explorers's profile picture Hugging Face TB Research's profile picture ggml.ai's profile picture Hugging Face Discord Community's profile picture Consumer AI Edge Hackathon (Meta, Hugging Face, Pytorch, Scaleway & Unaite)'s profile picture Mistral AI Game Jam's profile picture

ngxson's activity

liked a model 5 days ago
liked a model 11 days ago
liked a model 13 days ago
liked a dataset 20 days ago
liked a dataset 21 days ago
liked a model about 1 month ago
liked a Space about 1 month ago
liked a model 3 months ago
liked a Space 3 months ago
liked a dataset 3 months ago
liked a model 3 months ago
liked a Space 3 months ago
liked a model 3 months ago
liked a Space 3 months ago
liked 2 datasets 3 months ago
liked 2 models 3 months ago
liked a model 4 months ago
liked a model 5 months ago