facebook/blt
Updated
•
30
Mitko Vasilev
mitkox
AI & ML interests
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
Recent Activity
liked
a model
5 days ago
facebook/blt
liked
a model
5 days ago
facebook/blt-7b
liked
a model
14 days ago
nvidia/Llama-3_1-Nemotron-Ultra-253B-v1
Organizations
mitkox's activity
liked
2
models
5 days ago
liked
a
model
14 days ago
liked
a
dataset
15 days ago
liked
a
model
19 days ago
liked
a
dataset
21 days ago
liked
a
model
27 days ago
liked
a
model
29 days ago
liked
2
models
about 2 months ago
liked
2
datasets
2 months ago
upvoted
2
collections
3 months ago
liked
a
model
3 months ago
updated
a
model
3 months ago
published
a
model
3 months ago
posted
an
update
3 months ago
Post
2815
llama.cpp is 26.8% faster than ollama.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
- 7 replies
liked
a
dataset
3 months ago
posted
an
update
3 months ago
Post
591
Stargate to the west of me
DeepSeek to the east
Here I am
Stuck in the middle with the EU
It will likely be a matter of sparkle to get export control on frontier research and models on both sides, leaving us in a vacuum.
Decentralized training infrastructure and on device inferencing are the future.
DeepSeek to the east
Here I am
Stuck in the middle with the EU
It will likely be a matter of sparkle to get export control on frontier research and models on both sides, leaving us in a vacuum.
Decentralized training infrastructure and on device inferencing are the future.