System requirements
What are the requirements for running the 560m model at a decent speed?
I use a Intel 12th gen Core i5 and an Nvidia 3060. Did a test with the prompt "food is" on 1024 max length a few times and GPU took from 5-15 seconds while CPU normally around 15-30 seconds with the longest output at 1m35s. Minimum RAM seems to be at least 4 GB RAM and VRAM (only if using GPU) available, would recommend at least 6 GB available on both though. I would not recommend this model for normal use and would recommend looking for a different one as the outputs don't seem that good, an example output: "food is really good, is it? Also, will it be so much tastier with the stuffing? Thanks.
Hi, and you want, there aren’t many options out there for it. It’s a whole different flavor – they tend to stuff it with a stuffing or a combination of the two.
It depends on how you cook it. Your broth should work just fine, because the best cooking for this is marinating the soup. (Sourts don’t marinate well.) You’d need more water for cooking the broth to keep the flavor from getting lost. You’ll need the oil for frying to marinate."
bloom-560m has ~1.2GB weights in half precision. If you have a GPU, a consumer type (around 16GB RAM GPU) would suffise.
If you want to run bloom-560m with your CPU, you need to load your model in full precision, hence need around 2.4GB available RAM. If you have between 8-16GB CPU RAM it should be enough to run this model.