Spaces:

hf-accelerate

/

model-memory-usage

like 557

Running on CPU Upgrade

Really cool tool! I'm not too aware of all the math that goes on when calculating these things, so this might be a dumb question: since the calculation is for batch size 1, does this scale 1-1 linearly with higher batch sizes? And what about sequence length? (related: https://huggingface.co/spaces/hf-accelerate/model-memory-usage/discussions/7)

Thanks!