SporkySporkness/FLUX.1-Fill-dev-LoRA-extract
Updated
Sporkness
SporkySporkness
AI & ML interests
None yet
Recent Activity
New activity
4 days ago
bartowski/Behemoth-123B-v1-GGUF:Failing to load Q6K
updated
a model
4 days ago
SporkySporkness/FLUX.1-Fill-dev-LoRA-extract
New activity
4 days ago
SporkySporkness/FLUX.1-Fill-dev-LoRA-extract:你测试过了吗
Organizations
SporkySporkness's activity
Failing to load Q6K
#1 opened 4 days ago
by
SporkySporkness
SporkySporkness
updated
a
model
4 days ago
ss996
lora version?
6
#1 opened 4 days ago
by
cchance27
cchance27
updated
3
models
4 days ago
Longcal?
1
#1 opened 5 days ago
by
SporkySporkness
SporkySporkness
Thanks
#1 opened 5 days ago
by
SporkySporkness
SporkySporkness
Quant request
#1 opened 7 days ago
by
SporkySporkness
SporkySporkness
Thanks
#1 opened 18 days ago
by
SporkySporkness
SporkySporkness
Request for 3.75bpw
2
#1 opened 18 days ago
by
SporkySporkness
SporkySporkness
Great
#2 opened 21 days ago
by
SporkySporkness
SporkySporkness
Props to how you handle the example dialogue.
2
#1 opened 25 days ago
by
jackboot
jackboot
upvoted
a
collection
24 days ago
EXL2 quant request
#1 opened about 1 month ago
by
SporkySporkness
SporkySporkness
Reacted to
ImranzamanML's
post with 👍
about 1 month ago
Post
1306
Here is how we can calculate the size of any LLM model:
Each parameter in LLM models is typically stored as a floating-point number. The size of each parameter in bytes depends on the precision.
32-bit precision: Each parameter takes 4 bytes.
16-bit precision: Each parameter takes 2 bytes
To calculate the total memory usage of the model:
Memory usage (in bytes) = No. of Parameters × Size of Each Parameter
For example:
32-bit Precision (FP32)
In 32-bit floating-point precision, each parameter takes 4 bytes.
Memory usage in bytes = 1 billion parameters × 4 bytes
1,000,000,000 × 4 = 4,000,000,000 bytes
In gigabytes: ≈ 3.73 GB
16-bit Precision (FP16)
In 16-bit floating-point precision, each parameter takes 2 bytes.
Memory usage in bytes = 1 billion parameters × 2 bytes
1,000,000,000 × 2 = 2,000,000,000 bytes
In gigabytes: ≈ 1.86 GB
It depends on whether you use 32-bit or 16-bit precision, a model with 1 billion parameters would use approximately 3.73 GB or 1.86 GB of memory, respectively.
Each parameter in LLM models is typically stored as a floating-point number. The size of each parameter in bytes depends on the precision.
32-bit precision: Each parameter takes 4 bytes.
16-bit precision: Each parameter takes 2 bytes
To calculate the total memory usage of the model:
Memory usage (in bytes) = No. of Parameters × Size of Each Parameter
For example:
32-bit Precision (FP32)
In 32-bit floating-point precision, each parameter takes 4 bytes.
Memory usage in bytes = 1 billion parameters × 4 bytes
1,000,000,000 × 4 = 4,000,000,000 bytes
In gigabytes: ≈ 3.73 GB
16-bit Precision (FP16)
In 16-bit floating-point precision, each parameter takes 2 bytes.
Memory usage in bytes = 1 billion parameters × 2 bytes
1,000,000,000 × 2 = 2,000,000,000 bytes
In gigabytes: ≈ 1.86 GB
It depends on whether you use 32-bit or 16-bit precision, a model with 1 billion parameters would use approximately 3.73 GB or 1.86 GB of memory, respectively.
Reacted to
bartowski's
post with ❤️
about 1 month ago
Post
17588
In regards to the latest mistral model and GGUFs for it:
Yes, they may be subpar and may require changes to llama.cpp to support the interleaved sliding window
Yes, I got excited when a conversion worked and released them ASAP
That said, generation seems to work right now and seems to mimic the output from spaces that are running the original model
I have appended -TEST to the model names in an attempt to indicate that they are not final or perfect, but if people still feel mislead and that it's not the right thing to do, please post (civilly) below your thoughts, I will highly consider pulling the conversions if that's what people think is best. After all, that's what I'm here for, in service to you all !
Yes, they may be subpar and may require changes to llama.cpp to support the interleaved sliding window
Yes, I got excited when a conversion worked and released them ASAP
That said, generation seems to work right now and seems to mimic the output from spaces that are running the original model
I have appended -TEST to the model names in an attempt to indicate that they are not final or perfect, but if people still feel mislead and that it's not the right thing to do, please post (civilly) below your thoughts, I will highly consider pulling the conversions if that's what people think is best. After all, that's what I'm here for, in service to you all !
- 6 replies
3.5 - 4.0 bpw quant request
1
#1 opened about 1 month ago
by
SporkySporkness
SporkySporkness
liked
a
model
2 months ago
