Sleeping
🌖
Adrian Saez Martinez PRO
adriszmar
·
AI & ML interests
LLM - NLP en español
Recent Activity
updated
a Space
about 1 month ago
adriszmar/MeetingDemo
updated
a model
about 1 month ago
adriszmar/Qwen2.5-Math-7B-Instruct-AWQ
updated
a model
about 1 month ago
adriszmar/QAIMath-Qwen2.5-7B-TIES
Organizations
adriszmar's activity
updated
a
Space
about 1 month ago
updated
2
models
about 1 month ago
reacted to
ImranzamanML's
post with 👍
about 2 months ago
Post
1356
Here is how we can calculate the size of any LLM model:
Each parameter in LLM models is typically stored as a floating-point number. The size of each parameter in bytes depends on the precision.
32-bit precision: Each parameter takes 4 bytes.
16-bit precision: Each parameter takes 2 bytes
To calculate the total memory usage of the model:
Memory usage (in bytes) = No. of Parameters × Size of Each Parameter
For example:
32-bit Precision (FP32)
In 32-bit floating-point precision, each parameter takes 4 bytes.
Memory usage in bytes = 1 billion parameters × 4 bytes
1,000,000,000 × 4 = 4,000,000,000 bytes
In gigabytes: ≈ 3.73 GB
16-bit Precision (FP16)
In 16-bit floating-point precision, each parameter takes 2 bytes.
Memory usage in bytes = 1 billion parameters × 2 bytes
1,000,000,000 × 2 = 2,000,000,000 bytes
In gigabytes: ≈ 1.86 GB
It depends on whether you use 32-bit or 16-bit precision, a model with 1 billion parameters would use approximately 3.73 GB or 1.86 GB of memory, respectively.
Each parameter in LLM models is typically stored as a floating-point number. The size of each parameter in bytes depends on the precision.
32-bit precision: Each parameter takes 4 bytes.
16-bit precision: Each parameter takes 2 bytes
To calculate the total memory usage of the model:
Memory usage (in bytes) = No. of Parameters × Size of Each Parameter
For example:
32-bit Precision (FP32)
In 32-bit floating-point precision, each parameter takes 4 bytes.
Memory usage in bytes = 1 billion parameters × 4 bytes
1,000,000,000 × 4 = 4,000,000,000 bytes
In gigabytes: ≈ 3.73 GB
16-bit Precision (FP16)
In 16-bit floating-point precision, each parameter takes 2 bytes.
Memory usage in bytes = 1 billion parameters × 2 bytes
1,000,000,000 × 2 = 2,000,000,000 bytes
In gigabytes: ≈ 1.86 GB
It depends on whether you use 32-bit or 16-bit precision, a model with 1 billion parameters would use approximately 3.73 GB or 1.86 GB of memory, respectively.
updated
a
model
about 2 months ago
liked
a
model
about 2 months ago
updated
a
Space
3 months ago
updated
a
model
3 months ago
liked
a
Space
6 months ago
updated
2
Spaces
6 months ago
updated
2
models
6 months ago
updated
a
Space
6 months ago