Repository: ggml-org/llama.cpp
Pull request: ggml-quants : fix make_qp_quants NANs and IQ1 assertion errors

This is kind of a follow up to #11773, with the fix from <https://github.com/ggml-org/llama.cpp/pull/11773#discussion_r2066664121> (which *should* fix <https://github.com/ggml-org/llama.cpp/issues/11764> and <https://github.com/ggml-org/llama.cpp/issues/12439>).

`make_qp_quants` doesn't check for divisions by zero with its `suml2` when calculating the resulting scale, and so I've added a check to avoid producing NAN scales that way. That function is used for `Q2_K`, `Q4_K`, `Q5_K`, and `IQ2_XXS`.

In the future, I'm hoping to replace `make_qp_quants` with a saner quantization function from #12557. But until that's ready, might as well prevent some known existing errors.

Somewhat relatedly, I've been studying the i-quants implementation, and in `IQ1_S` and `IQ1_M`, `-FLT_MIN` being used as the initial `best_score` seems wrong.

https://github.com/ggml-org/llama.cpp/blob/21c17b5befc5f6be5992bc87fc1ba99d388561df/ggml/src/ggml-quants.c#L4269-L4287

`-FLT_MIN` is `-1.1754944e-38`, (which is the first normal non-zero value below 0). 

The assertion can fail in the situation where `sumq2` is always smaller or equal to `5.9604645e-08`, and `sumqx * sumqx` is zero (e.g. if `sumqx` is on the order of `1e-24`) and then `-FLT_MIN * 5.9604645e-08` is zero, so the branch setting the new `best_score` is never taken in that case.

I'm pretty sure the intention was for the initial value of `best_scores` to be `-FLT_MAX` (aka `-3.4028235e+38`), which doesn't have the problem of rounding to zero (it can overflow to `-INFINITY`, but that's also a reasonable starting value, since it would also be smaller than any `sumqx*sumqx`).

This should hopefully fix variants of <https://github.com/ggml-org/llama.cpp/issues/14788>, and <https://github.com/ggml-org/llama.cpp/issues/6067>.

cc @nicoboss, @bartowski1182, @schmorp since you've previously reported issues linked to this. 

---

*Make sure to read the [contributing guidelines](https://github.com/ggml-org/llama.cpp/blob/master/CONTRIBUTING.md) before submitting a PR*


Changed files:
filename: ggml/src/ggml-quants.c
patch: @@ -566,7 +566,7 @@ static float make_q3_quants(int n, int nmax, const float * GGML_RESTRICT x, int8
         for (int i = 0; i < n; ++i) {
             L[i] += nmax;
         }
-        return sumlx / suml2;
+        return suml2 > 0.0f ? sumlx / suml2 : 0.0f;
     }
     for (int i = 0; i < n; ++i) {
         int l = nearest_int(iscale * x[i]);
@@ -901,7 +901,7 @@ static float make_qp_quants(int n, int nmax, const float * GGML_RESTRICT x, uint
     for (int i = 0; i < n; ++i) {
         max = MAX(max, x[i]);
     }
-    if (!max) { // all zero
+    if (max < GROUP_MAX_EPS) { // all zero
         for (int i = 0; i < n; ++i) { L[i] = 0; }
         return 0.f;
     }
@@ -966,7 +966,7 @@ static float make_qp_quants(int n, int nmax, const float * GGML_RESTRICT x, uint
             break;
         }
     }
-    return sumlx/suml2;
+    return suml2 > 0.0f ? sumlx / suml2 : 0.0f;
 }
 
 static void quantize_row_q2_K_impl(const float * GGML_RESTRICT x, block_q2_K * GGML_RESTRICT y, int k, const float * GGML_RESTRICT quant_weights) {
@@ -4266,7 +4266,7 @@ static void quantize_row_iq1_s_impl(const float * GGML_RESTRICT x, void * GGML_R
                     sumw[j+1] = sumw[j] + weight[i];
                 }
             }
-            float best_score = -FLT_MIN, scale = max;
+            float best_score = -FLT_MAX, scale = max;
             int besti1 = -1, besti2 = -1, best_shift = 0;
             for (int i1 = 0; i1 <= block_size; ++i1) {
                 for (int i2 = i1; i2 <= block_size; ++i2) {
@@ -4442,7 +4442,7 @@ static void quantize_row_iq1_m_impl(const float * GGML_RESTRICT x, void * GGML_R
                 idx[2*j] = j;
             }
             qsort(pairs, block_size, 2*sizeof(float), iq1_sort_helper);
-            float best_score = -FLT_MIN, scale = max;
+            float best_score = -FLT_MAX, scale = max;
             int besti1 = -1, besti2 = -1, best_k = -1;
             // 0: +, +
             // 1: +, -