Repository: materialsproject/pymatgen (Python)

Issue:
[BUG] Incorrect usage of `np.all() > tol` for element-wise value comparison
The following method: https://github.com/materialsproject/pymatgen/blob/bd9fba9ec62437b5b62fbd0b2c2c723216cc5a2c/src/pymatgen/io/lobster/outputs.py#L1694-L1737

Used a pattern like `np.array().all() > tolerance` with the intention to perform element-wise value comparison, however [this seems to be misused as `all()`](https://numpy.org/doc/stable/reference/generated/numpy.all.html#numpy.all):  
> "Test whether all array elements along a given axis evaluate to True."

```python
import numpy as np


arr = np.array([1, 2, 3])

print(arr.all())  # True

print(np.array([False, 1, 2]).all())  # False
```

Should be something like:
```diff
- if abs(band2 - 1.0).all() > limit_deviation:
# However this logic might need double check as it only evaluates to True 
# when ALL deviations are greater than threshold
+ if np.all(np.abs(band2 - 1) > limit_deviation):
# Perhaps it should be the following?
+ if np.any(np.abs(band2 - 1) > limit_deviation):

# The same to the other 
- elif band2.all() > limit_deviation:
+ elif np.all(band2 > limit_deviation)
# The following seems more sensible?
+ elif np.any(band2 > limit_deviation)

```

I might go ahead and change this if I didn't misunderstand anything? @JaGeo 



Discussion:
@naik-aakash Could you kindly check this (after Thursday ;))? Thanks a lot. 
Hi @DanielYang59 , thanks for pointing this out. Indeed there seems to have been an error with comparison logic here. Using np.any should be preferable here as you suggested I think.

I assume you have already started working on this issue, if not am also happy to raise a PR for this. Let me know.
Hi @naik-aakash greetings thanks for looking into this :)

> Using np.any should be preferable here as you suggested I think.

I might have caused confusion here, sorry. The main issue is not the usage of `any()` over `all()`, but the order of comparison vs `all/any` call. And we should do comparison **before** `all/any` evaluation:
> Test whether all array elements along a given axis evaluate to True.

Current implementation does the other way round, i.e. calling `array.all()` would evaluate the array to a `bool`, and we're basically comparing the bool (`0/1`) with `limit_deviation`.

---

> I assume you have already started working on this issue, if not am also happy to raise a PR for this. Let me know.

I might need your help to fix this, and it's apparently not covered by unit test. Perhaps you could enhance the unit test on top of fixing the implementation? Thank you!


Hey 😃 ,

I have an additional suggestion to improve this function further. Instead of looping through each value like that, 
https://github.com/materialsproject/pymatgen/blob/bd9fba9ec62437b5b62fbd0b2c2c723216cc5a2c/src/pymatgen/io/lobster/outputs.py#L1713-L1735
maybe we could aim for a more compact comparison:
```python
import numpy as np

def validate_band_overlaps(matrices, num_occ_bands, limit_deviation):
    """
    Validate diagonal and off-diagonal elements for band overlaps.
    """
    for matrix in matrices:
        matrix = np.array(matrix)
        # Limit matrix to occupied bands
        sub_matrix = matrix[:num_occ_bands, :num_occ_bands]

        # Check diagonal elements
        if np.any(np.abs(np.diag(sub_matrix) - 1.0) > limit_deviation):
            return False

        # Check off-diagonal elements
        if np.any(np.abs(sub_matrix - np.diag(np.diag(sub_matrix))) > limit_deviation):
            return False

    return True


def check_band_overlaps(band_overlaps_dict, number_occ_bands_spin_up, limit_deviation, spin_polarized=False, number_occ_bands_spin_down=None):
    """
    Check band overlaps for spin-up and optionally spin-down configurations.
    """
    # Validate spin-up matrices
    if not validate_band_overlaps(band_overlaps_dict[Spin.up]["matrices"], number_occ_bands_spin_up, limit_deviation):
        return False

    # Validate spin-down matrices if spin-polarized
    if spin_polarized:
        if number_occ_bands_spin_down is None:
            raise ValueError("number_occ_bands_spin_down must be specified.")
        if not validate_band_overlaps(band_overlaps_dict[Spin.down]["matrices"], number_occ_bands_spin_down, limit_deviation):
            return False

    return True
```
(*Disclaimer*: this is ChatGPT-generated but looks about like the code structure I would have proposed, maybe one can condense it a little bit).

What do you think about that, @naik-aakash ?

@QuantumChemist Brilliant idea, I didn't look closely into the code itself at the time of opening this issue, but I fully support your idea to reduce code repetition and nest level.

I'm not quite familiar with the expected type and shape of the `matrix` from `band_overlaps_dict[Spin.up]["matrices"]`, but looks like it's a 2D square array of floats? If that's the case, perhaps comparing its top-left `num_occ_bands x num_occ_bands` leading principal sub-array with an identity array? Something like:

```python
if spin_polarized and number_occ_bands_spin_down is None:
    raise ValueError("number_occ_bands_spin_down has to be specified")

for spin in (Spin.up, Spin.down) if spin_polarized else (Spin.up,):
    num_occ_bands = number_occ_bands_spin_up if spin == Spin.up else number_occ_bands_spin_down

    for array in self.band_overlaps_dict[spin]["matrices"]:
        # Looks like it is already np.array?
        sub_array = np.asarray(array)[:num_occ_bands, :num_occ_bands]

        if not np.allclose(sub_array, np.identity(num_occ_bands), atol=limit_deviation, rtol=0):
            return False

return True
```

---

I realized I might be misusing the word "matrix" as we're not doing linear algebra operations, I believe "array" might be a better term. I'm mentioning this because [NumPy is discouraging the usage of `np.matrix` even for real linear algebra usage](https://numpy.org/doc/stable/reference/generated/numpy.matrix.html):
> It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future.
Thank you so much, @QuantumChemist  and @DanielYang59, for the suggestions to reduce the code complexity. I will implement the suggestions and raise a PR soon  

And @DanielYang59 , you are right, it is not using matrix but already an numpy array but the term "matrix" is used in Bandoverlaps class just to ensure consistency with actual output files from LOBSTER.



Patch:
diff --git a/src/pymatgen/io/lobster/outputs.py b/src/pymatgen/io/lobster/outputs.py
index bc5f8e17800..300e2f68c90 100644
--- a/src/pymatgen/io/lobster/outputs.py
+++ b/src/pymatgen/io/lobster/outputs.py
@@ -1710,29 +1710,17 @@ def has_good_quality_check_occupied_bands(
         Returns:
             bool: True if the quality of the projection is good.
         """
-        for matrix in self.band_overlaps_dict[Spin.up]["matrices"]:
-            for iband1, band1 in enumerate(matrix):
-                for iband2, band2 in enumerate(band1):
-                    if iband1 < number_occ_bands_spin_up and iband2 < number_occ_bands_spin_up:
-                        if iband1 == iband2:
-                            if abs(band2 - 1.0).all() > limit_deviation:
-                                return False
-                        elif band2.all() > limit_deviation:
-                            return False
-
-        if spin_polarized:
-            for matrix in self.band_overlaps_dict[Spin.down]["matrices"]:
-                for iband1, band1 in enumerate(matrix):
-                    for iband2, band2 in enumerate(band1):
-                        if number_occ_bands_spin_down is None:
-                            raise ValueError("number_occ_bands_spin_down has to be specified")
-
-                        if iband1 < number_occ_bands_spin_down and iband2 < number_occ_bands_spin_down:
-                            if iband1 == iband2:
-                                if abs(band2 - 1.0).all() > limit_deviation:
-                                    return False
-                            elif band2.all() > limit_deviation:
-                                return False
+        if spin_polarized and number_occ_bands_spin_down is None:
+            raise ValueError("number_occ_bands_spin_down has to be specified")
+
+        for spin in (Spin.up, Spin.down) if spin_polarized else (Spin.up,):
+            num_occ_bands = number_occ_bands_spin_up if spin is Spin.up else number_occ_bands_spin_down
+
+            for overlap_matrix in self.band_overlaps_dict[spin]["matrices"]:
+                sub_array = np.asarray(overlap_matrix)[:num_occ_bands, :num_occ_bands]
+
+                if not np.allclose(sub_array, np.identity(num_occ_bands), atol=limit_deviation, rtol=0):
+                    return False
 
         return True
 


Test patch:
diff --git a/tests/io/lobster/test_outputs.py b/tests/io/lobster/test_outputs.py
index 30ea62e687f..4dac8c4a01b 100644
--- a/tests/io/lobster/test_outputs.py
+++ b/tests/io/lobster/test_outputs.py
@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import copy
 import json
 import os
 from unittest import TestCase
@@ -1481,7 +1482,7 @@ def test_get_bandstructure(self):
 
 class TestBandoverlaps(TestCase):
     def setUp(self):
-        # test spin-polarized calc and non spinpolarized calc
+        # test spin-polarized calc and non spin-polarized calc
 
         self.band_overlaps1 = Bandoverlaps(f"{TEST_DIR}/bandOverlaps.lobster.1")
         self.band_overlaps2 = Bandoverlaps(f"{TEST_DIR}/bandOverlaps.lobster.2")
@@ -1515,9 +1516,18 @@ def test_attributes(self):
         assert self.band_overlaps2.max_deviation[-1] == approx(1.48451e-05)
         assert self.band_overlaps2_new.max_deviation[-1] == approx(0.45154)
 
-    def test_has_good_quality(self):
+    def test_has_good_quality_maxDeviation(self):
         assert not self.band_overlaps1.has_good_quality_maxDeviation(limit_maxDeviation=0.1)
         assert not self.band_overlaps1_new.has_good_quality_maxDeviation(limit_maxDeviation=0.1)
+
+        assert self.band_overlaps1.has_good_quality_maxDeviation(limit_maxDeviation=100)
+        assert self.band_overlaps1_new.has_good_quality_maxDeviation(limit_maxDeviation=100)
+        assert self.band_overlaps2.has_good_quality_maxDeviation()
+        assert not self.band_overlaps2_new.has_good_quality_maxDeviation()
+        assert not self.band_overlaps2.has_good_quality_maxDeviation(limit_maxDeviation=0.0000001)
+        assert not self.band_overlaps2_new.has_good_quality_maxDeviation(limit_maxDeviation=0.0000001)
+
+    def test_has_good_quality_check_occupied_bands(self):
         assert not self.band_overlaps1.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=9,
             number_occ_bands_spin_down=5,
@@ -1545,65 +1555,58 @@ def test_has_good_quality(self):
         assert not self.band_overlaps1.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=1,
             number_occ_bands_spin_down=1,
-            limit_deviation=0.000001,
+            limit_deviation=1e-6,
             spin_polarized=True,
         )
         assert not self.band_overlaps1_new.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=1,
             number_occ_bands_spin_down=1,
-            limit_deviation=0.000001,
+            limit_deviation=1e-6,
             spin_polarized=True,
         )
         assert not self.band_overlaps1.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=1,
             number_occ_bands_spin_down=0,
-            limit_deviation=0.000001,
+            limit_deviation=1e-6,
             spin_polarized=True,
         )
         assert not self.band_overlaps1_new.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=1,
             number_occ_bands_spin_down=0,
-            limit_deviation=0.000001,
+            limit_deviation=1e-6,
             spin_polarized=True,
         )
         assert not self.band_overlaps1.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=0,
             number_occ_bands_spin_down=1,
-            limit_deviation=0.000001,
+            limit_deviation=1e-6,
             spin_polarized=True,
         )
         assert not self.band_overlaps1_new.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=0,
             number_occ_bands_spin_down=1,
-            limit_deviation=0.000001,
+            limit_deviation=1e-6,
             spin_polarized=True,
         )
         assert not self.band_overlaps1.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=4,
             number_occ_bands_spin_down=4,
-            limit_deviation=0.001,
+            limit_deviation=1e-3,
             spin_polarized=True,
         )
         assert not self.band_overlaps1_new.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=4,
             number_occ_bands_spin_down=4,
-            limit_deviation=0.001,
+            limit_deviation=1e-3,
             spin_polarized=True,
         )
-
-        assert self.band_overlaps1.has_good_quality_maxDeviation(limit_maxDeviation=100)
-        assert self.band_overlaps1_new.has_good_quality_maxDeviation(limit_maxDeviation=100)
-        assert self.band_overlaps2.has_good_quality_maxDeviation()
-        assert not self.band_overlaps2_new.has_good_quality_maxDeviation()
-        assert not self.band_overlaps2.has_good_quality_maxDeviation(limit_maxDeviation=0.0000001)
-        assert not self.band_overlaps2_new.has_good_quality_maxDeviation(limit_maxDeviation=0.0000001)
         assert not self.band_overlaps2.has_good_quality_check_occupied_bands(
-            number_occ_bands_spin_up=10, limit_deviation=0.0000001
+            number_occ_bands_spin_up=10, limit_deviation=1e-7
         )
         assert not self.band_overlaps2_new.has_good_quality_check_occupied_bands(
-            number_occ_bands_spin_up=10, limit_deviation=0.0000001
+            number_occ_bands_spin_up=10, limit_deviation=1e-7
         )
-        assert not self.band_overlaps2.has_good_quality_check_occupied_bands(
+        assert self.band_overlaps2.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=1, limit_deviation=0.1
         )
 
@@ -1614,7 +1617,7 @@ def test_has_good_quality(self):
             number_occ_bands_spin_up=1, limit_deviation=1e-8
         )
         assert self.band_overlaps2.has_good_quality_check_occupied_bands(number_occ_bands_spin_up=10, limit_deviation=1)
-        assert not self.band_overlaps2_new.has_good_quality_check_occupied_bands(
+        assert self.band_overlaps2_new.has_good_quality_check_occupied_bands(
             number_occ_bands_spin_up=2, limit_deviation=0.1
         )
         assert self.band_overlaps2.has_good_quality_check_occupied_bands(number_occ_bands_spin_up=1, limit_deviation=1)
@@ -1622,6 +1625,78 @@ def test_has_good_quality(self):
             number_occ_bands_spin_up=1, limit_deviation=2
         )
 
+    def test_has_good_quality_check_occupied_bands_patched(self):
+        """Test with patched data."""
+
+        limit_deviation = 0.1
+
+        rng = np.random.default_rng(42)  # set seed for reproducibility
+
+        band_overlaps = copy.deepcopy(self.band_overlaps1_new)
+
+        number_occ_bands_spin_up_all = list(range(band_overlaps.band_overlaps_dict[Spin.up]["matrices"][0].shape[0]))
+        number_occ_bands_spin_down_all = list(
+            range(band_overlaps.band_overlaps_dict[Spin.down]["matrices"][0].shape[0])
+        )
+
+        for actual_deviation in [0.05, 0.1, 0.2, 0.5, 1.0]:
+            for spin in (Spin.up, Spin.down):
+                for number_occ_bands_spin_up, number_occ_bands_spin_down in zip(
+                    number_occ_bands_spin_up_all, number_occ_bands_spin_down_all, strict=False
+                ):
+                    for i_arr, array in enumerate(band_overlaps.band_overlaps_dict[spin]["matrices"]):
+                        number_occ_bands = number_occ_bands_spin_up if spin is Spin.up else number_occ_bands_spin_down
+
+                        shape = array.shape
+                        assert np.all(np.array(shape) >= number_occ_bands)
+                        assert len(shape) == 2
+                        assert shape[0] == shape[1]
+
+                        # Generate a noisy background array
+                        patch_array = rng.uniform(0, 10, shape)
+
+                        # Patch the top-left sub-array (the part that would be checked)
+                        patch_array[:number_occ_bands, :number_occ_bands] = np.identity(number_occ_bands) + rng.uniform(
+                            0, actual_deviation, (number_occ_bands, number_occ_bands)
+                        )
+
+                        band_overlaps.band_overlaps_dict[spin]["matrices"][i_arr] = patch_array
+
+                    result = band_overlaps.has_good_quality_check_occupied_bands(
+                        number_occ_bands_spin_up=number_occ_bands_spin_up,
+                        number_occ_bands_spin_down=number_occ_bands_spin_down,
+                        spin_polarized=True,
+                        limit_deviation=limit_deviation,
+                    )
+                    # Assert for expected results
+                    if (
+                        actual_deviation == 0.05
+                        and number_occ_bands_spin_up <= 7
+                        and number_occ_bands_spin_down <= 7
+                        and spin is Spin.up
+                        or actual_deviation == 0.05
+                        and spin is Spin.down
+                        or actual_deviation == 0.1
+                        or actual_deviation in [0.2, 0.5, 1.0]
+                        and number_occ_bands_spin_up == 0
+                        and number_occ_bands_spin_down == 0
+                    ):
+                        assert result
+                    else:
+                        assert not result
+
+    def test_exceptions(self):
+        with pytest.raises(ValueError, match="number_occ_bands_spin_down has to be specified"):
+            self.band_overlaps1.has_good_quality_check_occupied_bands(
+                number_occ_bands_spin_up=4,
+                spin_polarized=True,
+            )
+        with pytest.raises(ValueError, match="number_occ_bands_spin_down has to be specified"):
+            self.band_overlaps1_new.has_good_quality_check_occupied_bands(
+                number_occ_bands_spin_up=4,
+                spin_polarized=True,
+            )
+
     def test_msonable(self):
         dict_data = self.band_overlaps2_new.as_dict()
         bandoverlaps_from_dict = Bandoverlaps.from_dict(dict_data)