Repository: PyPSA/PyPSA

Issue:
Bug with statistics function
### Version Checks (indicate both or one)

- [x] I have confirmed this bug exists on the lastest [release](https://github.com/pypsa/pypsa/releases) of PyPSA.

- [ ] I have confirmed this bug exists on the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch of PyPSA.


### Issue Description

Since the release Pypsa 0.32.0, all the extendable generators are not in the list when I do the statistics command: expanded_capacity(comps='Generator', groupby=False) or  optimal_capacity(comps='Generator', groupby=False)

When I execute the same command with pypsa 0.31, all works fine, I have the expanded capacity of extendable generators.
      

### Reproducible Example

```python
import pypsa
n = pypsa.Network()
stats = self.n.statistics
expanded_capacity = stats.expanded_capacity(comps='Generator', groupby=False).fillna(0)
```

### Expected Behavior

expanded_capacity should contain the expanded capacity for all generators but when I look the list, there is only the ones with p_nom_extendable = False and not the extendable generators.

### Installed Versions

<details>
Replace this line.
</details>



Discussion:
Please provide a reproducible example. Your example yields an empty list in any case.
Finally, the bug is not linked with the parameter "extendable" but with the build_year. Since pypsa 0.32.0, when you use the statistics function, only the generators/storage_units (and probably line/links too) built at the "start_year" or before are considered. If you execute the code below, you will see that only "gen1" and "su1" are in the list of "expanded_capacity":

### Code to execute
```python
import pypsa
import pandas as pd
n = pypsa.examples.ac_dc_meshed(from_master=True)
n.add(
    "Generator",
    "gen1",
    bus="Manchester",
    build_year = 2020)
n.add(
    "Generator",
    "gen2",
    bus="Manchester",
    build_year = 2021)
n.add(
    "Generator",
    "gen3",
    bus="Manchester",
    build_year = 2030)
n.add(
    "StorageUnit",
    "su1",
    bus="Manchester",
    build_year = 2020)
n.add(
    "StorageUnit",
    "su2",
    bus="Manchester",
    build_year = 2050)
years = [2020, 2030, 2040, 2050]
freq = "24"

snapshots = pd.DatetimeIndex([])
for year in years:
    period = pd.date_range(
        start=f"{year}-01-01 00:00",
        freq=f"{freq}H",
        periods=8760 / float(freq),
    )
    snapshots = snapshots.append(period)

# convert to multiindex and assign to network
n.snapshots = pd.MultiIndex.from_arrays([snapshots.year, snapshots])
n.investment_periods = years
n.optimize(multi_investment_periods=True)

stats = n.statistics
print(stats.expanded_capacity(comps='Generator', groupby=False).fillna(0))
print(stats.expanded_capacity(comps='StorageUnit', groupby=False).fillna(0))
```
Hello @lkstrp , Is there somebody on the subject? I haven't managed to solve the problem on my side...


Thanks for reporting that issue @adpr1. Even though it took us a while to drill down into it, we were able to uncover a serious regression we introduced in version 0.32 thanks to you.

Sorry for the wait, a bugfix release will follow in the coming days.



Patch:
diff --git a/doc/references/release-notes.rst b/doc/references/release-notes.rst
index 070e3d82e..ad7c2e743 100644
--- a/doc/references/release-notes.rst
+++ b/doc/references/release-notes.rst
@@ -30,6 +30,11 @@ Features
   (https://github.com/PyPSA/PyPSA/pull/1155)
 
 
+Bug fixes
+---------
+
+* **Regression hotfix**: Fixed a critical bug in statistics functions for multi-investment networks where built years and lifetimes were not being correctly considered. In version `v0.32.0`, only components active in the first time period were being included in statistics calculations. The fix ensures all components are properly represented according to their respective built years and lifetimes across all investment periods. (https://github.com/PyPSA/PyPSA/pull/1172)
+
 * The expressions function `n.optimize.expressions.capacity` now uses the absolute efficiency 
   to calculate the capacity at link ports, unless a `bus_carrier` is defined or `at_port` is set to True. 
   This is in line with the behavior of the statistics functions (`statistics.installed_capacity`, `statistics.optimal_capacity`). 
diff --git a/pypsa/statistics/abstract.py b/pypsa/statistics/abstract.py
index 2374b6d06..c37f4f694 100644
--- a/pypsa/statistics/abstract.py
+++ b/pypsa/statistics/abstract.py
@@ -245,14 +245,12 @@ def _filter_active_assets(self, n: Network, c: str, obj: Any) -> Any:
         idx = self._get_component_index(obj, c)
         if not self.is_multi_indexed:
             mask = n.get_active_assets(c)
-            idx = mask.index[mask].intersection(idx)
-            return obj.loc[idx]
+            return obj.loc[mask.index[mask].intersection(idx)]
 
         per_period = {}
         for p in n.investment_periods:
             mask = n.get_active_assets(c, p)
-            idx = mask.index[mask].intersection(idx)
-            per_period[p] = obj.loc[idx]
+            per_period[p] = obj.loc[mask.index[mask].intersection(idx)]
 
         return self._concat_periods(per_period, c)
 


Test patch:
diff --git a/test/test_bugs.py b/test/test_bugs.py
index 97b494dbd..17f258916 100644
--- a/test/test_bugs.py
+++ b/test/test_bugs.py
@@ -5,6 +5,17 @@
 import pypsa
 
 
+def test_1144():
+    """
+    See https://github.com/PyPSA/PyPSA/issues/1144.
+    """
+    n = pypsa.examples.ac_dc_meshed()
+    n.generators["build_year"] = [2020, 2020, 2030, 2030, 2040, 2040]
+    n.investment_periods = [2020, 2030, 2040]
+    capacity = n.statistics.installed_capacity(comps="Generator")
+    assert capacity[2020].sum() < capacity[2030].sum() < capacity[2040].sum()
+
+
 def test_890():
     """
     See https://github.com/PyPSA/PyPSA/issues/890.