anon-iclr-submission/benchname-bug-localization huggingface dataset:
Repository: pypsa/pypsa
Issue: Unjustified consistency check warnings with multi-input/-output links

In a network allowing for n:1 or 1:m links (links with multiple input or output buses) `network.consistency_check()` throws warnings if there exist links in the network which do not use all input/output buses.

E.g.:
* the `link` component is extended to support `bus0` to `bus2`
* if a link only uses `bus0` and `bus1`
* then `network.consistency_check()` throws the following warning:

```
2022-11-17 08:09:08 pypsa.components WARNING  The following links have bus2 which are not defined:
```

## Checklist

- [x] I am using the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch or the latest [release](https://github.com/pypsa/pypsa/releases). Please indicate:

## Describe the Bug

In earlier versions of `consistency_check()` only the correct setting of `bus0` and `bus1` was evaluated.
In the newer version there seems to be no way to suppress the warning despite the link having intentionally no `bus2` connection.

## Error Message

```
2022-11-17 08:09:08 pypsa.components WARNING  The following links have bus2 which are not defined:
```


Changed files:
pypsa/components.py

diff --git a/pypsa/components.py b/pypsa/components.py
index 32263d32..03987e57 100644
--- a/pypsa/components.py
+++ b/pypsa/components.py
@@ -1327,6 +1327,10 @@ class Network(Basic):
         for c in self.iterate_components():
             for attr in bus_columns(c.df):
                 missing = ~c.df[attr].isin(self.buses.index)
+                # if bus2, bus3... contain empty strings do not warn
+                if c.name in self.branch_components:
+                    if int(attr[-1]) > 1:
+                        missing &= c.df[attr] != ""
                 if missing.any():
                     msg = "The following %s have %s which are not defined:\\n%s"
                     logger.warning(msg, c.list_name, attr, c.df.index[missing])
Repository: pypsa/pypsa
Issue: wrong curtailment calculation in statistics module

- [x] I am using the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch or the latest [release](https://github.com/pypsa/pypsa/releases). Please indicate:


```
p = (n.pnl(c).p_max_pu * n.df(c).p_nom_opt - n.pnl(c).p).clip(upper=0)
```
should be
```
p = (n.pnl(c).p_max_pu * n.df(c).p_nom_opt - n.pnl(c).p).clip(lower=0)
```


Changed files:
pypsa/statistics.py

diff --git a/pypsa/statistics.py b/pypsa/statistics.py
index f753776c..09db9ac5 100644
--- a/pypsa/statistics.py
+++ b/pypsa/statistics.py
@@ -368,6 +368,9 @@ class StatisticsAccessor:
         """
         Calculate the curtailment of components in the network.
 
+        The calculation only considers assets with a `p_max_pu` time series,
+        which is used to quantify the available power potential.
+
         For information on the list of arguments, see the docs in
         `Network.statistics` or `pypsa.statitics.StatisticsAccessor`.
         """
@@ -375,7 +378,7 @@ class StatisticsAccessor:
 
         @pass_empty_series_if_keyerror
         def func(n, c):
-            p = (n.pnl(c).p_max_pu * n.df(c).p_nom_opt - n.pnl(c).p).clip(upper=0)
+            p = (n.pnl(c).p_max_pu * n.df(c).p_nom_opt - n.pnl(c).p).clip(lower=0)
             weights = get_weightings(n, c)
             return aggregate_timeseries(p, weights, agg=aggregate_time)
Repository: pypsa/pypsa
Issue: `optimize.assign_solution` does not copy nominal capacities to optimal for fixed assets

<!-- Please do not post usage questions here. Ask them on the PyPSA mailing list: https://groups.google.com/forum/#!forum/pypsa -->

## Checklist

- [x] I am using the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch or the latest [release](https://github.com/pypsa/pypsa/releases). Please indicate:

## Describe the Bug

https://github.com/PyPSA/PyPSA/blob/af49562c08cb2bbbb755cda576a7a5035b9ba8f9/pypsa/optimization/optimize.py#L275-L278
does not trigger as soon as there are any extendable assets.



Changed files:
pypsa/optimization/optimize.py

diff --git a/pypsa/optimization/optimize.py b/pypsa/optimization/optimize.py
index c464ac72..e5544075 100644
--- a/pypsa/optimization/optimize.py
+++ b/pypsa/optimization/optimize.py
@@ -274,8 +274,9 @@ def assign_solution(n):
 
     # if nominal capacity was no variable set optimal value to nominal
     for (c, attr) in lookup.query("nominal").index:
-        if f"{c}-{attr}" not in m.variables:
-            n.df(c)[attr + "_opt"] = n.df(c)[attr]
+        fix_i = n.get_non_extendable_i(c)
+        if not fix_i.empty:
+            n.df(c).loc[fix_i, f"{attr}_opt"] = n.df(c).loc[fix_i, attr]
 
     # recalculate storageunit net dispatch
     if not n.df("StorageUnit").empty:
Repository: pypsa/pypsa
Issue: pypsa 0.22 doesn't work with matplotlib 3.4.2

 (ImportError: cannot import name 'colormaps' from 'matplotlib'), but it does work with 3.6.3 - add to dependencies environment.yaml

Changed files:
pypsa/plot.py

diff --git a/pypsa/plot.py b/pypsa/plot.py
index 542897b0..d709f2cb 100644
--- a/pypsa/plot.py
+++ b/pypsa/plot.py
@@ -21,7 +21,6 @@ import pandas as pd
 logger = logging.getLogger(__name__)
 
 import matplotlib.pyplot as plt
-from matplotlib import colormaps
 from matplotlib.collections import LineCollection, PatchCollection
 from matplotlib.legend_handler import HandlerPatch
 from matplotlib.patches import Circle, FancyArrow, Patch, Wedge
@@ -306,7 +305,7 @@ def plot(
 
         if bus_cmap is not None and c.dtype is np.dtype("float"):
             if isinstance(bus_cmap, str):
-                bus_cmap = colormaps.get_cmap(bus_cmap)
+                bus_cmap = plt.get_cmap(bus_cmap)
             if not bus_norm:
                 bus_norm = plt.Normalize(vmin=c.min(), vmax=c.max())
             c = c.apply(lambda cval: bus_cmap(bus_norm(cval)))
Repository: pypsa/pypsa
Issue: "p_nom": pd.Series.min aggregation default reduces capacity

<!-- Please do not post usage questions here. Ask them on the PyPSA mailing list: https://groups.google.com/forum/#!forum/pypsa -->

## Checklist

- [x] I am using the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch or the latest [release](https://github.com/pypsa/pypsa/releases). Please indicate:

## Describe the Bug

The default aggregation strategies are set in `networkclustering.py`
https://github.com/PyPSA/PyPSA/blob/daead70515d877fd273cc294ffa1734c218f2831/pypsa/networkclustering.py#L86-L98

We observed [here](https://github.com/pypsa-meets-africa/pypsa-africa/issues/431) the following behaviour using the PyPSA default aggregation strategies:

PyPSA-Africa: Powerplants in Nigeria **8GW**, after applying the aggregation strategy `p_nom: min` **5GW**, after fix `p_nom: sum` **8GW** 
PyPSA-Africa: Powerplants in Africa **121GW**, after applying the aggregation strategy `p_nom: min` **21GW**
PyPSA-Eur: Powerplant in Europe **476GW**, after applying the aggregation strategy `p_nom: min` **339GW**, after fix `p_nom: sum` **476GW**

I think the `"p_nom": pd.Series.sum` is an untended default setting introducing major capacity reductions. I would even argue that using `min` can be considered as bug. When clustering the powerplant capacities of e.g. 5 nodes, I want to have all 5 powerplants represented as `sum` in the resulting clustered node.  Therefore, I think we should make `"p_nom": pd.Series.sum` the default. 

Changed files:
pypsa/networkclustering.py

diff --git a/pypsa/networkclustering.py b/pypsa/networkclustering.py
index 7ac9ea46..4acb1b47 100644
--- a/pypsa/networkclustering.py
+++ b/pypsa/networkclustering.py
@@ -86,7 +86,7 @@ def aggregategenerators(
     strategies = {
         "p_nom_max": pd.Series.min,
         "weight": pd.Series.sum,
-        "p_nom": pd.Series.min,
+        "p_nom": pd.Series.sum,
         "capital_cost": pd.Series.sum,
         "efficiency": pd.Series.mean,
         "ramp_limit_up": pd.Series.mean,
Repository: pypsa/pypsa
Issue: Minor issue: network.objective not updated after failed lopf

A failed run with solvers in lopf (possibly other variants too) does not cause an Exception, which is good I think. Unless there's min_up_time and so forth some failed days don't hurt and can investigate after.
Minor issue is that network.objective is not updated it seems, but keeps the last value (of the day before). A simple check against network.objective to see whether the run succeeded fails, as does the sum of the objective. 
Both pyomo True and False are affected, also tested with Gurobi.

```
import pypsa
import numpy as np

n = pypsa.Network()
n.add('Bus', 'bus')
n.add('Generator', bus='bus', name='generator', p_nom=100, control='slack', marginal_cost=1)
n.add('Load', 'load', bus='bus')

n.set_snapshots(np.arange(3))
n.loads_t['p_set'].loc[:, 'load'] = np.array([80,120,90])

objective = []
for i in [0,1,2]:
    n.lopf(snapshots=i, solver_name='cbc', pyomo=True)
    objective.append(n.objective)

print(objective)
```

.. will yield [80,80,90] instead of [80, nan, 90].

Changed files:
pypsa/opf.py, pypsa/linopf.py

diff --git a/pypsa/linopf.py b/pypsa/linopf.py
index 9e5291da..994a9e3d 100644
--- a/pypsa/linopf.py
+++ b/pypsa/linopf.py
@@ -1489,6 +1489,7 @@ def network_lopf(
             f"Optimization failed with status {status} and "
             f"termination condition {termination_condition}"
         )
+        n.objective = np.nan
         return status, termination_condition
 
     n.objective = obj
diff --git a/pypsa/opf.py b/pypsa/opf.py
index beb07794..3e0364df 100644
--- a/pypsa/opf.py
+++ b/pypsa/opf.py
@@ -2340,6 +2340,7 @@ def network_lopf_solve(
             "Optimisation failed with status %s and terminal condition %s"
             % (status, termination_condition)
         )
+        network.objective = np.nan
 
     return status, termination_condition
Repository: pypsa/pypsa
Issue: pypsa does not work with Pyomo 6.4.3

<!-- Please do not post usage questions here. Ask them on the PyPSA mailing list: https://groups.google.com/forum/#!forum/pypsa -->

## Checklist

- [ ] I am using the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch or the latest [release](https://github.com/pypsa/pypsa/releases). Please indicate: 0.21.1

## Describe the Bug

`from pyomo.core.expr.logical_expr import inequality` in `opt.py` produces an error with pyomo 6.4.3

`ImportError: cannot import name 'inequality' from 'pyomo.core.expr.logical_expr'` as it was moved to `pyomo.core.expr.relational_expr`

To fix this, the import should be `from pyomo.environ import inequality` as this is meant as the public import without assuming the internal package structure of pyomo which might change again in the future.
https://github.com/Pyomo/pyomo/blob/main/pyomo/core/tests/unit/test_relational_expr.py#L24

Changed files:
pypsa/opt.py

diff --git a/pypsa/opt.py b/pypsa/opt.py
index e709161d..0b2da39b 100644
--- a/pypsa/opt.py
+++ b/pypsa/opt.py
@@ -27,9 +27,8 @@ import tempfile
 from contextlib import contextmanager
 
 from pyomo.core.base.constraint import _GeneralConstraintData
-from pyomo.core.expr.logical_expr import inequality
 from pyomo.core.expr.numeric_expr import LinearExpression
-from pyomo.environ import Constraint, Objective, Var, minimize
+from pyomo.environ import Constraint, Objective, Var, inequality, minimize
 from six.moves import cPickle as pickle
 
 # =============================================================================
Repository: pypsa/pypsa
Issue: linopf doesn't report time-dependent optimisation results if components are added after an optimisation

The `pyomo=False` version of LOPF in `linopf` has a small bug. If you:

- run an optimisation
- then add a generator/link/store
- rerun the optimisation
- then the time-dependent variables (p, p1) for the new components are not saved in `n.generators_t.p` etc

In the old code in `opf.py` a function called `allocate_series_dataframes` from `descriptors.py` was called before assigning results to avoid this behaviour. It essentially reindexes the dataframes with the latest component names.
Probably @FabianHofmann doesn't like this solution, otherwise I would add it myself :-).

Changed files:
test/test_bugs.py, pypsa/linopf.py

diff --git a/pypsa/linopf.py b/pypsa/linopf.py
index 72fb71eb..345bc8f4 100644
--- a/pypsa/linopf.py
+++ b/pypsa/linopf.py
@@ -950,7 +950,7 @@ def assign_solution(n, sns, variables_sol, constraints_dual,
         elif pnl[attr].empty:
             pnl[attr] = df.reindex(n.snapshots, fill_value=0)
         else:
-            pnl[attr].loc[sns, :] = df.reindex(columns=pnl[attr].columns)
+            pnl[attr].loc[sns, df.columns] = df
 
     pop = not keep_references
     def map_solution(c, attr):
diff --git a/test/test_bugs.py b/test/test_bugs.py
index 8842c36f..7c17c283 100644
--- a/test/test_bugs.py
+++ b/test/test_bugs.py
@@ -15,3 +15,14 @@ def test_344():
     network.add("Generator", "a", bus="a", p_nom=5)
 
     network.lopf(pyomo=False)
+
+
+def test_331():
+    n = pypsa.Network()
+    n.add("Bus", 'bus')
+    n.add('Load', 'load', bus='bus', p_set=10)
+    n.add('Generator', 'generator1', bus='bus', p_nom=15, marginal_cost=10)
+    n.lopf(pyomo=False)
+    n.add('Generator', 'generator2', bus='bus', p_nom=5, marginal_cost=5)
+    n.lopf(pyomo=False)
+    assert 'generator2' in n.generators_t.p
Repository: pypsa/pypsa
Issue: Linearised UC only valid if start up costs = shut down costs

<!-- Please do not post usage questions here. Ask them on the PyPSA mailing list: https://groups.google.com/forum/#!forum/pypsa -->

## Checklist

- [ ] I am using the current [`master`](https://github.com/pypsa/pypsa/tree/master) branch or the latest [release](https://github.com/pypsa/pypsa/releases). Please indicate:

## Describe the Bug

The linearised unit commitment formulation from [Hua et al.](https://ieeexplore.ieee.org/abstract/document/8000610) to tighten the relaxation of the unit commitment is only valid if the start up costs = shut down costs. There should be a check added and the additional constraints only added in case that the above condition is met.




Changed files:
pypsa/optimization/constraints.py

diff --git a/pypsa/optimization/constraints.py b/pypsa/optimization/constraints.py
index 94f1bfe1..1a2c82ac 100644
--- a/pypsa/optimization/constraints.py
+++ b/pypsa/optimization/constraints.py
@@ -233,7 +233,11 @@ def define_operational_constraints_for_committables(n, sns, c):
         n.model.add_constraints(status, "=", 0, name, mask=mask)
 
     # linearized approximation because committable can partly start up and shut down
-    if n._linearized_uc:
+    # only valid additional constraints if start up costs equal to shut down costs
+    cost_equal = all(
+        n.df(c).loc[com_i, "start_up_cost"] == n.df(c).loc[com_i, "shut_down_cost"]
+    )
+    if n._linearized_uc and cost_equal:
         # dispatch limit for partly start up/shut down for t-1
         lhs = (
             p.shift(snapshot=1)
Repository: adafruit/adafruit_learning_system_guides
Issue: ItsyBitsy Infinity Cube code comments and print statements incorrect

I believe the Infinity Cube code was adapted from my Circuit Playground Bluefruit NeoPixel Animator and Remote Control code, and the comments and print statements reflect the original code. The original code was meant to work with two Circuit Playground Bluefruits (so, button A and button B), and the Infinity Cube code is designed to work with the Bluefruit LE Connect app (so right arrow and left arrow). As it is, the comments and print statements are confusing.

Comments and print statements need to be updated to reflect usage of arrows in the app versus buttons on a CPB.

Changed files:
ItsyBitsy_Infinity_Cube/code.py

diff --git a/ItsyBitsy_Infinity_Cube/code.py b/ItsyBitsy_Infinity_Cube/code.py
index d34f36eb..26ecc1d3 100755
--- a/ItsyBitsy_Infinity_Cube/code.py
+++ b/ItsyBitsy_Infinity_Cube/code.py
@@ -1,5 +1,5 @@
 """
-NeoPixel Animator code for ItsyBitsy nRF52840 NeoPixel Animation and Color Remote Control.
+Code for ItsyBitsy nRF52840 Infinity Cube.
 """
 
 import board
@@ -61,7 +61,7 @@ while True:
             animations.animate()  # Run the animations.
         if ble.connected:  # If BLE is connected...
             was_connected = True
-            if uart.in_waiting:  # Check to see if any data is available from the Remote Control.
+            if uart.in_waiting:  # Check to see if any data is available.
                 try:
                     packet = Packet.from_stream(uart)  # Create the packet object.
                 except ValueError:
@@ -75,14 +75,14 @@ while True:
                         animations.color = animation_color  # Freeze the animation color.
                         print("Color:", animation_color)
                 elif isinstance(packet, ButtonPacket):  # If the packet is a button packet...
-                    if packet.pressed:  # If the buttons on the Remote Control are pressed...
-                        if packet.button == ButtonPacket.LEFT:  # If button A is pressed...
+                    if packet.pressed:  # If the buttons in the app are pressed...
+                        if packet.button == ButtonPacket.LEFT:  # If left arrow is pressed...
                             print("A pressed: animation mode changed.")
                             animations.next()  # Change to the next animation.
-                        elif packet.button == ButtonPacket.RIGHT:  # If button B is pressed...
+                        elif packet.button == ButtonPacket.RIGHT:  # If right arrow is pressed...
                             mode += 1  # Increase the mode by 1.
                             if mode == 1:  # If mode is 1, print the following:
-                                print("B pressed: color frozen!")
+                                print("Right pressed: color frozen!")
                             if mode > 1:  # If mode is > 1...
                                 mode = 0  # Set mode to 0, and print the following:
-                                print("B pressed: color changing!")
+                                print("Right pressed: color changing!")