Repository: Pyomo/pyomo

Issue:
NL writers linear presolver fails when one variable is left
## Summary
The NL writer's linear presolver will fail if a single variable is left in the model.

### Steps to reproduce the issue

```python
from pyomo.environ import *                                                                                                                          

m = ConcreteModel()
m.x = Var(bounds=(0, None))
m.y = Var()
m.c = Constraint(expr = m.y == m.x)
m.o = Objective(expr=0)

s = SolverFactory("ipopt_v2")
# s.config.writer_config.linear_presolve = False
s.solve(m)
```

### Error Message

<!-- If Pyomo reported an error, provide the error message. PLEASE INCLUDE THE ENTIRE STACK TRACE. -->

<!-- If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
Traceback (most recent call last):
  File "/Users/bknueven/Software/pyomo_test.py", line 11, in <module>
    s.solve(m)
  File "/Users/bknueven/Software/pyomo/pyomo/contrib/solver/base.py", line 522, in solve
    results: Results = super().solve(model)
  File "/Users/bknueven/Software/pyomo/pyomo/contrib/solver/ipopt.py", line 318, in solve
    nl_info = self._writer.write(
  File "/Users/bknueven/Software/pyomo/pyomo/repn/plugins/nl_writer.py", line 409, in write
    return impl.write(model)
  File "/Users/bknueven/Software/pyomo/pyomo/repn/plugins/nl_writer.py", line 1086, in write
    _vmap[_id] = nl.rstrip() % tuple(_vmap[_id] for _id in args)
  File "/Users/bknueven/Software/pyomo/pyomo/repn/plugins/nl_writer.py", line 1086, in <genexpr>
    _vmap[_id] = nl.rstrip() % tuple(_vmap[_id] for _id in args)
KeyError: 140241181484176
```

### Information on your system

Pyomo version: main (f8b02390e3a6e4c14c6486f2c969482f3420ccfb)
Python version: 3.10.9
Operating system: macOS
How Pyomo was installed (PyPI, conda, source): source
Solver (if applicable): N/A


### Additional information

<!-- If you have any additional information, please list it here. -->


<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question (how to use Pyomo, what it can do, etc.), please refer to our [online documentation](https://github.com/Pyomo/pyomo#getting-help) on ways to get help. -->

Turning off the linear presolver gives the expected result.



Discussion:
Thanks for testing this out!
OK - this is an interesting edge case: you have a model with a non-square subsystem that can be completely presolved out of the model. That is, there is a set of _n_ constraints that act on _m_ (with _m_>_n_) variables, and those variables do not appear in any other constraints / objectives.  In the original writer (or the new writer with presolve off), those variables and constraints are sent to the solver and the solver will pick (random) values for the _m-n_ degrees of freedom.  After presolve, the _n_ constraints are removed from the model - so when the writer collects / categorizes the model variables, none of the _m_ variables are found.  This is generally fine - but causes issues when we attempt to find values for the eliminated variables (as the _m-n_ degrees of freedom are not in the `var_map`).

What do you think the expected behavior should be?
1. leave the variables removed from the model alone (and they will be marked stale)
2. arbitrarily pick a value for the DOF and propagate the values through; but which value to pick?
  a. pick a value as close to 0 as allowed by the bounds
  b. pick the lower (or upper) bound
  c. pick the midpoint between the bounds

I think I would start with proposing 2.a.
I tried out a few different LP solvers, and their own preprocessing techniques tend to pick a bound -- and this is what an active set method without presolve does too. But if you use an interior point method without presolve you get something closer to 2.c. If the free variable(s) don't have any bounds you seem to get 0.

So I agree that 2.a is the best of the options available. It conforms to existing practice while not being onerous to implement and maintain.
2.a was the consensus when this was discussed at the Pyomo dev call but we also think this case should log a warning about a modeling issue.

Resolving patch:
diff --git a/pyomo/repn/plugins/nl_writer.py b/pyomo/repn/plugins/nl_writer.py
--- a/pyomo/repn/plugins/nl_writer.py
+++ b/pyomo/repn/plugins/nl_writer.py
@@ -1081,9 +1081,37 @@ def write(self, model):
 
         # Update any eliminated variables to point to the (potentially
         # scaled) substituted variables
-        for _id, expr_info in eliminated_vars.items():
+        for _id, expr_info in list(eliminated_vars.items()):
             nl, args, _ = expr_info.compile_repn(visitor)
-            _vmap[_id] = nl.rstrip() % tuple(_vmap[_id] for _id in args)
+            for _i in args:
+                # It is possible that the eliminated variable could
+                # reference another variable that is no longer part of
+                # the model and therefore does not have a _vmap entry.
+                # This can happen when there is an underdetermined
+                # independent linear subsystem and the presolve removed
+                # all the constraints from the subsystem.  Because the
+                # free variables in the subsystem are not referenced
+                # anywhere else in the model, they are not part of the
+                # `variables` list.  Implicitly "fix" it to an arbitrary
+                # valid value from the presolved domain (see #3192).
+                if _i not in _vmap:
+                    lb, ub = var_bounds[_i]
+                    if lb is None:
+                        lb = -inf
+                    if ub is None:
+                        ub = inf
+                    if lb <= 0 <= ub:
+                        val = 0
+                    else:
+                        val = lb if abs(lb) < abs(ub) else ub
+                    eliminated_vars[_i] = AMPLRepn(val, {}, None)
+                    _vmap[_i] = expr_info.compile_repn(visitor)[0]
+                    logger.warning(
+                        "presolve identified an underdetermined independent "
+                        "linear subsystem that was removed from the model.  "
+                        f"Setting '{var_map[_i]}' == {val}"
+                    )
+            _vmap[_id] = nl.rstrip() % tuple(_vmap[_i] for _i in args)
 
         r_lines = [None] * n_cons
         for idx, (con, expr_info, lb, ub) in enumerate(constraints):


Test patch:
diff --git a/pyomo/repn/tests/ampl/test_nlv2.py b/pyomo/repn/tests/ampl/test_nlv2.py
--- a/pyomo/repn/tests/ampl/test_nlv2.py
+++ b/pyomo/repn/tests/ampl/test_nlv2.py
@@ -1812,6 +1812,78 @@ def test_presolve_zero_coef(self):
             )
         )
 
+    def test_presolve_independent_subsystem(self):
+        # This is derived from the example in #3192
+        m = ConcreteModel()
+        m.x = Var()
+        m.y = Var()
+        m.z = Var()
+        m.d = Constraint(expr=m.z == m.y)
+        m.c = Constraint(expr=m.y == m.x)
+        m.o = Objective(expr=0)
+
+        ref = """g3 1 1 0       #problem unknown
+ 0 0 1 0 0     #vars, constraints, objectives, ranges, eqns
+ 0 0 0 0 0 0   #nonlinear constrs, objs; ccons: lin, nonlin, nd, nzlb
+ 0 0   #network constraints: nonlinear, linear
+ 0 0 0 #nonlinear vars in constraints, objectives, both
+ 0 0 0 1       #linear network variables; functions; arith, flags
+ 0 0 0 0 0     #discrete variables: binary, integer, nonlinear (b,c,o)
+ 0 0   #nonzeros in Jacobian, obj. gradient
+ 1 0   #max name lengths: constraints, variables
+ 0 0 0 0 0     #common exprs: b,c,o,c1,o1
+O0 0   #o
+n0
+x0     #initial guess
+r      #0 ranges (rhs's)
+b      #0 bounds (on variables)
+k-1    #intermediate Jacobian column lengths
+"""
+
+        OUT = io.StringIO()
+        with LoggingIntercept() as LOG:
+            nlinfo = nl_writer.NLWriter().write(
+                m, OUT, symbolic_solver_labels=True, linear_presolve=True
+            )
+        self.assertEqual(
+            LOG.getvalue(),
+            "presolve identified an underdetermined independent linear subsystem "
+            "that was removed from the model.  Setting 'z' == 0\n",
+        )
+
+        self.assertEqual(*nl_diff(ref, OUT.getvalue()))
+
+        m.x.lb = 5.0
+
+        OUT = io.StringIO()
+        with LoggingIntercept() as LOG:
+            nlinfo = nl_writer.NLWriter().write(
+                m, OUT, symbolic_solver_labels=True, linear_presolve=True
+            )
+        self.assertEqual(
+            LOG.getvalue(),
+            "presolve identified an underdetermined independent linear subsystem "
+            "that was removed from the model.  Setting 'z' == 5.0\n",
+        )
+
+        self.assertEqual(*nl_diff(ref, OUT.getvalue()))
+
+        m.x.lb = -5.0
+        m.z.ub = -2.0
+
+        OUT = io.StringIO()
+        with LoggingIntercept() as LOG:
+            nlinfo = nl_writer.NLWriter().write(
+                m, OUT, symbolic_solver_labels=True, linear_presolve=True
+            )
+        self.assertEqual(
+            LOG.getvalue(),
+            "presolve identified an underdetermined independent linear subsystem "
+            "that was removed from the model.  Setting 'z' == -2.0\n",
+        )
+
+        self.assertEqual(*nl_diff(ref, OUT.getvalue()))
+
     def test_scaling(self):
         m = pyo.ConcreteModel()
         m.x = pyo.Var(initialize=0)