Repository: Pyomo/pyomo

Issue:
EXPR.differentiate() get stuck for a quadratic expression with mode=reverse_symbolic
## Summary

<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `pyomo solve` to optimize a model and ..." -->

In the PR https://github.com/Pyomo/pyomo/pull/3034, the [linux/pypy-3.9](https://github.com/Pyomo/pyomo/actions/runs/7534575156/job/20509188360?pr=3034) test failed due to a timeout issue.
Upon investigating, it appears that the bottleneck lies in the `EXPR.differentiate()` function when applied to `model.c1`. This constraint is a simple quadratic inequation as follows.

```
c1 : Size=1, Index=None, Active=True
    Key  : Lower : Body                                          : Upper : Active
    None :  -Inf : (y1 - 0.5)*(y1 - 0.5) + (y2 - 0.5)*(y2 - 0.5) :  0.25 :   True
```
### Steps to reproduce the issue

<!-- Please include the command run and/or provide an example script that demonstrates the problem. -->

```
# example.py
from pyomo.contrib.mindtpy.tests.feasibility_pump1 import FeasPump1
import pyomo.core.expr as EXPR

model = FeasPump1()
model.c1.pprint()
vars_in_constr = list(EXPR.identify_variables(model.c1.body))

mode = EXPR.differentiate.Modes.reverse_symbolic
jac_list = EXPR.differentiate(model.c1.body, wrt_list=vars_in_constr, mode=mode)
print(jac_list)
```

### Error Message

<!-- If Pyomo reported an error, provide the error message. PLEASE INCLUDE THE ENTIRE STACK TRACE. -->

<!-- If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
$ # Output message here, including entire stack trace, if available
c1 : Size=1, Index=None, Active=True
    Key  : Lower : Body                                          : Upper : Active
    None :  -Inf : (y1 - 0.5)*(y1 - 0.5) + (y2 - 0.5)*(y2 - 0.5) :  0.25 :   True
```

### Information on your system

Pyomo version: 6.7.1.dev0
Python version: Python 3.9.18 (pypy)
Operating system: Linux (CentOS)
How Pyomo was installed (PyPI, conda, source): source


### Additional information

<!-- If you have any additional information, please list it here. -->


<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question (how to use Pyomo, what it can do, etc.), please refer to our [online documentation](https://github.com/Pyomo/pyomo#getting-help) on ways to get help. -->


EXPR.differentiate() get stuck for a quadratic expression with mode=reverse_symbolic
## Summary

<!-- Explain, in a clear and concise way, the command you ran and the result you were trying to achieve.
Example: "I ran `pyomo solve` to optimize a model and ..." -->

In the PR https://github.com/Pyomo/pyomo/pull/3034, the [linux/pypy-3.9](https://github.com/Pyomo/pyomo/actions/runs/7534575156/job/20509188360?pr=3034) test failed due to a timeout issue.
Upon investigating, it appears that the bottleneck lies in the `EXPR.differentiate()` function when applied to `model.c1`. This constraint is a simple quadratic inequation as follows.

```
c1 : Size=1, Index=None, Active=True
    Key  : Lower : Body                                          : Upper : Active
    None :  -Inf : (y1 - 0.5)*(y1 - 0.5) + (y2 - 0.5)*(y2 - 0.5) :  0.25 :   True
```
### Steps to reproduce the issue

<!-- Please include the command run and/or provide an example script that demonstrates the problem. -->

```
# example.py
from pyomo.contrib.mindtpy.tests.feasibility_pump1 import FeasPump1
import pyomo.core.expr as EXPR

model = FeasPump1()
model.c1.pprint()
vars_in_constr = list(EXPR.identify_variables(model.c1.body))

mode = EXPR.differentiate.Modes.reverse_symbolic
jac_list = EXPR.differentiate(model.c1.body, wrt_list=vars_in_constr, mode=mode)
print(jac_list)
```

### Error Message

<!-- If Pyomo reported an error, provide the error message. PLEASE INCLUDE THE ENTIRE STACK TRACE. -->

<!-- If it did not report an error but the output appears incorrect, provide the incorrect output. If there was no error message and no output but the result is incorrect, describe how it does not match what you expect. -->
```console
$ # Output message here, including entire stack trace, if available
c1 : Size=1, Index=None, Active=True
    Key  : Lower : Body                                          : Upper : Active
    None :  -Inf : (y1 - 0.5)*(y1 - 0.5) + (y2 - 0.5)*(y2 - 0.5) :  0.25 :   True
```

### Information on your system

Pyomo version: 6.7.1.dev0
Python version: Python 3.9.18 (pypy)
Operating system: Linux (CentOS)
How Pyomo was installed (PyPI, conda, source): source


### Additional information

<!-- If you have any additional information, please list it here. -->


<!-- We encourage you to try, as much as possible, to reduce your problem to the minimal example that still reproduces the issue. That would help us a lot in fixing it quickly and effectively!

If you want to ask a question (how to use Pyomo, what it can do, etc.), please refer to our [online documentation](https://github.com/Pyomo/pyomo#getting-help) on ways to get help. -->




Discussion:
Wow! Us being "lazy" and not computing the Jacobian by hand revealed this other issue! For #3034, we can compute the Jacobians by hand and check that the tests pass, but it's a really interesting find.
@ZedongPeng: Thanks for digging into this! It is indeed a bug in Pyomo's symbolic differentiation.  The fix is up and should be merged in in a couple hours.
Wow! Us being "lazy" and not computing the Jacobian by hand revealed this other issue! For #3034, we can compute the Jacobians by hand and check that the tests pass, but it's a really interesting find.
@ZedongPeng: Thanks for digging into this! It is indeed a bug in Pyomo's symbolic differentiation.  The fix is up and should be merged in in a couple hours.

Resolving patch:
diff --git a/pyomo/core/expr/numeric_expr.py b/pyomo/core/expr/numeric_expr.py
--- a/pyomo/core/expr/numeric_expr.py
+++ b/pyomo/core/expr/numeric_expr.py
@@ -1162,13 +1162,30 @@ def nargs(self):
 
     @property
     def args(self):
-        if len(self._args_) != self._nargs:
-            self._args_ = self._args_[: self._nargs]
-        return self._args_
+        # We unconditionally make a copy of the args to isolate the user
+        # from future possible updates to the underlying list
+        return self._args_[: self._nargs]
 
     def getname(self, *args, **kwds):
         return 'sum'
 
+    def _trunc_append(self, other):
+        _args = self._args_
+        if len(_args) > self._nargs:
+            _args = _args[: self._nargs]
+        _args.append(other)
+        return self.__class__(_args)
+
+    def _trunc_extend(self, other):
+        _args = self._args_
+        if len(_args) > self._nargs:
+            _args = _args[: self._nargs]
+        if len(other._args_) == other._nargs:
+            _args.extend(other._args_)
+        else:
+            _args.extend(other._args_[: other._nargs])
+        return self.__class__(_args)
+
     def _apply_operation(self, result):
         return sum(result)
 
@@ -1821,17 +1838,13 @@ def _add_native_monomial(a, b):
 def _add_native_linear(a, b):
     if not a:
         return b
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_native_sum(a, b):
     if not a:
         return b
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_native_other(a, b):
@@ -1872,15 +1885,11 @@ def _add_npv_monomial(a, b):
 
 
 def _add_npv_linear(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_npv_sum(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_npv_other(a, b):
@@ -1942,9 +1951,7 @@ def _add_param_linear(a, b):
         a = a.value
         if not a:
             return b
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_param_sum(a, b):
@@ -1952,9 +1959,7 @@ def _add_param_sum(a, b):
         a = value(a)
         if not a:
             return b
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_param_other(a, b):
@@ -1999,15 +2004,11 @@ def _add_var_monomial(a, b):
 
 
 def _add_var_linear(a, b):
-    args = b.args
-    args.append(MonomialTermExpression((1, a)))
-    return b.__class__(args)
+    return b._trunc_append(MonomialTermExpression((1, a)))
 
 
 def _add_var_sum(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_var_other(a, b):
@@ -2046,15 +2047,11 @@ def _add_monomial_monomial(a, b):
 
 
 def _add_monomial_linear(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_monomial_sum(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_monomial_other(a, b):
@@ -2069,15 +2066,11 @@ def _add_monomial_other(a, b):
 def _add_linear_native(a, b):
     if not b:
         return a
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_linear_npv(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_linear_param(a, b):
@@ -2085,33 +2078,23 @@ def _add_linear_param(a, b):
         b = b.value
         if not b:
             return a
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_linear_var(a, b):
-    args = a.args
-    args.append(MonomialTermExpression((1, b)))
-    return a.__class__(args)
+    return a._trunc_append(MonomialTermExpression((1, b)))
 
 
 def _add_linear_monomial(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_linear_linear(a, b):
-    args = a.args
-    args.extend(b.args)
-    return a.__class__(args)
+    return a._trunc_extend(b)
 
 
 def _add_linear_sum(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_linear_other(a, b):
@@ -2126,15 +2109,11 @@ def _add_linear_other(a, b):
 def _add_sum_native(a, b):
     if not b:
         return a
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_sum_npv(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_sum_param(a, b):
@@ -2142,39 +2121,27 @@ def _add_sum_param(a, b):
         b = b.value
         if not b:
             return a
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_sum_var(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_sum_monomial(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_sum_linear(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 def _add_sum_sum(a, b):
-    args = a.args
-    args.extend(b.args)
-    return a.__class__(args)
+    return a._trunc_extend(b)
 
 
 def _add_sum_other(a, b):
-    args = a.args
-    args.append(b)
-    return a.__class__(args)
+    return a._trunc_append(b)
 
 
 #
@@ -2213,9 +2180,7 @@ def _add_other_linear(a, b):
 
 
 def _add_other_sum(a, b):
-    args = b.args
-    args.append(a)
-    return b.__class__(args)
+    return b._trunc_append(a)
 
 
 def _add_other_other(a, b):
@@ -2628,8 +2593,8 @@ def _neg_var(a):
 
 
 def _neg_monomial(a):
-    args = a.args
-    return MonomialTermExpression((-args[0], args[1]))
+    coef, var = a.args
+    return MonomialTermExpression((-coef, var))
 
 
 def _neg_sum(a):


Test patch:
diff --git a/pyomo/core/tests/unit/test_derivs.py b/pyomo/core/tests/unit/test_derivs.py
--- a/pyomo/core/tests/unit/test_derivs.py
+++ b/pyomo/core/tests/unit/test_derivs.py
@@ -322,6 +322,18 @@ def test_nested_named_expressions(self):
         self.assertAlmostEqual(derivs[m.y], pyo.value(symbolic[m.y]), tol + 3)
         self.assertAlmostEqual(derivs[m.y], approx_deriv(e, m.y), tol)
 
+    def test_linear_exprs_issue_3096(self):
+        m = pyo.ConcreteModel()
+        m.y1 = pyo.Var(initialize=10)
+        m.y2 = pyo.Var(initialize=100)
+        e = (m.y1 - 0.5) * (m.y1 - 0.5) + (m.y2 - 0.5) * (m.y2 - 0.5)
+        derivs = reverse_ad(e)
+        self.assertEqual(derivs[m.y1], 19)
+        self.assertEqual(derivs[m.y2], 199)
+        symbolic = reverse_sd(e)
+        self.assertExpressionsEqual(symbolic[m.y1], m.y1 - 0.5 + m.y1 - 0.5)
+        self.assertExpressionsEqual(symbolic[m.y2], m.y2 - 0.5 + m.y2 - 0.5)
+
 
 class TestDifferentiate(unittest.TestCase):
     @unittest.skipUnless(sympy_available, "test requires sympy")
diff --git a/pyomo/core/tests/unit/test_numeric_expr.py b/pyomo/core/tests/unit/test_numeric_expr.py
--- a/pyomo/core/tests/unit/test_numeric_expr.py
+++ b/pyomo/core/tests/unit/test_numeric_expr.py
@@ -1618,9 +1618,9 @@ def test_nestedProduct2(self):
             ),
         )
         # Verify shared args...
-        self.assertIsNot(e1._args_, e2._args_)
-        self.assertIs(e1._args_, e3._args_)
-        self.assertIs(e1._args_, e.arg(1)._args_)
+        self.assertIs(e1._args_, e2._args_)
+        self.assertIsNot(e1._args_, e3._args_)
+        self.assertIs(e1._args_, e.arg(0)._args_)
         self.assertIs(e.arg(0).arg(0), e.arg(1).arg(0))
         self.assertIs(e.arg(0).arg(1), e.arg(1).arg(1))
 
diff --git a/pyomo/core/tests/unit/test_numeric_expr_api.py b/pyomo/core/tests/unit/test_numeric_expr_api.py
--- a/pyomo/core/tests/unit/test_numeric_expr_api.py
+++ b/pyomo/core/tests/unit/test_numeric_expr_api.py
@@ -950,7 +950,14 @@ def test_sum(self):
         f = e.create_node_with_local_data(e.args)
         self.assertIsNot(f, e)
         self.assertIs(type(f), type(e))
-        self.assertIs(f.args, e.args)
+        self.assertIsNot(f._args_, e._args_)
+        self.assertIsNot(f.args, e.args)
+
+        f = e.create_node_with_local_data(e._args_)
+        self.assertIsNot(f, e)
+        self.assertIs(type(f), type(e))
+        self.assertIs(f._args_, e._args_)
+        self.assertIsNot(f.args, e.args)
 
         f = e.create_node_with_local_data((m.x, 2, 3))
         self.assertIsNot(f, e)