Repository: Pyomo/pyomo

Issue:
Major performance regression for constraint creation in pyomo 5.7.1
**Summary**

There seems to be a major performance regression for creating constraints in version 5.7.1 compared to 5.7.0.

I compared the constraint creation for a model from a real use-case with around 150k constraints in a single `Constraint(set_a * set_b, rule=constraint_rule)`. The execution times were:

1. Pyomo 5.7.0: 1min 5sec
2. Pyomo 5.7.1: 45min 30sec

So Pyomo 5.7.1 is slower by about 40x compared to 5.7.0. 


**Background Info**

`set_a` has around 500 members, `set_b` around 300, so in total this results in about 150k constraints.

`constraint_rule` looks roughly like this:

```
def constraint_rule(model, id_a, id_b):
    constraint = sum(lookup_b_c[id_c, id_b] * model.x[id_a, id_c] for id_c in list_c) <= lookup_a_b
    if constraint is True:
        return Constraint.Feasible
    else:
        return constraint
```

I looked at the time spent in `constraint_rule` and it is not where the regression seems to come from, rather in between calls.

It took quite a bit of debugging to narrow this down, but I was very careful to make sure that for the timing above only the version of Pyomo and no other code / dependencies were changed.

This is only a guess, but #1592 might be a good first candidate for where the regression could have been introduced.

I have the model at hand, so I'd be happy to debug this further, if anyone can give me an indication on how to narrow it down.


Discussion:
Can you provide a minimal working example?  In particular, can you define the `lookup_b_c` and `list_c` objects and provide the Constraint declaration?  Also, your rule is missing a relational operator (is it something simple like `<= 0` or is it more complex)?
Thanks for the response, I'll provide a more complete example. You're right, I forgot the `<= ...`, I'll correct this in the example.
FWIW, I am currently unable to reproduce the behavior you are reporting.  Using the following simple example based on your report:
```python
# issue-1672.py
import sys
from pyomo.environ import *
print("python", sys.version)
print("pyomo", pyomo.version.version)
pyomo.common.timing.report_timing()

m = ConcreteModel()
m.A = RangeSet(500)
m.B = RangeSet(300)
m.C = RangeSet(100)
m.x = Var(m.A, m.C)
lookup = {(c, b): 2 for b in m.B for c in m.C}

def con(m, a, b):
    constraint = sum(lookup[c, b]* m.x[a,c] for c in m.C) <= 0
    if constraint is True:
        return Constraint.Feasible
    else:
        return constraint
m.con = Constraint(m.A, m.B, rule=con)
```
I see the following:
```
~ % module load pyomo/5.7
~ % python issue-1672.py
python 3.8.3 (default, Jun 18 2020, 23:23:48) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
pyomo 5.7
           0 seconds to construct Block ConcreteModel; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.06 seconds to construct Var x; 50000 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
       45.11 seconds to construct Constraint con; 150000 indicies total
~ % module unload pyomo
~ % module load pyomo/5.7.1
~ % python issue-1672.py
python 3.8.3 (default, Jun 18 2020, 23:23:48) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
pyomo 5.7.1
           0 seconds to construct Block ConcreteModel; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.05 seconds to construct Var x; 50000 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
       45.22 seconds to construct Constraint con; 150000 indicies total
~ % module unload pyomo
~ % module load pyomo/devel
~ % python issue-1672.py
python 3.8.3 (default, Jun 18 2020, 23:23:48) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
pyomo 5.7.2 (devel {master})
           0 seconds to construct Block ConcreteModel; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct RangeSet FiniteSimpleRangeSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.05 seconds to construct Var x; 50000 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
       55.77 seconds to construct Constraint con; 150000 indicies total
```
(The additional time for the development master is expected, as 5.7 and 5.7.1 are using cythonized modules and the development master is running in pure python).
Thanks a lot for the example, that helped a lot to get me started. I get similar results with your example.

I'm trying o narrow down the differences to my real-world example in order to reproduce what I see there.
So far no success on reproducing it with the smaller model, below are the timings for the big model (for a  slightly smaller instance than I used before). The problematic constraints are `con_A_B1` and `con_A_B2`, which are constructed the same, only that one is indexed by B1 and the other by B2 (which have different sizes).

```
python 3.8.6 (default, Sep 25 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)]
pyomo 5.7.1
           0 seconds to construct Block ConcreteModel; 1 index total
           0 seconds to construct Set C; 1 index total
           0 seconds to construct Param p_C; 1373 indicies total
           0 seconds to construct Set A; 1 index total
        0.08 seconds to construct Set D; 1 index total
           0 seconds to construct Set B1; 1 index total
           0 seconds to construct Set B2; 1 index total
        0.11 seconds to construct Param p1_D; 29620 indicies total
        0.11 seconds to construct Param p2_D; 29620 indicies total
        0.11 seconds to construct Param p3_D; 29620 indicies total
        0.15 seconds to construct Var x; 29620 indicies total
           0 seconds to construct Set E; 1 index total
           0 seconds to construct Var y; 114 indicies total
           0 seconds to construct Var z; 114 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.85 seconds to construct Param p_C_B1; 315790 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        3.28 seconds to construct Param p_A_B1; 120290 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
      1183.63 seconds to construct Constraint con_A_B1; 120290 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.25 seconds to construct Param p_C_B2; 68650 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        3.53 seconds to construct Param p_A_B2; 26150 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
      175.00 seconds to construct Constraint con_A_B2; 26150 indicies total
        0.08 seconds to construct Constraint con_C; 1373 indicies total
        0.02 seconds to construct Constraint con1_E; 114 indicies total
        0.02 seconds to construct Constraint con2_E; 114 indicies total
        0.08 seconds to construct Param p4_D; 29620 indicies total
           0 seconds to construct Param p1_E; 114 indicies total
           0 seconds to construct Param p2_E; 114 indicies total
        0.09 seconds to construct Objective obj; 1 index total
        
        
python 3.8.6 (default, Sep 25 2020, 00:00:00) 
[GCC 10.2.1 20200723 (Red Hat 10.2.1-1)]
pyomo 5.7
           0 seconds to construct Block ConcreteModel; 1 index total
           0 seconds to construct Set C; 1 index total
           0 seconds to construct Param p_C; 1373 indicies total
           0 seconds to construct Set A; 1 index total
        0.08 seconds to construct Set D; 1 index total
           0 seconds to construct Set B1; 1 index total
           0 seconds to construct Set B2; 1 index total
        0.12 seconds to construct Param p1_D; 29620 indicies total
        0.11 seconds to construct Param p2_D; 29620 indicies total
        0.12 seconds to construct Param p3_D; 29620 indicies total
        0.15 seconds to construct Var x; 29620 indicies total
           0 seconds to construct Set E; 1 index total
           0 seconds to construct Var y; 114 indicies total
           0 seconds to construct Var z; 114 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.85 seconds to construct Param p_C_B1; 315790 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        3.37 seconds to construct Param p_A_B1; 120290 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
       14.51 seconds to construct Constraint con_A_B1; 120290 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        0.25 seconds to construct Param p_C_B2; 68650 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
        3.43 seconds to construct Param p_A_B2; 26150 indicies total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
           0 seconds to construct Set SetProduct_OrderedSet; 1 index total
       10.79 seconds to construct Constraint con_A_B2; 26150 indicies total
        0.07 seconds to construct Constraint con_C; 1373 indicies total
        0.02 seconds to construct Constraint con1_E; 114 indicies total
        0.02 seconds to construct Constraint con2_E; 114 indicies total
        0.08 seconds to construct Param p4_D; 29620 indicies total
           0 seconds to construct Param p1_E; 114 indicies total
           0 seconds to construct Param p2_E; 114 indicies total
        0.09 seconds to construct Objective obj; 1 index total
```

I renamed the variable names (sorry for not being able to share the model). For the parameters I tried to make clear in the name what they are indexed by. For example, `p_C` is a parameter is indexed by `C`, whereas `p_C_B1` is indexed by `C` and `B1`.

I am unsure what more to try, I do not see any obvious qualitative differences to between the small and the big model. Any ideas on what I could investigate further?
At this point, I would recommend profiling. Here are a few functions I regularly use:

```
import cProfile
import pstats
import io
import line_profiler

def enable_line_profiler(funcs):
    profiler = line_profiler.LineProfiler()
    for _func in funcs:
        profiler.add_function(_func)
    profiler.enable()
    return profiler


def disable_line_profiler(profiler):
    profiler.disable()
    profiler.print_stats()


def enable_c_profiler():
    pr = cProfile.Profile()
    pr.enable()
    return pr


def disable_c_profiler(pr):
    pr.disable()
    s = io.StringIO()
    stats = pstats.Stats(pr, stream=s).sort_stats('cumulative')
    stats.print_stats(40)
    print(s.getvalue())
    s = io.StringIO()
    stats = pstats.Stats(pr, stream=s).sort_stats('tottime')
    stats.print_stats(40)
    print(s.getvalue())
```

`CProfile` is part of Python, but `line_profiler` has to be pip installed. I would start with `CProfile`. All you need to do is call 

```
pr = enable_c_profiler()
```

right before constructing `con_A_B1` and call 

```
disable_c_profiler(pr)
```

right after constructing `con_A_B1`. If you can share the output, great. If not, it might point you to the functions which are problematic.
Can you comment on the data types used?  In particular:
- in your example, are `lookup_b_c` and `lookup_a_b` Pyomo Param objects?
- How are you initializing your Param objects (what data type do you pass to `initialize=`, and what data types does it contain)?
- How are you initializing your Sets?  Can you share the declaration, and the data type that you use to initialize it?
@michaelbynum Thanks for the helpful code, here are the results (only for `con_A_B2`):

**Pyomo 2.7.1**
 ```
      305.84 seconds to construct Constraint con_A_B2; 26150 indicies total
         868020057 function calls (867947914 primitive calls) in 311.235 seconds

   Ordered by: cumulative time
   List reduced from 231 to 40 due to restriction <40>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        3    0.000    0.000  311.181  103.727 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/block.py:526(__setattr__)
      6/3    0.000    0.000  311.181  103.727 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/block.py:902(add_component)
        1    0.063    0.063  305.836  305.836 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:793(construct)
    26150    0.068    0.000  284.089    0.011 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:746(_setitem_when_not_present)
    26150    0.128    0.000  283.990    0.011 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:408(set_value)
    11931   12.181    0.001  283.762    0.024 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/component.py:826(index)
 84954552   31.412    0.000  271.578    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:357(iteritems)
 84954552   41.812    0.000  203.107    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:330(_sparse_iter_gen)
156243920   40.927    0.000  161.279    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3791(<genexpr>)
157739136  112.175    0.000  121.803    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3608(_flatten_product)
 87930771   28.128    0.000   53.288    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:362(__getitem__)
    26150    0.053    0.000   21.639    0.001 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/util.py:297(__call__)
    26150    0.138    0.000   21.585    0.001 /REDACTED/model.py:191(constraint_rule)
    26150    0.856    0.000   21.154    0.001 {built-in method builtins.sum}
  1507150    3.738    0.000   20.298    0.000 /REDACTED/model.py:203(<genexpr>)
 89472423   12.867    0.000   12.867    0.000 {method 'get' of 'dict' objects}
  1507150    0.755    0.000   11.182    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:515(_validate_index)
  1507150    1.049    0.000   10.427    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3624(get)
162636065/162588329   10.005    0.000   10.040    0.000 {built-in method builtins.len}
  1507150    1.602    0.000    7.927    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3633(_find_val)
  1507156    0.844    0.000    5.880    0.000 {built-in method builtins.all}
        1    0.010    0.010    5.407    5.407 /REDACTED/model.py:43(create_constraint)
        2    0.000    0.000    5.345    2.673 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:853(construct)
        2    0.061    0.031    5.345    2.672 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:654(_initialize_from)
  4521450    1.315    0.000    5.036    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3654(<genexpr>)
    94800    0.055    0.000    4.697    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/misc.py:53(apply_indexed_rule)
    26150    0.050    0.000    4.602    0.000 /REDACTED/model.py:130(init_param_1)
    26150    0.065    0.000    4.519    0.000 /REDACTED/model.py:254(get_something_1)
   130750    1.563    0.000    4.101    0.000 /REDACTED/model.py:215(get_something_2)
  3110610    1.006    0.000    4.075    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:465(__contains__)
  3015810    1.307    0.000    2.758    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:1243(get)
  8239507    2.459    0.000    2.459    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py:279(__get__)
  3015810    1.345    0.000    1.452    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:31(normalize_index)
  1507150    0.763    0.000    1.119    0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
    94800    0.065    0.000    0.457    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:591(_setitem_when_not_present)
  3064347    0.445    0.000    0.445    0.000 {built-in method builtins.hasattr}
    94800    0.040    0.000    0.393    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:633(_validate_value)
    26150    0.032    0.000    0.350    0.000 /REDACTED/model.py:31(get_something_3)
    94800    0.079    0.000    0.311    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:2390(get)
    26150    0.034    0.000    0.275    0.000 /REDACTED/model.py:44(<listcomp>)



         868020057 function calls (867947914 primitive calls) in 311.235 seconds

   Ordered by: internal time
   List reduced from 231 to 40 due to restriction <40>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
157739136  112.175    0.000  121.803    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3608(_flatten_product)
 84954552   41.812    0.000  203.107    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:330(_sparse_iter_gen)
156243920   40.927    0.000  161.279    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3791(<genexpr>)
 84954552   31.412    0.000  271.578    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:357(iteritems)
 87930771   28.128    0.000   53.288    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:362(__getitem__)
 89472423   12.867    0.000   12.867    0.000 {method 'get' of 'dict' objects}
    11931   12.181    0.001  283.762    0.024 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/component.py:826(index)
162636065/162588329   10.005    0.000   10.040    0.000 {built-in method builtins.len}
  1507150    3.738    0.000   20.298    0.000 /REDACTED/model.py:203(<genexpr>)
  8239507    2.459    0.000    2.459    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py:279(__get__)
  1507150    1.602    0.000    7.927    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3633(_find_val)
   130750    1.563    0.000    4.101    0.000 /REDACTED/model.py:215(get_something_2)
  3015810    1.345    0.000    1.452    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:31(normalize_index)
  4521450    1.315    0.000    5.036    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3654(<genexpr>)
  3015810    1.307    0.000    2.758    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:1243(get)
  1507150    1.049    0.000   10.427    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3624(get)
  3110610    1.006    0.000    4.075    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:465(__contains__)
    26150    0.856    0.000   21.154    0.001 {built-in method builtins.sum}
  1507156    0.844    0.000    5.880    0.000 {built-in method builtins.all}
  1507150    0.763    0.000    1.119    0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
  1507150    0.755    0.000   11.182    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:515(_validate_index)
  3064347    0.445    0.000    0.445    0.000 {built-in method builtins.hasattr}
   131273    0.231    0.000    0.242    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/isoweek.py:93(day)
  1507897    0.161    0.000    0.161    0.000 {built-in method builtins.isinstance}
  1495219    0.157    0.000    0.157    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/pyomoobject.py:31(is_expression_type)
    26150    0.138    0.000   21.585    0.001 /REDACTED/model.py:191(constraint_rule)
    26150    0.128    0.000  283.990    0.011 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:408(set_value)
    11934    0.121    0.000    0.181    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3785(_iter_impl)
  1481000    0.120    0.000    0.120    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/component.py:90(is_component_type)
  1481009    0.119    0.000    0.119    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/component.py:917(is_indexed)
    94800    0.114    0.000    0.165    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/range.py:168(__contains__)
  1481000    0.113    0.000    0.113    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/var.py:147(is_constant)
   120632    0.096    0.000    0.172    0.000 /REDACTED/model.py:196(get_something_4)
    94800    0.079    0.000    0.311    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:2390(get)
    26150    0.068    0.000  284.089    0.011 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:746(_setitem_when_not_present)
    26150    0.065    0.000    4.519    0.000 /REDACTED/model.py:254(get_something_1)
    94800    0.065    0.000    0.457    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:591(_setitem_when_not_present)
        1    0.063    0.063  305.836  305.836 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:793(construct)
        2    0.061    0.031    5.345    2.672 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:654(_initialize_from)
    94800    0.055    0.000    4.697    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/misc.py:53(apply_indexed_rule)
```


**Pyomo 2.7.0**

```
              21.97 seconds to construct Constraint con_A_B2; 26150 indicies total
         59456183 function calls (59455625 primitive calls) in 27.394 seconds

   Ordered by: cumulative time
   List reduced from 202 to 40 due to restriction <40>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        3    0.000    0.000   27.333    9.111 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/block.py:526(__setattr__)
      6/3    0.000    0.000   27.333    9.111 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/block.py:902(add_component)
   120950    0.085    0.000   26.434    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/misc.py:53(apply_indexed_rule)
        1    0.031    0.031   21.969   21.969 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:720(construct)
    26150    0.101    0.000   21.684    0.001 /REDACTED/model.py:191(constraint_rule)
    26150    0.830    0.000   21.318    0.001 {built-in method builtins.sum}
  1507150    3.418    0.000   20.488    0.000 /REDACTED/model.py:203(<genexpr>)
  2988150    2.702    0.000   16.815    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:360(__getitem__)
  1507150    0.751    0.000   12.137    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:513(_validate_index)
  1507150    0.996    0.000   11.387    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3585(get)
  1507150    2.908    0.000    8.986    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3594(_find_val)
  1507156    0.798    0.000    5.632    0.000 {built-in method builtins.all}
        1    0.010    0.010    5.425    5.425 /REDACTED/model.py:43(create_constraint)
        2    0.000    0.000    5.364    2.682 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:853(construct)
        2    0.061    0.031    5.364    2.682 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:654(_initialize_from)
  4521450    1.256    0.000    4.834    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3615(<genexpr>)
    26150    0.050    0.000    4.624    0.000 /REDACTED/model.py:130(init_param_1)
    26150    0.065    0.000    4.541    0.000 /REDACTED/model.py:254(get_something_1)
   130750    1.521    0.000    4.119    0.000 /REDACTED/model.py:215(get_something_2)
  3110610    1.008    0.000    3.929    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:465(__contains__)
  3015810    1.213    0.000    2.612    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:1243(get)
  8239507    2.521    0.000    2.521    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py:279(__get__)
  1628100    1.414    0.000    1.529    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3569(_flatten_product)
  3015810    1.294    0.000    1.399    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:31(normalize_index)
  1507150    0.736    0.000    1.084    0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
  4517868    0.899    0.000    0.899    0.000 {method 'get' of 'dict' objects}
6429579/6429567    0.463    0.000    0.463    0.000 {built-in method builtins.len}
    94800    0.063    0.000    0.453    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:591(_setitem_when_not_present)
  3014335    0.418    0.000    0.418    0.000 {built-in method builtins.hasattr}
    94800    0.040    0.000    0.390    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:633(_validate_value)
    26150    0.031    0.000    0.354    0.000 /REDACTED/model.py:31(get_something_3)
    94800    0.078    0.000    0.309    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:2375(get)
    26150    0.035    0.000    0.279    0.000 /REDACTED/model.py:44(<listcomp>)
   131273    0.235    0.000    0.246    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/isoweek.py:93(day)
    94800    0.020    0.000    0.218    0.000 {built-in method builtins.any}
   189600    0.045    0.000    0.211    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:2382(<genexpr>)
    26150    0.025    0.000    0.182    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:713(_setitem_when_not_present)
   120953    0.047    0.000    0.171    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3752(<genexpr>)
    94800    0.115    0.000    0.165    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/range.py:168(__contains__)
   120632    0.092    0.000    0.163    0.000 /REDACTED/model.py:196(get_something_4)



         59456183 function calls (59455625 primitive calls) in 27.394 seconds

   Ordered by: internal time
   List reduced from 202 to 40 due to restriction <40>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1507150    3.418    0.000   20.488    0.000 /REDACTED/model.py:203(<genexpr>)
  1507150    2.908    0.000    8.986    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3594(_find_val)
  2988150    2.702    0.000   16.815    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:360(__getitem__)
  8239507    2.521    0.000    2.521    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py:279(__get__)
   130750    1.521    0.000    4.119    0.000 /REDACTED/model.py:215(get_something_2)
  1628100    1.414    0.000    1.529    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3569(_flatten_product)
  3015810    1.294    0.000    1.399    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:31(normalize_index)
  4521450    1.256    0.000    4.834    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3615(<genexpr>)
  3015810    1.213    0.000    2.612    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:1243(get)
  3110610    1.008    0.000    3.929    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:465(__contains__)
  1507150    0.996    0.000   11.387    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3585(get)
  4517868    0.899    0.000    0.899    0.000 {method 'get' of 'dict' objects}
    26150    0.830    0.000   21.318    0.001 {built-in method builtins.sum}
  1507156    0.798    0.000    5.632    0.000 {built-in method builtins.all}
  1507150    0.751    0.000   12.137    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/indexed_component.py:513(_validate_index)
  1507150    0.736    0.000    1.084    0.000 <frozen importlib._bootstrap>:1017(_handle_fromlist)
6429579/6429567    0.463    0.000    0.463    0.000 {built-in method builtins.len}
  3014335    0.418    0.000    0.418    0.000 {built-in method builtins.hasattr}
   131273    0.235    0.000    0.246    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/isoweek.py:93(day)
  1495219    0.159    0.000    0.159    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/pyomoobject.py:31(is_expression_type)
  1507884    0.153    0.000    0.153    0.000 {built-in method builtins.isinstance}
  1481000    0.122    0.000    0.122    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/component.py:90(is_component_type)
  1481009    0.118    0.000    0.118    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/component.py:912(is_indexed)
    94800    0.115    0.000    0.165    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/range.py:168(__contains__)
  1481000    0.112    0.000    0.112    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/var.py:147(is_constant)
    26150    0.101    0.000   21.684    0.001 /REDACTED/model.py:191(constraint_rule)
   120632    0.092    0.000    0.163    0.000 /REDACTED/model.py:196(get_something_4)
   120950    0.085    0.000   26.434    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/misc.py:53(apply_indexed_rule)
    94800    0.078    0.000    0.309    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:2375(get)
    14219    0.073    0.000    0.100    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/constraint.py:385(set_value)
    26150    0.065    0.000    4.541    0.000 /REDACTED/model.py:254(get_something_1)
    94800    0.063    0.000    0.453    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:591(_setitem_when_not_present)
        2    0.061    0.031    5.364    2.682 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:654(_initialize_from)
    26150    0.050    0.000    4.624    0.000 /REDACTED/model.py:130(init_param_1)
   120953    0.047    0.000    0.171    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:3752(<genexpr>)
   189600    0.045    0.000    0.211    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/set.py:2382(<genexpr>)
    94800    0.040    0.000    0.390    0.000 /REDACTED/virtualenv/lib/python3.8/site-packages/pyomo/core/base/param.py:633(_validate_value)
    26150    0.035    0.000    0.279    0.000 /REDACTED/model.py:44(<listcomp>)
    68650    0.032    0.000    0.041    0.000 /REDACTED/model.py:72(init_param_2)
    26150    0.031    0.000    0.354    0.000 /REDACTED/model.py:31(get_something_3)
```

Sorry again for the redactions, I hope it is useful nonetheless.

@jsiirola It looks like a lot of time is spent in `_flatten_product` in `set.py`. It looks like that function is being called 157 million times. Any thoughts?
@jsiirola 
 - Yes, they are both Param objects.
 - Parameters are initialized as follows:

```
model.p_C_B2 = Param(
    model.C,
    model.B2,
    initialize=init_parameter_C_B2,
    within=NonNegativeIntegers,
)
```

`init_parameter_C_B2` is a function that returns `int`. It is mostly a wrappers to look up values in dictionaries that are pre-computed, except for calls to `get_something_1` as in profiles above. `list_c` is a list.
 - Sets are initialized as follows:

```
model.W = Set(
    initialize=sorted(some_set),
)
```
Here `some_set` is a set. The types contained in the set vary (but are all the same within one set), they are either of type `str`, `datetime.date` (`con_A_B1`), or `tuple[int]` (`con_A_B2`). 
Here are some specific examples for the values contained in each set:

- `m.A`: `"a52b0ddf-be6c-4485-8d17-d674d084ddee"`
 - `m.B1`: `datetime.date(2020, 10, 21)`
 - `m.B2`: `(2020, 10)`
 - `m.C`: `"d437495a-a1ab-47b4-8238-2e438e5f8408"`
 - `m.D`: `("6dd2b80f-ea38-4a95-9b79-8942dafe39a0", "073c8dfb-129f-4dfa-bc82-f9aeefa53c12")`
OK.  @lapseofreason, try converting your rule from `return Constraint.Feasible` to `return Constraint.Skip`.

Here's what I think is going on (mostly documentation for my and others' later reference):

We centralized some of the logic around handling `Skip` so that instead of forcing every component that wants to allow sparse initializers to implement their own `Foo.Skip` object and handlers, we handle that with a common `IndexedComponent.Skip` object within `IndexedComponent._setitem_when_not_present()`.  This is still reasonably efficient. The problem is that the centralized implementation of `_setitem_when_not_present()` doesn't know anything about other flags that a derived component allows (like `Constraint.Feasible`). `Constraint.Feasible` is handled synonymously to `Constraint.Skip`: the `_ConstraintData` is removed from the Component.  However, because that now occurs within `Constraint.set_value()`, we run afoul of two things:
 - the _ConstraintData object still gets created and then has to be subsequently removed
 - `_ConstraintData` objects do not know their own index.  For `set_value()` to be able to delete the `_ConstraintData`, it has to do a linear scan through the indexing set in order to find the appropriate index value to delete.

Long term, #1228 will fix things like this.

For the next release, I think implementing a `Constraint._setitem_when_not_present()` that maps `Constraint.Feasible` to `IndexedComponent.Skip` will be a reasonable patch (alternatively, we just make `Constraint.Feasible` be an alias for `Constraint.Skip` - which given how we handle it, would be simpler and even more eficient).
@jsiirola Thanks a lot, that does indeed resolve the issue! Keep up the good work, I really appreciate it ;)

Resolving patch:
diff --git a/pyomo/core/base/constraint.py b/pyomo/core/base/constraint.py
--- a/pyomo/core/base/constraint.py
+++ b/pyomo/core/base/constraint.py
@@ -494,15 +494,14 @@ def set_value(self, expr):
             self._upper = None
             self._equality = False
 
-            if expr is Constraint.Infeasible:
+            if expr is Constraint.Skip:
+                del self.parent_component()[self.index()]
+                return
+            elif expr is Constraint.Infeasible:
                 del self.parent_component()[self.index()]
                 raise ValueError(
                     "Constraint '%s' is always infeasible"
                     % (self.name,) )
-            elif ( expr is Constraint.Skip or
-                   expr is Constraint.Feasible ):
-                del self.parent_component()[self.index()]
-                return
             else:
                 raise ValueError(
                     "Constraint '%s' does not have a proper "
@@ -760,7 +759,7 @@ class Constraint(ActiveIndexedComponent):
 
     _ComponentDataClass = _GeneralConstraintData
     class Infeasible(object): pass
-    class Feasible(object): pass
+    Feasible = ActiveIndexedComponent.Skip
     NoConstraint = ActiveIndexedComponent.Skip
     Violated = Infeasible
     Satisfied = Feasible


Test patch: