Repository: Pyomo/pyomo

Issue:
Passing initial solution to CBC solver
Hello!
While working on a personal project I needed to pass an initial solution for a MIP problem (for example a solution generated with an heuristic) to the CBC solver. CBC supports that with the 'mipstart' option (passing a soln file with the variables' values). I couldn't find support for this in pyomo. Is there a way to achieve that?
I implemented a workaround:
I'd first set up the initial values in the concrete instance and then call this function:
```python
def write_solution(instance):
    with open('solution.soln', 'w') as solution:
        for v in instance.component_objects(Var, active=True):
            for index in v:
                if str(v[index].domain) == 'Binary' and v[index].value:
                    trans = str.maketrans('[,]', '(_)')
                    print(
                        '0 {} {}'.format(
                            v[index].name.translate(trans), v[index].value
                        ),
                        file=solution
                    )
```
If you think this could be useful, I could clean it up a little bit and submit a PR, what do you think?


Discussion:
I think this could be very useful and Pull Requests are always welcome! Some thoughts as you work on it:
  - For this to work, I believe that your “.soln writer” will need to use the same symbol_map as the LP writer. Your example hints that you are using symbolic solver labels (which is not the default). If you include this as part of the CBC Solver, you should be able to get access to the symbol_map. @michaelbynum: we should add his use case to the list for the new solver API. 
  - For robustness, if you cannot get the symbol msp that the LP writer generated, you should at least use the same labeler that the LP writer uses. 
  - Be cautious comparing to strings. It would probably be better to compare against the actual domain, or to use the “is_*” methods in the domain. 
  - Is there a reason you are only outputting the Binary variables? My understanding is that some solvers will only accept the warm start if it is completely feasible - I don’t know if that is the case for CBC. 
  - several Pyomo solvers accept a `warmstart=True` option to enable this kind of behavior. It would’ve great to hook into that api. 
   
Thank you for the comments. The section of code I pasted is what I'm using now in my project. I'll make it generic if I'm going to submit  a PR.
I'll try to access the symbol map, remove the hacky string comparison (I added because var.domain == Binary yield false in my code). 
I think cbc only wants initial values for the integer variables (which in my case were all binary). I'll make that generic as well.
I'd try to add it this to the warmstart option as well.
Do you happen to have any code pointers of any of the other solver plugins doing similar things?
Was just about to raise this issue.

Search for def _warm_start(self, instance) for inspiration on how this is done for Gurobi and CPLEX.

Changes needed:
add _warm_start_solve and _warm_start_file_name attributes
warm_start_capable to return true when problem format is LP and CBC version >= 2.8.0
Create temp soln file when user doesn't provide a file
Populate temp file with discrete values using _warm_start function when user doesn't provide a file
Print filename when keepfiles=True and user doesn't provide a file
Edit create_command_line to include mipstart
@quimey Are you considering doing a PR for this? Because I wouldn't mind having a crack at it.
I started working at it but then I realize I was mostly going to copy a bunch of code from the cplex wrapper. I thought that some refactoring would be appropriate but I haven't had the time to read and understand the code well enough to do that.
What do you think?
I think it would be useful to stick up a WIP branch with whatever stage you got to, then we can look over the changes and discuss :)
I don't have much done but let me clean it up and push it after.
I just checked and I have basically nothing. Also I'm busy right now, so go ahead and have your crack at it!
I'll be open to discuss some details if that'd helpful for you
I was looking through the source for the CBC plugin, but I could not figure out where symbol_map is used there. I noticed that passing `symbolic_solver_labels` to `solver.solve` for CBC will use them in the temporary LP-File, but I'm not sure how to get the map.

@jsiirola: Given an instance, how does one get the symbol_map that is used?
It depends on the model, and whether the top block comes from kernel or the normal aml.  Looking at the `_warm_start()` method in the CPLEX / Gurobi interfaces, the idiom appears to be:
```
if isinstance(instance, IBlock):
    smap = getattr(instance,"._symbol_maps")[self._smap_id]
else:
    smap = instance.solutions.symbol_map[self._smap_id]
```
@jsiirola Thanks, that was helpful. I'm working on this.

Resolving patch:
diff --git a/pyomo/solvers/plugins/solvers/CBCplugin.py b/pyomo/solvers/plugins/solvers/CBCplugin.py
--- a/pyomo/solvers/plugins/solvers/CBCplugin.py
+++ b/pyomo/solvers/plugins/solvers/CBCplugin.py
@@ -12,15 +12,19 @@
 
 import os
 import re
+import time
 import logging
 
 from six import iteritems
+from six import string_types
 
 import pyomo.common
 import pyutilib.misc
 import pyutilib.common
 import pyutilib.subprocess
 
+from pyomo.core.base import Var
+from pyomo.core.kernel.block import IBlock
 from pyomo.opt.base import *
 from pyomo.opt.base.solvers import _extract_version
 from pyomo.opt.results import *
@@ -130,7 +134,14 @@ def __init__(self, **kwds):
         # Call base constructor
         #
         kwds['type'] = 'cbc'
-        SystemCallSolver.__init__(self, **kwds)
+        super(CBCSHELL, self).__init__(**kwds)
+
+        # NOTE: eventually both of the following attributes should be migrated to a common base class.
+        # is the current solve warm-started? a transient data member to communicate state information
+        # across the _presolve, _apply_solver, and _postsolve methods.
+        self._warm_start_solve = False
+        # related to the above, the temporary name of the SOLN warm-start file (if any).
+        self._warm_start_file_name = None
 
         #
         # Set up valid problem formats and valid results for each problem format
@@ -178,10 +189,104 @@ def _default_results_format(self, prob_format):
             return ResultsFormat.sol
         return ResultsFormat.soln
 
-    # Nothing needs to be done here
-    #def _presolve(self, *args, **kwds):
-    #    # let the base class handle any remaining keywords/actions.
-    #    SystemCallSolver._presolve(self, *args, **kwds)
+    def warm_start_capable(self):
+        if self._problem_format == ProblemFormat.cpxlp \
+                and _cbc_version >= (2,8,0,0):
+            return True
+        else:
+            return False
+
+    def _write_soln_file(self, instance, filename):
+
+        # Maybe this could be a useful method for any instance.
+
+        if isinstance(instance, IBlock):
+            smap = getattr(instance, "._symbol_maps")[self._smap_id]
+        else:
+            smap = instance.solutions.symbol_map[self._smap_id]
+        byObject = smap.byObject
+
+        column_index = 0
+        with open(filename, 'w') as solnfile:
+            for var in instance.component_data_objects(Var):
+                # Cbc only expects integer variables with non-zero values for mipstart.
+                if var.value \
+                        and (var.is_integer() or var.is_binary()) \
+                        and (id(var) in byObject):
+                    name = byObject[id(var)]
+                    solnfile.write(
+                        '{} {} {}\n'.format(
+                            column_index, name, var.value
+                        )
+                    )
+                    # Cbc ignores column indexes, so the value does not matter.
+                    column_index += 1
+
+    #
+    # Write a warm-start file in the SOLN format.
+    #
+    def _warm_start(self, instance):
+
+        self._write_soln_file(instance, self._warm_start_file_name)
+
+    # over-ride presolve to extract the warm-start keyword, if specified.
+    def _presolve(self, *args, **kwds):
+
+        # create a context in the temporary file manager for
+        # this plugin - is "pop"ed in the _postsolve method.
+        pyutilib.services.TempfileManager.push()
+
+        # if the first argument is a string (representing a filename),
+        # then we don't have an instance => the solver is being applied
+        # to a file.
+        self._warm_start_solve = kwds.pop('warmstart', False)
+        self._warm_start_file_name = kwds.pop('warmstart_file', None)
+        user_warmstart = False
+        if self._warm_start_file_name is not None:
+            user_warmstart = True
+
+        # the input argument can currently be one of two things: an instance or a filename.
+        # if a filename is provided and a warm-start is indicated, we go ahead and
+        # create the temporary file - assuming that the user has already, via some external
+        # mechanism, invoked warm_start() with a instance to create the warm start file.
+        if self._warm_start_solve and \
+                isinstance(args[0], string_types):
+            # we assume the user knows what they are doing...
+            pass
+        elif self._warm_start_solve and \
+                (not isinstance(args[0], string_types)):
+            # assign the name of the warm start file *before* calling the base class
+            # presolve - the base class method ends up creating the command line,
+            # and the warm start file-name is (obviously) needed there.
+            if self._warm_start_file_name is None:
+                assert not user_warmstart
+                self._warm_start_file_name = pyutilib.services.TempfileManager.\
+                                             create_tempfile(suffix = '.cbc.soln')
+
+        # let the base class handle any remaining keywords/actions.
+        # let the base class handle any remaining keywords/actions.
+        super(CBCSHELL, self)._presolve(*args, **kwds)
+
+        # NB: we must let the base class presolve run first so that the
+        # symbol_map is actually constructed!
+
+        if (len(args) > 0) and (not isinstance(args[0], string_types)):
+
+            if len(args) != 1:
+                raise ValueError(
+                    "CBCplugin _presolve method can only handle a single "
+                    "problem instance - %s were supplied" % (len(args),))
+
+            # write the warm-start file - currently only supports MIPs.
+            # we only know how to deal with a single problem instance.
+            if self._warm_start_solve and (not user_warmstart):
+
+                start_time = time.time()
+                self._warm_start(args[0])
+                end_time = time.time()
+                if self._report_timing is True:
+                    print("Warm start write time=%.2f seconds" % (end_time-start_time))
+
 
     def _default_executable(self):
         executable = pyomo.common.Executable("cbc")
@@ -293,6 +398,8 @@ def _check_and_escape_options(options):
             cmd.extend(["-printingOptions", "all",
                         "-import", problem_files[0]])
             cmd.extend(action_options)
+            if self._warm_start_solve:
+                cmd.extend(["-mipstart",self._warm_start_file_name])
             cmd.extend(["-stat=1",
                         "-solve",
                         "-solu", self._soln_file])
@@ -757,6 +864,20 @@ def process_soln_file(self, results):
             results.solution.insert(solution)
 
 
+    def _postsolve(self):
+
+        # let the base class deal with returning results.
+        results = super(CBCSHELL, self)._postsolve()
+
+        # finally, clean any temporary files registered with the temp file
+        # manager, created populated *directly* by this plugin. does not
+        # include, for example, the execution script. but does include
+        # the warm-start file.
+        pyutilib.services.TempfileManager.pop(remove=not self._keepfiles)
+
+        return results
+
+
 @SolverFactory.register('_mock_cbc')
 class MockCBC(CBCSHELL,MockMIP):
     """A Mock CBC solver used for testing


Test patch:
diff --git a/pyomo/solvers/tests/checks/test_cbc.py b/pyomo/solvers/tests/checks/test_cbc.py
new file mode 100644
--- /dev/null
+++ b/pyomo/solvers/tests/checks/test_cbc.py
@@ -0,0 +1,51 @@
+#  ___________________________________________________________________________
+#
+#  Pyomo: Python Optimization Modeling Objects
+#  Copyright 2017 National Technology and Engineering Solutions of Sandia, LLC
+#  Under the terms of Contract DE-NA0003525 with National Technology and
+#  Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
+#  rights in this software.
+#  This software is distributed under the 3-clause BSD License.
+#  ___________________________________________________________________________
+
+
+from pyomo.environ import *
+import pyutilib.th as unittest
+from pyutilib.misc import capture_output
+
+opt_cbc = SolverFactory('cbc')
+cbc_available = opt_cbc.available(exception_flag=False)
+
+
+class CBCTests(unittest.TestCase):
+
+    @unittest.skipIf(not cbc_available,
+                     "The CBC solver is not available")
+    def test_warm_start(self):
+
+        m = ConcreteModel()
+        m.x = Var()
+        m.z = Var(domain=Integers)
+        m.w = Var(domain=Boolean)
+        m.c = Constraint(expr=m.x + m.z + m.w >= 0)
+        m.o = Objective(expr=m.x + m.z + m.w)
+
+        # Set some initial values for warm start.
+        m.x.set_value(10)
+        m.z.set_value(5)
+        m.w.set_value(1)
+
+        with SolverFactory("cbc") as opt:
+            with capture_output() as output:
+                opt.solve(m, tee=True, warmstart=True)
+
+                # Check if CBC loaded the warmstart file.
+                self.assertIn('opening mipstart file', output.getvalue())
+                # Only integer and binary variables are considered by CBC.
+                self.assertIn('MIPStart values read for 2 variables.', output.getvalue())
+                # m.x is ignored because it is continuous, so cost should be 5+1
+                self.assertIn('MIPStart provided solution with cost 6', output.getvalue())
+
+
+if __name__ == "__main__":
+    unittest.main()
diff --git a/pyomo/solvers/tests/testcases.py b/pyomo/solvers/tests/testcases.py
--- a/pyomo/solvers/tests/testcases.py
+++ b/pyomo/solvers/tests/testcases.py
@@ -78,7 +78,7 @@
 #
 
 ExpectedFailures['cbc', 'lp', 'LP_duals_maximize'] = \
-    (lambda v: v <= _trunk_version,
+    (lambda v: v <= (2,10,1,0),
     "For a maximization problem where a variable is pushed to its "
     "lower bound, Cbc reports the reduced cost as a positive number. In "
     "practice this should be reported as a negative number. A ticket has "