Repository: Pyomo/pyomo

Issue:
Blank space in project path leads to wrong file type in CPLEX
When path to python file is '/Users/XXX/Desktop/…/Pyomo Spam/eggs.py' 
(blank space after …/Pyomo)

CPLEX will throw an error containing:

`Logfile '/Users/XXX/Desktop/…/Pyomo' open.
CPLEX> Cannot interpret file type.`

when path is  '/Users/XXX/Desktop/…/Pyomo_Spam/eggs.py'  it works.



But when path to python file is '/Users/XXX/Desktop/…/Pyomo Spam/e g g s.py' so when the file itself contains the spacing, no problems occur. 



Discussion:
@phaabe This looks like a bug on the CPLEX side, not parsing a space in the file name given to it by Pyomo properly. Are you sure this should be a Pyomo bug report?

Separately, I tend to avoid using spaces in programming contexts: https://superuser.com/questions/29111/what-technical-reasons-exist-for-not-using-space-characters-in-file-names
I agree, it’s more likely to be a bug in CPLEX. I also agree, that white spaces should be avoided anyway. 

Maybe it would be useful, pyomo triggers a warning. It took me a while to figure that out since blank spaces in the file name didn’t cause the type error. Just the path is problematic. 

What do you think? 
What was the original error message? We want to consider the balance between providing helpful error messages to the user and adding to the maintenance burden and complexity of this code base.
Is it possible we just need to add quotes around the file name inside the CPLEX run file that we create? Can someone try doing this manually to see if that fixes the problem?

> On May 15, 2018, at 12:59 PM, Qi Chen <notifications@github.com> wrote:
> 
> What was the original error message? We want to consider the balance between providing helpful error messages to the user and adding to the maintenance burden and complexity of this code base.
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub, or mute the thread.
> 

@qtothec thing is, there was no error message. [Have a look(PDF of my jupyter notebook)](https://drive.google.com/open?id=1SmbrXw9vMcVIf5lrOvHQnn3EWYReA2GM)

As you can see, all that what happens:
- solver doesn't print the solution section
- no new values in the model
I was able to reproduce this locally, and adding quotes around paths in the CPLEX script file seems to fix things.

Opening a PR now.

@phaabe: Do you mind testing on that branch?
Actually, this is in master now. Feel free to test there and reopen this issue if things are still broken.

Resolving patch:
diff --git a/pyomo/solvers/plugins/solvers/CPLEX.py b/pyomo/solvers/plugins/solvers/CPLEX.py
--- a/pyomo/solvers/plugins/solvers/CPLEX.py
+++ b/pyomo/solvers/plugins/solvers/CPLEX.py
@@ -2,8 +2,8 @@
 #
 #  Pyomo: Python Optimization Modeling Objects
 #  Copyright 2017 National Technology and Engineering Solutions of Sandia, LLC
-#  Under the terms of Contract DE-NA0003525 with National Technology and 
-#  Engineering Solutions of Sandia, LLC, the U.S. Government retains certain 
+#  Under the terms of Contract DE-NA0003525 with National Technology and
+#  Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
 #  rights in this software.
 #  This software is distributed under the 3-clause BSD License.
 #  ___________________________________________________________________________
@@ -36,6 +36,37 @@
 except:
     basestring = unicode = str
 
+def _validate_file_name(cplex, filename, description):
+    """Validate filenames against the set of allowable chaacters in CPLEX.
+
+    Returns the filename, possibly enclosed in double-quotes, or raises
+    a ValueError is unallowable characters are found.
+
+    """
+    if filename is None:
+        return filename
+    matches = _validate_file_name.illegal_characters.search(filename)
+    if matches:
+        raise ValueError(
+            "Unallowed character (%s) found in CPLEX %s file path/name.\n\t"
+            "For portability reasons, only [%s] are allowed."
+            % (matches.group(), description,
+               _validate_file_name.allowed_characters.replace("\\",'')))
+    # CPLEX only supports quoting spaces starting in v12.8.
+    if ' ' in filename:
+        if cplex.version()[:2] >= (12,8):
+            filename = '"'+filename+'"'
+        else:
+            raise ValueError(
+                "Space detected in CPLEX %s file path/name\n\t%s\nand "
+                "CPLEX older than version 12.8.  Please either upgrade "
+                "CPLEX or remove the space from the %s path."
+                % (description, filename, description))
+    return filename
+_validate_file_name.allowed_characters = r"a-zA-Z0-9 \.\-_\%s" % (os.path.sep,)
+_validate_file_name.illegal_characters = re.compile(
+    '[^%s]' % (_validate_file_name.allowed_characters,))
+
 class CPLEX(OptSolver):
     """The CPLEX LP/MIP solver
     """
@@ -182,10 +213,9 @@ def _presolve(self, *args, **kwds):
         # then we don't have an instance => the solver is being applied
         # to a file.
         self._warm_start_solve = kwds.pop('warmstart', False)
-        self._warm_start_file_name = kwds.pop('warmstart_file', None)
-        user_warmstart = False
-        if self._warm_start_file_name is not None:
-            user_warmstart = True
+        self._warm_start_file_name = _validate_file_name(
+            self, kwds.pop('warmstart_file', None), "warm start")
+        user_warmstart = self._warm_start_file_name is not None
 
         # the input argument can currently be one of two things: an instance or a filename.
         # if a filename is provided and a warm-start is indicated, we go ahead and
@@ -259,6 +289,7 @@ def create_command_line(self, executable, problem_files):
         if self._log_file is None:
             self._log_file = pyutilib.services.TempfileManager.\
                             create_tempfile(suffix = '.cplex.log')
+        self._log_file = _validate_file_name(self, self._log_file, "log")
 
         #
         # Define solution file
@@ -267,11 +298,12 @@ def create_command_line(self, executable, problem_files):
         if self._soln_file is None:
             self._soln_file = pyutilib.services.TempfileManager.\
                               create_tempfile(suffix = '.cplex.sol')
+        self._soln_file = _validate_file_name(self, self._soln_file, "solution")
 
         #
         # Write the CPLEX execution script
         #
-        script = 'set logfile "%s"\n' % (self._log_file,)
+        script = 'set logfile %s\n' % (self._log_file,)
         if self._timelimit is not None and self._timelimit > 0.0:
             script += 'set timelimit %s\n' % ( self._timelimit, )
 
@@ -288,19 +320,21 @@ def create_command_line(self, executable, problem_files):
             else:
                 opt = ' '.join(key.split('_'))+' '+str(self.options[key])
             script += 'set %s\n' % ( opt, )
-        script += 'read "%s"\n' % ( problem_files[0], )
+
+        _lp_file = _validate_file_name(self, problem_files[0], "LP")
+        script += 'read %s\n' % ( _lp_file, )
 
         # if we're dealing with an LP, the MST file will be empty.
         if self._warm_start_solve and \
            (self._warm_start_file_name is not None):
-            script += 'read "%s"\n' % (self._warm_start_file_name,)
+            script += 'read %s\n' % (self._warm_start_file_name,)
 
         if 'relax_integrality' in self.options:
             script += 'change problem lp\n'
 
         script += 'display problem stats\n'
         script += 'optimize\n'
-        script += 'write "%s"\n' % (self._soln_file,)
+        script += 'write %s\n' % (self._soln_file,)
         script += 'quit\n'
 
         # dump the script and warm-start file names for the


Test patch:
diff --git a/pyomo/solvers/tests/checks/test_cplex.py b/pyomo/solvers/tests/checks/test_cplex.py
new file mode 100644
--- /dev/null
+++ b/pyomo/solvers/tests/checks/test_cplex.py
@@ -0,0 +1,58 @@
+#  ___________________________________________________________________________
+#
+#  Pyomo: Python Optimization Modeling Objects
+#  Copyright 2017 National Technology and Engineering Solutions of Sandia, LLC
+#  Under the terms of Contract DE-NA0003525 with National Technology and
+#  Engineering Solutions of Sandia, LLC, the U.S. Government retains certain
+#  rights in this software.
+#  This software is distributed under the 3-clause BSD License.
+#  ___________________________________________________________________________
+
+import os
+import pyutilib.th as unittest
+
+from pyomo.solvers.plugins.solvers.CPLEX import _validate_file_name
+
+class _mock_cplex_128(object):
+    def version(self):
+        return (12,8,0)
+
+class _mock_cplex_126(object):
+    def version(self):
+        return (12,6,0)
+
+class CPLEX_utils(unittest.TestCase):
+    def test_validate_file_name(self):
+        _126 = _mock_cplex_126()
+        _128 = _mock_cplex_128()
+
+        # Check plain file
+        fname = 'foo.lp'
+        self.assertEqual(fname, _validate_file_name(_126, fname, 'xxx'))
+        self.assertEqual(fname, _validate_file_name(_128, fname, 'xxx'))
+
+        # Check spaces in the file
+        fname = 'foo bar.lp'
+        with self.assertRaisesRegexp(
+                ValueError, "Space detected in CPLEX xxx file"):
+            _validate_file_name(_126, fname, 'xxx')
+        self.assertEqual('"%s"' % (fname,),
+                         _validate_file_name(_128, fname, 'xxx'))
+
+        # check OK path separators
+        fname = 'foo%sbar.lp' % (os.path.sep,)
+        self.assertEqual(fname, _validate_file_name(_126, fname, 'xxx'))
+        self.assertEqual(fname, _validate_file_name(_128, fname, 'xxx'))
+
+        # check BAD path separators
+        bad_char = '/\\'.replace(os.path.sep,'')
+        fname = 'foo%sbar.lp' % (bad_char,)
+        msg = 'Unallowed character \(%s\) found in CPLEX xxx file' % (
+            repr(bad_char)[1:-1],)
+        with self.assertRaisesRegexp(ValueError, msg):
+            _validate_file_name(_126, fname, 'xxx')
+        with self.assertRaisesRegexp(ValueError, msg):
+            _validate_file_name(_128, fname, 'xxx')
+
+if __name__ == "__main__":
+    unittest.main()