Repository: Pylons/pyramid

Issue:
Development mode server kills itself on syntax errors
Hi there,

I've noticed an annoying feature of the development mode of pyramid, that is, it's server kills itself if it encounters a syntax error.

While that is certainly safe, what I think it should really do is just try again after the next file change (that usually fixes the syntax error....).



Discussion:
A pull request would be awesome.

Matt Good just made a solution for this for Flask: https://github.com/mgood/flask-failsafe

Daniel Holth also created something a number of moons ago that we might dredge up.  I don't have a link to it at the moment.

Anybody have a better idea than the one described by the commit above?

Well, I'm testing this right now.
1) Too much description text. 
2) I would rather press ^C and restart manually than change my `__init__` file.
3) It doesn't restart when I'm fixing syntaxerror in my views.py file of `starter` project. But restarts when I'm changing pserve.py! :))
4) When it 'freezes' on files check we don't handle ^C = showing useless traceback.

K, welp, I have no better ideas.

(the above things you described are all expected).

https://github.com/Pylons/pyramid/pull/698 - I'm sorry I don't know how to attach commit to this issue :)

The existing behavior, stopping the server on syntax errors, seems best to me.  Sometimes after the error that stops the server, your next editing activity in your code returns it to a state that is runnable (sans syntax problems) but may not yet be ready or safe in some other way.  Some of us might not want it actually running again until after a manual restart.  Auto-restart after error could be made an option, but how about not default?

Maybe you are better of if you disable auto reloading at in your situation.

Because I would highly doubt that you can even guarantee a high probability that your code gets into situations where you don't want to run it only after you made a change that doesn't let the code compile  / load anymore.

That being said, maybe the real solution for you may be to create a separate development environment where software failure does not have harsh consequences.


Resolving patch:
diff --git a/pyramid/scripts/pserve.py b/pyramid/scripts/pserve.py
--- a/pyramid/scripts/pserve.py
+++ b/pyramid/scripts/pserve.py
@@ -14,9 +14,11 @@
 import logging
 import optparse
 import os
+import py_compile
 import re
 import subprocess
 import sys
+import tempfile
 import textwrap
 import threading
 import time
@@ -820,7 +822,7 @@ class Monitor(object): # pragma: no cover
         if %errorlevel% == 3 goto repeat
 
     or run a monitoring process in Python (``pserve --reload`` does
-    this).  
+    this).
 
     Use the ``watch_file(filename)`` function to cause a reload/restart for
     other non-Python files (e.g., configuration files).  If you have
@@ -843,9 +845,19 @@ def __init__(self, poll_interval):
         self.poll_interval = poll_interval
         self.extra_files = list(self.global_extra_files)
         self.instances.append(self)
+        self.syntax_error_files = set()
+        self.pending_reload = False
         self.file_callbacks = list(self.global_file_callbacks)
+        temp_pyc_fp = tempfile.NamedTemporaryFile(delete=False)
+        self.temp_pyc = temp_pyc_fp.name
+        temp_pyc_fp.close()
 
     def _exit(self):
+        try:
+            os.unlink(self.temp_pyc)
+        except IOError:
+            # not worried if the tempfile can't be removed
+            pass
         # use os._exit() here and not sys.exit() since within a
         # thread sys.exit() just closes the given thread and
         # won't kill the process; note os._exit does not call
@@ -876,6 +888,7 @@ def check_reload(self):
                 continue
             if filename is not None:
                 filenames.append(filename)
+        new_changes = False
         for filename in filenames:
             try:
                 stat = os.stat(filename)
@@ -887,11 +900,42 @@ def check_reload(self):
                 continue
             if filename.endswith('.pyc') and os.path.exists(filename[:-1]):
                 mtime = max(os.stat(filename[:-1]).st_mtime, mtime)
-            if filename not in self.module_mtimes:
-                self.module_mtimes[filename] = mtime
-            elif self.module_mtimes[filename] < mtime:
-                print("%s changed; reloading..." % filename)
-                return False
+                pyc = True
+            else:
+                pyc = False
+            old_mtime = self.module_mtimes.get(filename)
+            self.module_mtimes[filename] = mtime
+            if old_mtime is not None and old_mtime < mtime:
+                new_changes = True
+                if pyc:
+                    filename = filename[:-1]
+                is_valid = True
+                if filename.endswith('.py'):
+                    is_valid = self.check_syntax(filename)
+                if is_valid:
+                    print("%s changed ..." % filename)
+        if new_changes:
+            self.pending_reload = True
+            if self.syntax_error_files:
+                for filename in sorted(self.syntax_error_files):
+                    print("%s has a SyntaxError; NOT reloading." % filename)
+        if self.pending_reload and not self.syntax_error_files:
+            self.pending_reload = False
+            return False
+        return True
+
+    def check_syntax(self, filename):
+        # check if a file has syntax errors.
+        # If so, track it until it's fixed.
+        try:
+            py_compile.compile(filename, cfile=self.temp_pyc, doraise=True)
+        except py_compile.PyCompileError as ex:
+            print(ex.msg)
+            self.syntax_error_files.add(filename)
+            return False
+        else:
+            if filename in self.syntax_error_files:
+                self.syntax_error_files.remove(filename)
         return True
 
     def watch_file(self, cls, filename):


Test patch: