Repository: geopython/pywps
Issue: workdir is removed before process has started

# Description

The process `workdir` is removed *before* the process has started. 

The suspicious line of code is here:
https://github.com/geopython/pywps/blob/fbd5b3a39f2f7b6da735b94bd57caad12c267a8c/pywps/response/execute.py#L209

When I disable it then it works.

@gschwind @jachym Could you please check the reasoning for this code line? Otherwise I would just comment it with a PR.

If your process has `ComplexInputs` then the `workdir` is created again and you might not notice this bug.

# Steps to Reproduce

Try a process with no `ComplexInputs` and which creates a file in `self.workdir`:
https://github.com/cehbrecht/ecas-wps-demo/blob/759a83d390ef28641825eaa1aafde791a4c4a815/ecaswps/processes/wps_index_tn.py#L77

# Additional Information

The mentioned line of code was introduced in PR #358.

Changed files:
pywps/response/execute.py

diff --git a/pywps/response/execute.py b/pywps/response/execute.py
index 9f30d52..c836171 100755
--- a/pywps/response/execute.py
+++ b/pywps/response/execute.py
@@ -50,35 +50,54 @@ class ExecuteResponse(WPSResponse):
 
     # override WPSResponse._update_status
     def _update_status(self, status, message, status_percentage, clean=True):
+        """
+        Updates status report of currently running process instance:
+
+        * Updates the status document.
+        * Updates the status file (if requested).
+        * Cleans the working directory when process has finished.
+
+        This method is *only* called by pywps internally.
+        """
         super(ExecuteResponse, self)._update_status(status, message, status_percentage)
+        LOGGER.debug("_update_status: status={}, clean={}".format(status, clean))
+        self._update_status_doc()
         if self.store_status_file:
-            self.update_status_file(clean)
+            self._update_status_file()
+        if clean:
+            if self.status == WPS_STATUS.SUCCEEDED or self.status == WPS_STATUS.FAILED:
+                LOGGER.debug("clean workdir: status={}".format(status))
+                self.process.clean()
 
     def update_status(self, message, status_percentage=None):
         """
-        Update status report of currently running process instance
+        Update status report of currently running process instance.
+
+        This method is *only* called by the user provided process.
+        The status is handled internally in pywps.
 
         :param str message: Message you need to share with the client
         :param int status_percentage: Percent done (number betwen <0-100>)
         """
         if status_percentage is None:
             status_percentage = self.status_percentage
-        self._update_status(self.status, message, status_percentage)
+        self._update_status(self.status, message, status_percentage, False)
 
-    def update_status_file(self, clean):
-        # TODO: check if file/directory is still present, maybe deleted in mean time
+    def _update_status_doc(self):
         try:
-            # rebuild the doc and update the status xml file
+            # rebuild the doc
             self.doc = self._construct_doc()
+        except Exception as e:
+            raise NoApplicableCode('Building Response Document failed with : %s' % e)
 
+    def _update_status_file(self):
+        # TODO: check if file/directory is still present, maybe deleted in mean time
+        try:
+            # update the status xml file
             with open(self.process.status_location, 'w') as f:
                 f.write(self.doc)
                 f.flush()
                 os.fsync(f.fileno())
-
-            if (self.status == WPS_STATUS.SUCCEEDED or self.status == WPS_STATUS.FAILED) and clean:
-                self.process.clean()
-
         except Exception as e:
             raise NoApplicableCode('Writing Response Document failed with : %s' % e)
 
@@ -195,17 +214,6 @@ class ExecuteResponse(WPSResponse):
                 return Response(wps_output_value.data,
                                 mimetype=self.wps_request.outputs[wps_output_identifier]['mimetype'])
         else:
-            doc = None
-            try:
-                doc = self._construct_doc()
-                if self.store_status_file:
-                    # TODO: disabled this clean as workaround for #370
-                    # self.process.clean()
-                    pass
-            # TODO: If an exception occur here we must generate a valid status file
-            except HTTPException as httpexp:
-                return httpexp
-            except Exception as exp:
-                return NoApplicableCode(exp)
-
-            return Response(doc, mimetype='text/xml')
+            if not self.doc:
+                return NoApplicableCode("Output was not generated")
+            return Response(self.doc, mimetype='text/xml')