mirror/calibre — pull request #11 by john-peterson (2013-02-07 06:23:05+00:00)
Title: Printing `calibre-parallel` output connection

### Changing `calibre-parallel` output connection

The goal is to print stdout and stderr from calibre-parallel because

* it helps locating errors (compared to not seeing stdout and stderr)

The IPC is changed from a log file to pipe because the subprocess output doesn't reach the file in these tests (in Ubuntu and Windows).

utils/ipc/worker.py
def main():
print 'test_stdout'
test_stderr

utils/ipc/launch.py
def close_log_file(self):
print 'close_log_file: ', self._file.name
statinfo = os.stat(self._file.name)
print 'st_size: ', statinfo.st_size

close_log_file: /tmp/calibre_0.9.17_tmp_aubqLb/MOg4Qh_worker_redirect.log
st_size: 0

close_log_file: C:\Documents\Programs\cygwin\tmp\calibre_0.9.17_tmp_u3r0s4\1sdfrk_worker_redirect.log
st_size: 0

The output from other subprocess variations is

# args['stdout'] = subprocess.STDOUT
args['stderr'] = self._file._fd

args['stdout'] = self._file._fd
args['stderr'] = self._file._fd

OSError: [Errno 2] No such file or directory: '/tmp/calibre_0.9.17_tmp_KsLrq0/GMt72S_worker_redirect.log'

### Printing exception from metadata thread

It's beneficial to print the exception from a thread because

* it helps locating the error because the file and line is given

### Removing IPC file

If communication is done through a file the removal should wait for the file handle to be closed because this call

utils/ipc/simple_worker.py
def fork_job
finally:
os.remove(w.log_path)

doesn't wait for

utils/ipc/launch.py
def close_log_file(self):
self._file.close()

in Windows so that the folder is not removed

WindowsError: [Error 32] The process cannot access the file because it is being used by another process: u'C:\\Documents\\Programs\\cygwin\\tmp\\calibre_0.9.17_tmp_fi77r4\\tkzkeh_worker_redirect.log'

ls /tmp|grep calibre|wc -l
862
ls -t /tmp|grep calibre|tail -1
drwxr-xr-x 1 User None 0 2012-10-27 22:42 calibre_0.9.4_tmp_0i3o3j
ls -t /tmp|grep calibre|head -1
drwxr-xr-x 1 User None 0 2013-02-07 01:18 calibre_0.9.17_tmp_fi77r4