Repository: dask/dask

Issue:
Profiler AssertionError: can only join a child process
I am finding when using the dask Profiler that I get errors like "AssertionError: can only join a child process" and this causes the IPython kernel to enter a state where it cannot be shut down from within Jupyter and needs to be killed manually. Example here: https://gist.github.com/alimanfoo/68f15f57c41db3df1652b54ff9c26a84



Discussion:
Not sure what's going on with this one.  CCing @jcrist, he might not be around much until tomorrow though.

I won't have time to dig into this much until tomorrow. The issue is with the `ResourceProfiler`, which forks off a process for measuring memory and cpu stats. Two quick thoughts:
- Move the `visualize` call out of the context manager. Ideally that should only be called after the `__exit__` code has run for each profiler. It might work fine the way you have it, it might not. I don't think that's what's causing the issue here, but it's not the best way to use `visualize`.
- `%timeit`/`%time` might be doing odd things with the profiler here. Do you still get the errors if you don't wrap the call in one of these?

I get the errors whether visualize() is called in or out of the context manager. Also the errors occur still when running outside of %time.

I ran your notebook as is, and am unable to reproduce. A few questions:
1. Does it always fail for you, or just sometimes?
2. What operating system are you using? What version of python?

Thanks for looking into this, it fails always but usually after the second
time a profiler is run not the first. I'm on Ubuntu 15.10 and ran the
notebook with Python 3.4. I'll try to replicate on another computer and
pare down to a minimal example.

On Wed, Apr 13, 2016 at 6:40 PM, Jim Crist notifications@github.com wrote:

> I ran your notebook as is, and am unable to reproduce. A few questions:
> 1. Does it always fail for you, or just sometimes?
> 2. What operating system are you using? What version of python?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly or view it on GitHub
> https://github.com/dask/dask/issues/1094#issuecomment-209559474

## 

Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Email: alimanfoo@googlemail.com alimanfoo@gmail.com
Web: http://purl.org/net/aliman
Twitter: https://twitter.com/alimanfoo
Tel: +44 (0)1865 287721


Resolving patch:
diff --git a/dask/diagnostics/profile.py b/dask/diagnostics/profile.py
--- a/dask/diagnostics/profile.py
+++ b/dask/diagnostics/profile.py
@@ -139,6 +139,15 @@ def __init__(self, dt=1):
         self._tracker.start()
         self.results = []
 
+    def __enter__(self):
+        self.clear()
+        return super(ResourceProfiler, self).__enter__()
+
+    def __exit__(self, *args):
+        if self._tracker.is_alive():
+            self._tracker.parent_conn.send('stop')
+        super(ResourceProfiler, self).__exit__(*args)
+
     def _start(self, dsk):
         assert self._tracker.is_alive(), "Resource tracker is shutdown"
         self._tracker.parent_conn.send('start')
@@ -151,6 +160,8 @@ def close(self):
         """Shutdown the resource tracker process"""
         self._tracker.shutdown()
 
+    __del__ = close
+
     def clear(self):
         self.results = []
 
@@ -185,12 +196,11 @@ def shutdown(self):
             self.parent_conn.close()
         self.join()
 
-    __del__ = shutdown
-
     def _collect(self):
         data = []
         pid = current_process()
-        ps = [self.parent] + [p for p in self.parent.children() if p.pid != pid]
+        ps = [self.parent] + [p for p in self.parent.children()
+                              if p.pid != pid and p.status() != 'zombie']
         while not self.child_conn.poll():
             tic = default_timer()
             mem = sum(p.memory_info().rss for p in ps)/1e6


Test patch: