Don't join with non-threads.  Fixes 323