Fix segfaults on OSX

This reverts commit 30d0897c79f48ee883589abea0bbc319240c44cb.

This has been [causing segfaults](https://github.com/JuliaLang/julia/issues/7344) on OSX builds.  Reverting this reversion fixes the issue.