Only take the lock when the ref count hits zero
Merge pull request #12 from pcwalton/slow-drop

Only take the lock when the ref count hits zero