Be selective about how file descriptors are removed since they may be reused for a different purpose.

Kombu was removing them after they were being reused by another worker process.