We do care, its actually kind of bad. 

For a full replica set, use of IO#select basically limits the pool size to ~85 (give or take a few fds in use for ismaster/replica set monitoring). Anything more would result in a buffer overflow.

However, this is actually kind of an issue with Ruby in general and there's not currently a reasonable way to solve this. Ruby doesn't implement IO#poll so we'd have to implement it ourselves if we want to fix this, and as a result we'd also have to fork anything we're relying on that also uses IO#select.