The 'core' user's soft fd limit is 1024, so if we set the max to 512 for both servers, we'll exhaust that limit with just established TCP connections. Keep in mind that limit of 1024 also covers connections that are not established, i.e. being created or shut down. I'd prefer to aim for a much lower limit than 512 to force us to fix the core issues in the raft state machine - maybe 256? 

We do have two other options:
1) modify the default fd soft limit in coreos
2) dynamically raise our limit of 1024 to the hard limit of 4096 in-process

I feel like these two alternatives would be painting over the issue, though.

What do you think?