ganglia/monitor-core — pull request #66 by stevedonnelly (2012-10-30 18:30:33+00:00)
Title: making gmetad's data_thread non-blocking

I've had issues with gmetad's data threads hanging, and subsequently reporting that the corresponding gmonds are down. After reviewing the code, it didn't look like this problem should be happening so I simply tried removing all possible infinite loops.

- Reading from the socket only reads as much data is currently available. The bytes available is determined using ioctl(FIONREAD), but still limited to 1023 bytes.

- The read call is no longer retried infinitely if continuous EINTR errors occur (since I removed the SYS_CALL wrapper). I'm not sure removing this was necessary, but the SYS_CALL loop wasn't necessary either since the data_thread will simply retry after sleeping. If an EINTR error occurs before any data is read, there is likely something wrong with the connection anyway, so retrying probably won't help.

After making these changes, I haven't had any problem with gmetad hanging, although I'm still not sure if the cause of the problem is gmetad, gmond, or my hardware/networking setup. In the worst case, these changes should simply make gmetad more resilient to poorly-behaving gmond connections.