You can ask @jarv for more details about why it doesn't work behind an ELB, but—essentially, HTTP_X_FORWARDED_FOR wasn't getting set in non-ELB setups, and the IP was showing up as localhost rather than the actual IP.  My understanding is this is related to how nginx does proxying...