moshen/logstash-http-input — pull request #1 by vdavidoff (2014-05-16 13:56:57+00:00)
Title: A few features

Hi,

Let me start by stating that this is my first attempt at doing anything interesting with ruby, let alone jruby, so I may have done some of this incorrectly.

I am using your input plugin in an environment that required a few changes, and I wanted to submit them back to you on the chance that you'd like to merge them, in one way or another, with upstream.

We have logstash clients using your input that open persistent HTTP connections. They use Keep-Alives and send multiple events at a time in the POST body, in our case separated by newlines, which the ruby each_line handles just fine.

So, we need to be able to properly respond to GET Keep-Alive requests, and parse multiple events from each POST blob. I added a new config option to enable the parsing of multiple events from a single POST blob, which defaults to false, to retain existing default behavior. Keep-Alive support is hard coded in.

Additionally, as a nice to have, I made it so that the client remoteAddr is stored in each event in the host field, similar to what the syslog input does.

I can think of ways to make these changes better - like making keepalive support an option, making a config option for the event separator when multievent is enabled, etc, but am stopping short of that to get your feedback.

One concern I have is that I'm seeing some multievent parsing find truncated events at the end of a POST blob. It's not yet clear to me if this is the log generator sending a truncated POST blob, or if the way httpRequest.getInputStream() and the Scanner work needs to be modified.

Anyway, I hope these changes are helpful and interesting to you. I'd be happy to work on these further if you like them but think they need work.

Thanks.
Andy