logsearch/logsearch-boshrelease — pull request #85 by dpb587 (2014-11-04 16:38:13+00:00)
Title: Switch parsers to using a local elasticsearch client node

The logstash parsers currently have a single point of failure because they can only talk to one elasticsearch host at a time. The alternatives are to switch to logstash's embedded elasticsearch client or to run your own in order to utilize elasticsearch's built-in cluster discovery capabilities.

Here I've opted to using our own packaged version rather than the (generally outdated) embedded version logstash has because it allows us to run the same elasticsearch version throughout the cluster and avoid the occasional bugs in running differing versions. This also lets us continue to use the sonian plugin to log the bulk indexing requests logstash makes and graph those times. Since parsers should be very ephemeral, elasticsearch is configured as a client node and will not become master nor will it store any data.

I've included a boolean property `logstash_parser.use_local_elasticsearch` which allows disabling the functionality (in the case of co-locating everything on micro installs).