daithi-coombes/logsearch-development-flow — pull request #1 by daithi-coombes (2013-12-01 19:13:15+00:00)
Title: flowdock shipper pu

Seems there are syntax errors in
[srv/logstash/config/common.conf.erb](https://github.com/cityindex/logsearch/blob/master/srv/logstash/config/common.conf.erb#L6) and [srv/logstash/config/lumberjack_to_redis.conf.erb](https://github.com/cityindex/logsearch/blob/master/srv/logstash/config/redis_to_elasticsearch.conf.erb#L1)

I'm getting an error in `/var/log/app/logstash_redist-1.log`, the config file [redis_to_eslasticsearch.conf.efb](https://github.com/cityindex/logsearch/blob/master/srv/logstash/config/redis_to_elasticsearch.conf.erb#L1) requires `common.conf`, leading to [srv/logstash/config/common.conf.erb](https://github.com/cityindex/logsearch/blob/master/srv/logstash/config/common.conf.erb#L6) where the issue lies:
```
$ tail -f /var/log/app/logstash_redist-1.log
/usr/bin/java -Djava.io.tmpdir='/app/tmp' -jar '/app/vendor/logstash.jar' agent -w 1 -f '/app/app/srv/logstash/config/redis_to_elasticsearch.conf'
Error at line 6, column 8: "[type] == 'redis' or [type] == 'redis-input' {\n mutate {\n remove_field => [ 'type' ]\n }\n }\
n\n if [type] != '' {\n mutate {\n rename => [ \"type\", \"@type\" ]\n }\n }\n\n if [message] != '' {\n mutate
{\n rename => [ \"message\", \"@message\" ]\n }\n }\n\n #\n # ignore particularly useless lines\n #\n\n if [@message]
...
host => \"0.0.0.0\"\n flush_size => 100\n index_type => \"%{@type}\"\n }\n}\n\n"
Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (RuntimeError) Invalid Configuration. Check syntax of config file.
```

`/var/log/app/lumberjack_redis-1.log` is also reporting:
```
$ tail -f /var/log/app/lumberjack_redis-1.log
TMP=/app/tmp /usr/bin/java -jar '/app/vendor/logstash.jar' agent -w 1 -f '/app/app/srv/logstash/config/lumberjack_to_redis.conf'
Error at line 12, column 8: "[_logstash_input] == \"lumberjack\" {\n mutate {\n add_field => [ \"@source.host\", \"%{host}\" ]\n
add_field => [ \"@source.path\", \"%{file}\" ]\n add_field => [ \"@source.offset\", \"%{offset}\" ]\n remove_field => [ \"
host\", \"file\", \"offset\", \"_logstash_input\" ]\n }\n }\n}\n \n\noutput {\n # uncomment for debugging\n #stdout { debug => t
rue debug_format => \"json\"}\n\n redis {\n host => \"0.0.0.0\"\n data_type => \"list\"\n key => \"logstash\"\n batch =>
true\n batch_events => 50\n }\n}\n\n"
Error at line 12, column 8: "[_logstash_input] == \"lumberjack\" {\n mutate {\n add_field => [ \"@source.host\", \"%{host}\" ]\n
add_field => [ \"@source.path\", \"%{file}\" ]\n add_field => [ \"@source.offset\", \"%{offset}\" ]\n remove_field => [ \"
host\", \"file\", \"offset\", \"_logstash_input\" ]\n }\n }\n}\n \n\noutput {\n # uncomment for debugging\n #stdout { debug => t
rue debug_format => \"json\"}\n\n redis {\n host => \"0.0.0.0\"\n data_type => \"list\"\n key => \"logstash\"\n batch =>
true\n batch_events => 50\n }\n}\n\n"
Exception in thread "LogStash::Runner" org.jruby.exceptions.RaiseException: (RuntimeError) Invalid Configuration. Check syntax of config file.
```