logstash/logstash — pull request #251 by theduke (2012-11-27 23:50:01+00:00)
Title: New input plugin: Drupal DBLog

I have written a new input plugin: drupal_dblog

Drupal ships with the dblog plugin, which allows to store log messages in the database.
This plugin pulls all new log messages directly out of the MySQL database and properly encodes them in the json_event format.

Rationale:

There are other ways of getting your log messages to logstash, for example with the Drupal gelf module. I have written this plugin because it frees you from having to buffer the log messages within AMQP or something similar in case Logstash is not available and prevents Logstash from being overloaded, since it is pull based rather than push.

I think, considering the high popularity of Drupal, that this could be useful to others as well.

I did need 3 additional gems, which I have added to logstash.gemspec:

-) php-serialize for decoding some data in the database
-) mysql2 for MySQL support in CRuby
-) jdbc-mysql for MySQL support in JRuby

Multiple Drupal sites are supported.
All sites are pulled each INTERVAL minutes.
The interval can be specified in the config.

Config:

databases (required): a hash with the format: "sitename" => "db-uri"
add_usernames: whether the actual username of the current user should be added as a field
interval: the interval to pull for new entries, in MINUTES

input {
drupal_dblog {
databases => [
"NAME_OF_SITE", "mysql://user:password@localhost/drupal7_database",
"OTHER_SITE", "mysql://user:password@localhost/drupal6_database"
]
add_usernames => true
interval => 10
}
}

I have tested the plugin on both CRuby and JRuby, all seems to work fine.
The plugin also works with the built JAR.

------------------------------
I am quite the noob with Ruby, so comments regarding the code are very welcome.