logstash/logstash — pull request #783 by bfritz (2013-11-19 18:03:07+00:00)
Title: [imap] ignore extra items from imap.fetch()

`imap.fetch(id_set, "RFC822")` may return multiple items per
message where only one has the full email under the "RFC822"
map key.

Ignoring items without the "RFC822" is a simple workaround.

I am seeing an item with "RFC822" and a second item with "FLAGS".
When Mail.read_from_string() runs on the FLAGS item, an empty
message without a date is parsed and `mail.date.to_time.gmtime`
fails on the second message:

A plugin had an unrecoverable error. Will restart this plugin.
Plugin: "imap", host=>"mail.example.com", user=>"logstash@example.com">
Error: undefined method `to_time' for nil:NilClass
Exception: NoMethodError
Stack: lib/logstash/inputs/imap.rb:89:in `parse_mail'
lib/logstash/inputs/imap.rb:64:in `block (2 levels) in check_mail'
lib/logstash/inputs/imap.rb:62:in `each'
lib/logstash/inputs/imap.rb:62:in `block in check_mail'
lib/logstash/inputs/imap.rb:60:in `each_slice'
lib/logstash/inputs/imap.rb:60:in `check_mail'
lib/logstash/inputs/imap.rb:49:in `block in run'
vendor/bundle/ruby/1.9/gems/stud-0.0.17/lib/stud/interval.rb:11:in `call'
vendor/bundle/ruby/1.9/gems/stud-0.0.17/lib/stud/interval.rb:11:in `interval'
lib/logstash/inputs/imap.rb:48:in `run'
lib/logstash/pipeline.rb:151:in `inputworker'
lib/logstash/pipeline.rb:145:in `block in start_input' {:level=>:error, :file=>"logstash/pipeline.rb", :line=>"156"}

That's during testing against old Courier IMAP versions on a
Debian box:

$ dpkg -l courier-base courier-imap-ssl
||/ Name Version Description
+++-==============-==============-============================================
ii courier-base 0.65.0-3 Courier mail server - base system
ii courier-imap-s 4.8.0-3 Courier mail server - IMAP over SSL