avoid bottleneck in getAllRecipients

After some profiling I found that 'getAllRecipients' uses some message
fields that are not in the ENVELOPE - this causes every message to be
re-read at the time it is filtered. Avoid this by just looking at TO
address.