Don't iterate over files while looking for Maildirs

Should be marginaly faster