Add DFID to robots exclusions
Handle IDF type documents
Merge pull request #45 from alphagov/feature/handle-dfid

Handle International Development Fund Documents