Fixed part of the search indexing command (dpsearch index). Added a robots.txt to the site (JIRA 386). Fixed some bugs in the sitemaps command.
Merge branch 'master' of github.com:kcl-ddh/digipal