A few changes to finalize the new release system for AlwaysData

- Added robots.txt to the files that need to be linked from public.
- Updated django.fcgi, htaccess.txt and robots.txt with the latest content.
- Raise a 404 when a user accesses an unknown URL.