fixed problem with persistence

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18755 2913f559-6b04-0410-9a09-c530ee9f5186
added auto-update of EFO (checks every SUN)

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18763 2913f559-6b04-0410-9a09-c530ee9f5186
better handling of archive files to prevent unwanted open file handles to hang around

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18765 2913f559-6b04-0410-9a09-c530ee9f5186
should be visible in the log what version of EFO is being used

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18766 2913f559-6b04-0410-9a09-c530ee9f5186
will use this URL until I figure out what is going on with original one when accessing from London DC

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18767 2913f559-6b04-0410-9a09-c530ee9f5186
more logging verbosity for debugging

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18768 2913f559-6b04-0410-9a09-c530ee9f5186
- slightly enhanced email system
- added test of email system
- added EFO update report email

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18769 2913f559-6b04-0410-9a09-c530ee9f5186
trick to mitigate proxy timeout issue on long and slow queries

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18886 2913f559-6b04-0410-9a09-c530ee9f5186
fix for the previous commit :/

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@18887 2913f559-6b04-0410-9a09-c530ee9f5186
fix to ensure there are no problems parcing bogus html in description

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19081 2913f559-6b04-0410-9a09-c530ee9f5186
redeveloped file scan (now depending on filesystem, but runs times faster and provides great info on ownership and permissions

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19269 2913f559-6b04-0410-9a09-c530ee9f5186
will not break check if EFO version is in the format NN.NN.NN.NN etc

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19313 2913f559-6b04-0410-9a09-c530ee9f5186
small update to file scanner: stderr is kept alongsite just for audit and reporting

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19349 2913f559-6b04-0410-9a09-c530ee9f5186
small update to list files command

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19358 2913f559-6b04-0410-9a09-c530ee9f5186
fix NPE when EFO term is null

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19359 2913f559-6b04-0410-9a09-c530ee9f5186
adding robustness and better exception reporting here

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19360 2913f559-6b04-0410-9a09-c530ee9f5186
removed attributes we don't want to expose from API

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19361 2913f559-6b04-0410-9a09-c530ee9f5186
updated FTP files checker

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19366 2913f559-6b04-0410-9a09-c530ee9f5186
updated FTP files checker, 2

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19367 2913f559-6b04-0410-9a09-c530ee9f5186
updated FTP files checker, 3

git-svn-id: svn://bar.ebi.ac.uk/trunk/arrayexpress-interface@19369 2913f559-6b04-0410-9a09-c530ee9f5186