routine to rebuild all database, should be called on any upgrade
hidden fields in effect are not indexed (and so not shown to user)