added admin menu/permissions and modified crawling process to use update instead of insert