updated to use full db for queries and preprocessing