add a function to compute the score
some refactoring
update the crawler