Reestructured the code.

The code does the same as it did in the last commit, but now there is a well
defined interface that crawlers in general should implement. The idea is to
make possible to develop code that uses the information that is taken from the
crawlers without really having the crawlers yet.