Non-working version, crawls directory.

The way I synchronize around the whole thing is just wrong.

Maybe a channel created in the main() that would receive the messages
sent by the different workers.
#go-nuts curated version of the crawler