function to generate urls

I have added the ability to use a function in place of the urls option.

This allows the list of urls that are process to be dynamic.