allow spider to be specified in the yaml