added crawler generation up to given thread count