added sidekiq gem to test group
fixed some bugs around duplicate queued urls
updated worker spec