changed two queues on the broker to be multiprocessing.JoinableQueues, made the broker only send jobs with more than 1 url to work on because I was having a lot of jobs thrown at my workers that actually had no tasks, FINALLY fixed the encoding crash; turns out lxml was sometimes returning byte strings and once in a while returned a unicode string.

Signed-off-by: mep <miethpo@gmail.com>