irq: Set CPU affinity as soon as the irq thread is created