usb: u_bam: Fix spinlock leak followed by lockup from workqueue