You should wait at least 10 to 15 seconds before setting the hook when using worms as bait for catfish. This will give the catfish enough time to swallow the bait.