Yes, it would be best if you did that. Even if your baby is asleep, you should defiantly change the diaper to avoid any urinary tract infection. Most probably, your baby will be crying.