No, unfortunately you can’t add music to carousels with videos. Instagram currently only supports adding music to carousels. When creating a carousel post, click on the sticker icon, select Music, and pick a song to accompany your video carousel.