Yes, but they may take longer to cook. Baby carrots vary considerably in size. If your baby carrots run large, you may need to sauté them longer than 5 minutes before you add the glaze. Once you add the glaze, cook them until they're easy to pierce with a fork.