Yes, Xbox can listen to your party chat. However, please note that Microsoft does not guarantee the privacy of your communications.