On the Slack app, go to You. Here, you can change what, when, and how you want to be notified. Change the notification settings from the options available; that is, On mobile, notify me about..., Notify me on mobile..., and System options to customize your notification preferences. Make sure you have turned App notifications on for Slack in your phone's Settings app.