The best way to set up an on-call team is to outsource it to a professional Support as a Service provider like SupportYourApp.