Yes, Upstox is a registered stockbroker with SEBI. Their SEBI Registration No. is INZ000185137 (as RKSV Securities India Pvt Ltd).
Upstox is also a member of NSE, BSE, and MCX stock exchanges and CDSL and NSDL central depositories.