Yes you do.  You, your fiancé and a witness must make an appointment to come in.
https://saddlebrooknj.us/health-department-and-registrar/home/