fix errors in nms layer