It’s better. If you both are not present while issuing your marriage license, an additional paper is needed to be prepared for the absentee. It will take more time and cost even. So, it is better to be present both in that moment.