By default OBS will record your videos to .mkv format. You can change this by going to Settings > Recording > Recording Format and changing it to mp4.