After my sim became best friends with a mermaid NPC, she admitted she was a mermaid (full friendship bar). You must have a very close relationship with them in order for them to acknowledge they are mermaids. There were no other options available.