May be suboptimal to store both:
t-v4/tokenizer.json|tr -d \\n|ug -o '\[[^\]]*\]'|sed 's/", "//g'|cut -d\" -f2|rg anon ĠCanon Ġcanon ĠCanons Ġcanons
same with plural forms. Statistics insist on the distinction, but how to store it beter?
· Sign up or log in to comment
