Language support

#2
by crosscultural - opened

Hi,

I'm new to hugging face. Is there a way to add a new language to the model such as korean?

Thanks

Hi,

For this fine-tuned sentiment model, the underlying architecture is distilbert-base-multilingual-cased, which supports 104 languages, including Korean. In other words, this model has the capability to process Korean text to some extent. Therefore, I suggest you first evaluate this model on your Korean sentiment dataset to gauge its performance.

If you wish to further improve the model's performance on the Korean language, you can fine-tune it using Korean-specific sentiment analysis datasets. For a practical example, please refer to the notebook link attached to my model card.

Quick test on korean:

μ €λŠ”μ΄ μ˜ν™”κ°€ μ’‹μ•„μš”
>>> [
  [
    {
      "label": "positive",
      "score": 0.9288509488105774
    },
    {
      "label": "neutral",
      "score": 0.04762842506170273
    },
    {
      "label": "negative",
      "score": 0.02352064475417137
    }
  ]
]


λ‚˜λŠ” 이 μ˜ν™”λ₯Ό μ‹«μ–΄ν•œλ‹€.
>>> [
  [
    {
      "label": "negative",
      "score": 0.5530788898468018
    },
    {
      "label": "positive",
      "score": 0.2614102363586426
    },
    {
      "label": "neutral",
      "score": 0.18551082909107208
    }
  ]
]
lxyuan changed discussion status to closed

Sign up or log in to comment