Accuracy for "mention as is" does not match the paper results

#7
by KomalZorini - opened

Thank you for the excellent work. Could you clarify how you measured accuracy for "mention as is"? I employed the evaluation script from SAP and applied this model to assess accuracy by encoding mentions through KRISSbert. However, the accuracy I obtained is much lower than what was reported in the paper. SAPbert seems to show better accuracy across nearly all datasets, including MedMention. Perhaps I am not implementing it correctly. Could you please provide some additional details? Thank you in advance.

Sign up or log in to comment