Normalization of outputs

#2
by skirres - opened

First of all, thanks for the nice work and sharing your model and results ๐Ÿค—

In the model card you mention the normalization of outputs, but in your repository I stumbled upon this line. I got two questions:

  1. Did you normalize the vectors for the retrieval tasks?
  2. And (also for the retrieval tasks) did you report the results of the dot or the l2 metric in the model card?

And just a tiny remark, if you always want to use normalization, you could consider specifying it in the configuration like for this model.

Beijing Academy of Artificial Intelligence org

Hi,

  1. The default similarity for the retrieval task is cosine. So whether normalizing the embedding doesn't influence the results. This command only impacts the clustering and classification task.
  2. No. We compute the cosine similarity to retrieve relevant passages.
    Thanks for your advice!๐Ÿค— We will consider to update the configuration.
Beijing Academy of Artificial Intelligence org

We have updated the configuration. Thanks for your solution again!

Sign up or log in to comment