infgrad

/

stella-dialogue-large-zh-v3-1792d

like 2

Hi，there is no plan to open-source the training code for the time being. Below is the code for calculating loss:”

# teacher_vectors is from infgrad/stella-large-zh-v3-1792d
cosine_loss = 1 - (student_vectors * teacher_vectors).sum(axis=1).mean()
cosine_loss = cosine_loss * 10.0
sim_value_loss = F.mse_loss(
        input=torch.matmul(student_vectors, student_vectors.T),
         target=torch.matmul(teacher_vectors, teacher_vectors.T),
) * 100.0
final_loss  = cosine_loss +  sim_value_loss 

The whole process is very simple, good luck

thank u，it helps a lot~