yueyulin/rwkv6_crossencoder

This is a rwkv6 based cross encoder for query/context retreival reranker. To use this new architect, please fork https://github.com/yynil/rwkv_lm_ext_runner Try the codes to compare with bgem3-reranker, https://github.com/yynil/rwkv_lm_ext_runner/blob/main/tests/test_sentence_embedding_cross_encoder.py

With cuda on, the RNN bi-directional network can even run faster than transformer architect with almost the same accuracy.

      for i in range(test_loop):
        if i == 1:
            rwkv_time = 0
            bgm3_time = 0
        texts = ['每天吃苹果有什么好处？',
                '宁神安眠：苹果中含有的磷和铁等元素，易被肠壁吸收，有补脑养血、宁神安眠作用。苹果的香气是治疗抑郁和压抑感的良药。研究发现，在诸多气味中，苹果的香气对人的心理影响最大，它具有明显的消除心理压抑感的作用。',
                '美白养颜、降低胆固醇：苹果中的胶质和微量元素铬能保持血糖的稳定，还能有效地降低胆固醇。苹果中的粗纤维可促进肠胃蠕功，并富含铁、锌等微量元素，可使皮肤细润有光泽，起到美容瘦身的作用。',
                '苹果生吃治便秘，熟吃治腹泻：苹果中含有丰富的鞣酸、果胶、膳食纤维等特殊物质，鞣酸是肠道收敛剂，它能减少肠道分泌而使大便内水分减少，从而止泻。而果胶则是个“两面派”，未经加热的生果胶有软化大便缓解便秘的作用，煮过的果胶却摇身一变，具有收敛、止泻的功效。膳食纤维又起到通便作用。',
                '保护心脏：苹果的纤维、果胶、抗氧化物等能降低体内坏胆固醇并提高好胆固醇含量，所以每天吃一两个苹果不容易得心脏病。']
        bgm3_time += test_bgm3(reranker,texts)
        rwkv_time += test_texts(args, model, device, texts, tokenizer,dtype)
        

In the future, we will continue to train models with multilingual support