FuseAI

/

FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview

like 29

Follow

FuseAI 61

Hi. Out of curiosity, was there any attempts to merge the Qwen 2.5 32b coder model? If not, is this something you are willing to try or do?

Edit: I tried this myself by using their merge technique as they describe it. RDson/CoderO1-DeepSeekR1-Coder-32B-Preview and RDson/CoderO1-DeepSeekR1-Coder-14B-Preview

Have you tested or benchmarked your merges? Was thinking of giving the 14b a try.

We find the evaluation results for math and code are not correct in our current version. To address this issue, we use the code from Qwen2.5-Math and Qwen2.5-Coder for math and code evaluation. With this approach, we have successfully reproduced the results reported in the DeepSeek-R1 paper. We will update all the results—including those for this coding model—tomorrow. Please stay tuned. Thank you for use FuseO1-Preview.

Here is our merged coding model: https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Coder-32B-Preview

Hi. Out of curiosity, was there any attempts to merge the Qwen 2.5 32b coder model? If not, is this something you are willing to try or do?

Edit: I tried this myself by using their merge technique as they describe it. RDson/CoderO1-DeepSeekR1-Coder-32B-Preview and RDson/CoderO1-DeepSeekR1-Coder-14B-Preview

Have you tested or benchmarked your merges? Was thinking of giving the 14b a try.

I have not had the time to run any benchmarks due to limited resources. You would have to try it out yourself.

We find the evaluation results for math and code are not correct in our current version. To address this issue, we use the code from Qwen2.5-Math and Qwen2.5-Coder for math and code evaluation. With this approach, we have successfully reproduced the results reported in the DeepSeek-R1 paper. We will update all the results—including those for this coding model—tomorrow. Please stay tuned. Thank you for use FuseO1-Preview.

Thank you for your time making this coder model too. I hope it turned out well!

@Wanfq

With this approach, we have successfully reproduced the results reported in the DeepSeek-R1 paper.

Could you share your approach? Thanks

Please share the details as well, like temperature, top p&k, repeat penalty