deepseek-ai/DeepSeek-V3-Base
Updated
•
12.4k
•
1.23k
Mark Collier
sparkycollier
AI & ML interests
Open Source AI for fun and profit. Open Source Infrastructure Software for AI & ML.
Recent Activity
new activity
19 days ago
deepseek-ai/DeepSeek-V3-Base:License
liked
a model
19 days ago
deepseek-ai/DeepSeek-V3-Base
liked
a dataset
about 2 months ago
alpindale/two-million-bluesky-posts
Organizations
sparkycollier's activity
License
4
#2 opened 19 days ago
by
mrfakename
liked
a
model
19 days ago
liked
a
dataset
about 2 months ago
liked
a
model
about 2 months ago
upvoted
an
article
6 months ago
Article
Llama 3.1 - 405B, 70B & 8B with multilinguality and long context
•
226
liked
a
model
6 months ago
liked
a
model
8 months ago
liked
2
models
9 months ago
License
9
#3 opened 9 months ago
by
mrfakename
liked
2
models
9 months ago
reacted to
WizardLM's
post with 🤗
9 months ago
Post
39798
🔥🔥🔥 Introducing WizardLM-2!
📙Release Blog: https://wizardlm.github.io/WizardLM2
✅Model Weights: microsoft/wizardlm-661d403f71e6c8257dbd598a
🐦Twitter: https://twitter.com/WizardLM_AI/status/1779899325868589372
We introduce and opensource WizardLM-2, our next generation state-of-the-art large language models, which have improved performance on complex chat, multilingual, reasoning and agent. New family includes three cutting-edge models: WizardLM-2 8x22B, WizardLM-2 70B, and WizardLM-2 7B.
WizardLM-2 8x22B is our most advanced model, and the best opensource LLM in our internal evaluation on highly complex tasks. WizardLM-2 70B reaches top-tier reasoning capabilities and is the first choice in the same size. WizardLM-2 7B is the fastest and achieves comparable performance with existing 10x larger opensource leading models.
🤗 WizardLM 2 Capacities:
1. MT-Bench (Figure-1)
The WizardLM-2 8x22B even demonstrates highly competitive performance compared to the most advanced proprietary works such as GPT-4-Trubo and Glaude-3. Meanwhile, WizardLM-2 7B and WizardLM-2 70B are all the top-performing models among the other leading baselines at 7B to 70B model scales.
2. Human Preferences Evaluation (Figure 2)
Through this human preferences evaluation, WizardLM-2's capabilities are very close to the cutting-edge proprietary models such as GPT-4-1106-preview, and significantly ahead of all the other open source models.
🔍Method Overview:
As the natural world's human-generated data becomes increasingly exhausted through LLM training, we believe that: the data carefully created by AI and the model step-by-step supervised by AI will be the sole path towards more powerful AI.
In the past one year, we built a fully AI powered synthetic training system. (As shown in the Figure 3).
📙Release Blog: https://wizardlm.github.io/WizardLM2
✅Model Weights: microsoft/wizardlm-661d403f71e6c8257dbd598a
🐦Twitter: https://twitter.com/WizardLM_AI/status/1779899325868589372
We introduce and opensource WizardLM-2, our next generation state-of-the-art large language models, which have improved performance on complex chat, multilingual, reasoning and agent. New family includes three cutting-edge models: WizardLM-2 8x22B, WizardLM-2 70B, and WizardLM-2 7B.
WizardLM-2 8x22B is our most advanced model, and the best opensource LLM in our internal evaluation on highly complex tasks. WizardLM-2 70B reaches top-tier reasoning capabilities and is the first choice in the same size. WizardLM-2 7B is the fastest and achieves comparable performance with existing 10x larger opensource leading models.
🤗 WizardLM 2 Capacities:
1. MT-Bench (Figure-1)
The WizardLM-2 8x22B even demonstrates highly competitive performance compared to the most advanced proprietary works such as GPT-4-Trubo and Glaude-3. Meanwhile, WizardLM-2 7B and WizardLM-2 70B are all the top-performing models among the other leading baselines at 7B to 70B model scales.
2. Human Preferences Evaluation (Figure 2)
Through this human preferences evaluation, WizardLM-2's capabilities are very close to the cutting-edge proprietary models such as GPT-4-1106-preview, and significantly ahead of all the other open source models.
🔍Method Overview:
As the natural world's human-generated data becomes increasingly exhausted through LLM training, we believe that: the data carefully created by AI and the model step-by-step supervised by AI will be the sole path towards more powerful AI.
In the past one year, we built a fully AI powered synthetic training system. (As shown in the Figure 3).
- 35 replies
liked
a
model
9 months ago
liked
a
model
10 months ago
liked
2
models
12 months ago
reacted to
clem's
post with ❤️
12 months ago
Post
In 2024, we're expanding from open weights to open EVERYTHING (datasets, training scripts,...).
Excited to see this dataset release in French by @Pclanglais @carbonbasedLLM @anastasiastasenko :
PleIAs/French-PD-Newspapers
"To give you an idea of the size, the full French Wikipedia is about 2 billon words. This is 40 times larger."
Excited to see this dataset release in French by @Pclanglais @carbonbasedLLM @anastasiastasenko :
PleIAs/French-PD-Newspapers
"To give you an idea of the size, the full French Wikipedia is about 2 billon words. This is 40 times larger."
upvoted
a
paper
about 1 year ago
reacted to
dvilasuero's
post with 👍
about 1 year ago
Post
👋 Hi there!
This is my very first post.
I'll use it to share some old news: a math preference dataset for DPO!
I created this dataset some time ago while we were developing distilabel (https://github.com/argilla-io/distilabel).
Some days ago we found out people are actually using it! So I'll use this post to explain how I built it in case it's useful for the community.
1. I used distilabel's SelfInstruct-inspired task to generate instructions about different math topics. I curated the instructions with Argilla (on Spaces!).
2. Then I used a distilabel Pipeline to build a preference dataset using gpt3.5 as generator and gpt4 as labeller. If I recall correctly I used our JudgeLM implementation (see https://distilabel.argilla.io/latest/technical-reference/tasks/#judgelmtask)
(see the screenshot with the dataset in the Argilla UI)
3. Then I just binarized into chosen, rejected pairs and voilà:
argilla/distilabel-math-preference-dpo
The funny thing is that I used this to do a second DPO run over Notus-7B. I hoped to see an improvement on math/reasoning skills but it actually improved in STEM and Humanities and did worse on Math 🤣 .
In conclusion, this dataset was only a quick experiement. I'm happy to see the community found it useful. Data for DPO and fine-tuning are still a mystery, let's unveil these mysteries in 2024 together!
Follow me for the most exciting datasets for LLMs (and maybe some great, small, efficient models). I plan to announce all Argilla open-source work here!
This is my very first post.
I'll use it to share some old news: a math preference dataset for DPO!
I created this dataset some time ago while we were developing distilabel (https://github.com/argilla-io/distilabel).
Some days ago we found out people are actually using it! So I'll use this post to explain how I built it in case it's useful for the community.
1. I used distilabel's SelfInstruct-inspired task to generate instructions about different math topics. I curated the instructions with Argilla (on Spaces!).
2. Then I used a distilabel Pipeline to build a preference dataset using gpt3.5 as generator and gpt4 as labeller. If I recall correctly I used our JudgeLM implementation (see https://distilabel.argilla.io/latest/technical-reference/tasks/#judgelmtask)
(see the screenshot with the dataset in the Argilla UI)
3. Then I just binarized into chosen, rejected pairs and voilà:
argilla/distilabel-math-preference-dpo
The funny thing is that I used this to do a second DPO run over Notus-7B. I hoped to see an improvement on math/reasoning skills but it actually improved in STEM and Humanities and did worse on Math 🤣 .
In conclusion, this dataset was only a quick experiement. I'm happy to see the community found it useful. Data for DPO and fine-tuning are still a mystery, let's unveil these mysteries in 2024 together!
Follow me for the most exciting datasets for LLMs (and maybe some great, small, efficient models). I plan to announce all Argilla open-source work here!
- 2 replies