Update README.md
Browse files
README.md
CHANGED
|
@@ -57,6 +57,7 @@ license_link: LICENSE
|
|
| 57 |
# DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
|
| 58 |
|
| 59 |
All this fork does is take away the flash-attn so that it runs with ZeroGPU
|
|
|
|
| 60 |
|
| 61 |
## 1. Introduction
|
| 62 |
We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks. Compared to DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in various aspects of code-related tasks, as well as reasoning and general capabilities. Additionally, DeepSeek-Coder-V2 expands its support for programming languages from 86 to 338, while extending the context length from 16K to 128K.
|
|
|
|
| 57 |
# DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
|
| 58 |
|
| 59 |
All this fork does is take away the flash-attn so that it runs with ZeroGPU
|
| 60 |
+
UPDATE: might be unneeded now because they fixed it in the unstream repo.
|
| 61 |
|
| 62 |
## 1. Introduction
|
| 63 |
We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks. Compared to DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in various aspects of code-related tasks, as well as reasoning and general capabilities. Additionally, DeepSeek-Coder-V2 expands its support for programming languages from 86 to 338, while extending the context length from 16K to 128K.
|