NAN issue using FP16 to load the model
#15 opened 2 months ago
by
joeltseng
ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn. Run `pip install flash_attn`
#14 opened 6 months ago
by
kang1
How much memory is needed if you make the 128k context length
1
#13 opened 7 months ago
by
ggbondcxk
Implement MLA inference optimizations to DeepseekV2Attention
#12 opened 7 months ago
by
sy-chen
Can you provide a sample code for training with DeepSpeed ZeRO3?
2
#10 opened 7 months ago
by
SupercarryNg
Ollama support
1
#9 opened 7 months ago
by
Dao3
MoE offloading strategy?
2
#8 opened 8 months ago
by
Minami-su
Update README.md
#7 opened 8 months ago
by
VanishingPsychopath
kv cache
3
#6 opened 8 months ago
by
FrankWu
function/tool calling support
7
#5 opened 8 months ago
by
kaijietti
fail to run the example
8
#4 opened 8 months ago
by
Leymore
GPTQ plz
10
#3 opened 8 months ago
by
Parkerlambert123
vllm support
7
#2 opened 8 months ago
by
Sihangli
llama.cpp support
5
#1 opened 8 months ago
by
cpumaxx