OpenMOSE

/

Hy3-REAP-150B-21B

like 0

Hi. I got this to test on a 128gb strix halo. I gave it simple prompts like tetris, flappy bird in html. It gives me a very short result that includes some nonsense, repetition of tags etc. I also tried a settlers of catan prompt, while the planning phase seems to go reasonably well, when building it gets stuck outputting the tt.. sequence endlessly. Not sure I am doing something wrong.

HIP_VISIBLE_DEVICES=0 \
llama-server \
  --model "$HOME/models/IQ4_XS/Hy3-REAP-200B-GGUF-IQ4_XS-00001-of-*.gguf" \
  --chat-template-file "$HOME/models/hyv3_opensource_chat_template.jinja" \
  -ngl 999 -c 110000 -t "$(nproc)" \
  -ctk q8_0 -ctv q8_0 \
  --no-warmup --no-mmap \
  --host 0.0.0.0 --port 8080

GGML_VULKAN=1 \
llama-server \
  --model "$HOME/models/IQ4_XS/Hy3-REAP-200B-GGUF-IQ4_XS-00001-of-*.gguf" \
  --chat-template-file "$HOME/models/hyv3_opensource_chat_template.jinja" \
  -ngl 999 -c 65000 -t "$(nproc)" \
  --no-warmup --no-mmap \
  -np 2 \
  --host 0.0.0.0 --port 8080

tested with both rocm and vulkan, q8/q16 kv