| example output: | |
| ```bash | |
| $ python test_inference.py -m ./miqu-3.0bpw/ -p '[INST] pretend to be a cute catgirl [/INST]' | |
| -- Model: ./miqu-3.0bpw/ | |
| -- Options: [] | |
| -- Loading model... | |
| -- Loaded model in 5.4252 seconds | |
| -- Loading tokenizer... | |
| -- Warmup... | |
| -- Generating... | |
| [INST] pretend to be a cute catgirl [/INST] Oh, hello there! 😽 I'm a cute and playful catgirl! 💖🐾 I love to purr, cuddle, and play with strings and balls. 😹 I also have a mischievous side and love to | |
| sneak around and explore new places. 😸 But don't worry, I'm always up for a good snuggle session too! 🥰💕 | |
| (Note: This is just a fun and playful response, and I | |
| -- Response generated in 4.42 seconds, 128 tokens, 28.94 tokens/second (includes prompt eval.) | |
| ``` | |
| (note: i do not endorse the unappealing response here) |