Text Generation
Transformers
Safetensors
English
mistral
instruct
finetune
chatml
gpt4
conversational
text-generation-inference
Instructions to use FPHam/Writing_Partner_Mistral_7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FPHam/Writing_Partner_Mistral_7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FPHam/Writing_Partner_Mistral_7B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("FPHam/Writing_Partner_Mistral_7B") model = AutoModelForCausalLM.from_pretrained("FPHam/Writing_Partner_Mistral_7B") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use FPHam/Writing_Partner_Mistral_7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FPHam/Writing_Partner_Mistral_7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FPHam/Writing_Partner_Mistral_7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/FPHam/Writing_Partner_Mistral_7B
- SGLang
How to use FPHam/Writing_Partner_Mistral_7B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FPHam/Writing_Partner_Mistral_7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FPHam/Writing_Partner_Mistral_7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FPHam/Writing_Partner_Mistral_7B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FPHam/Writing_Partner_Mistral_7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use FPHam/Writing_Partner_Mistral_7B with Docker Model Runner:
docker model run hf.co/FPHam/Writing_Partner_Mistral_7B
Update README.md
Browse files
README.md
CHANGED
|
@@ -23,7 +23,6 @@ tags:
|
|
| 23 |
|
| 24 |
|
| 25 |
Hi there! I'm your writing assistant.
|
| 26 |
-
(Note: Generated by ChatGPT)
|
| 27 |
|
| 28 |
Think of me as your partner in crime---or rather, your partner in writing. When you hit a wall (and trust me, you will), I'll be there to knock it down for you.
|
| 29 |
|
|
@@ -48,6 +47,8 @@ How to plot my story?<|im_end|>
|
|
| 48 |
|
| 49 |
Here is an opening of my story:
|
| 50 |
|
|
|
|
|
|
|
| 51 |
In the distant reaches of the cosmos, where starlight weaves a tapestry of unknown wonders, Captain Alara Voss navigated her spacecraft, the Celestial Serenity, through the interstellar currents. The ship hummed with the pulsating energy of the quantum drive, propelling them toward the uncharted nebula known as the Veil of Eternity. Alara's gaze was fixed on the swirling cosmic clouds ahead, her mind ablaze with the anticipation of discovery. As the Serenity pierced the ethereal veil, the fabric of reality seemed to dance with a mysterious resonance. Little did the intrepid crew know that beyond the shimmering curtain lay a realm where time itself wove a narrative that defied mortal understanding, setting the stage for an odyssey that would reshape the boundaries of human perception.
|
| 52 |
|
| 53 |
>I like this opening! It sets up some interesting questions and intrigue right off the bat. But I do think you might want to reconsider your point of view in the first paragraph. Right now it sounds more like a summary than an actual scene because there isn't any dialogue or action. Consider starting with something more immediate, perhaps from Alara's perspective. You could even begin with her looking out at the stars, thinking about what she sees. This will give us a chance to get to know her better before you launch into the big picture stuff. Here's an example:
|
|
|
|
| 23 |
|
| 24 |
|
| 25 |
Hi there! I'm your writing assistant.
|
|
|
|
| 26 |
|
| 27 |
Think of me as your partner in crime---or rather, your partner in writing. When you hit a wall (and trust me, you will), I'll be there to knock it down for you.
|
| 28 |
|
|
|
|
| 47 |
|
| 48 |
Here is an opening of my story:
|
| 49 |
|
| 50 |
+
(Note: Generated by ChatGPT)
|
| 51 |
+
|
| 52 |
In the distant reaches of the cosmos, where starlight weaves a tapestry of unknown wonders, Captain Alara Voss navigated her spacecraft, the Celestial Serenity, through the interstellar currents. The ship hummed with the pulsating energy of the quantum drive, propelling them toward the uncharted nebula known as the Veil of Eternity. Alara's gaze was fixed on the swirling cosmic clouds ahead, her mind ablaze with the anticipation of discovery. As the Serenity pierced the ethereal veil, the fabric of reality seemed to dance with a mysterious resonance. Little did the intrepid crew know that beyond the shimmering curtain lay a realm where time itself wove a narrative that defied mortal understanding, setting the stage for an odyssey that would reshape the boundaries of human perception.
|
| 53 |
|
| 54 |
>I like this opening! It sets up some interesting questions and intrigue right off the bat. But I do think you might want to reconsider your point of view in the first paragraph. Right now it sounds more like a summary than an actual scene because there isn't any dialogue or action. Consider starting with something more immediate, perhaps from Alara's perspective. You could even begin with her looking out at the stars, thinking about what she sees. This will give us a chance to get to know her better before you launch into the big picture stuff. Here's an example:
|