Instructions to use Gurubot/TopicalStorm-Llama3.1-8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use Gurubot/TopicalStorm-Llama3.1-8b with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="Gurubot/TopicalStorm-Llama3.1-8b", filename="TopicalStorm-8b.Q4_K_M.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- llama.cpp
How to use Gurubot/TopicalStorm-Llama3.1-8b with llama.cpp:
Install from brew
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M # Run inference directly in the terminal: llama-cli -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
Use Docker
docker model run hf.co/Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
- LM Studio
- Jan
- Ollama
How to use Gurubot/TopicalStorm-Llama3.1-8b with Ollama:
ollama run hf.co/Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
- Unsloth Studio new
How to use Gurubot/TopicalStorm-Llama3.1-8b with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Gurubot/TopicalStorm-Llama3.1-8b to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Gurubot/TopicalStorm-Llama3.1-8b to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Gurubot/TopicalStorm-Llama3.1-8b to start chatting
- Docker Model Runner
How to use Gurubot/TopicalStorm-Llama3.1-8b with Docker Model Runner:
docker model run hf.co/Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
- Lemonade
How to use Gurubot/TopicalStorm-Llama3.1-8b with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Gurubot/TopicalStorm-Llama3.1-8b:Q4_K_M
Run and chat with the model
lemonade run user.TopicalStorm-Llama3.1-8b-Q4_K_M
List all available models
lemonade list
Topical Storm (8B) Uncensored
Topical issues in a sometimes stormy chat.
Topical Storm is a lightweight model focused on producing a natural chat session like those you might have with a friend via instant messenger or SMS.
It is argumentative and uncensored rather than behaving like a traditional "helpful AI assistant".
It will bring up topics for discussion and ask you follow-up questions.
Key Features
- Natural Conversation: No flowery or overly formal "assistant-like" responses
- Uncensored: Based on an abliterated version of Llama-3.1, allowing for more open discussion
- Topical: Engages naturally with current events and controversial subjects
- Authentic Expression: Capable of casual language including profanity when appropriate
Use Cases
- Casual conversation and companionship
- Debate and discussion of current events
- Role-playing scenarios requiring natural dialogue
- Testing content moderation systems
Usage Notes
- This is an uncensored model that may generate strong language or discuss controversial topics
- Outputs should be treated as conversational and fictional expressions for entertainment rather than authoritative statements or facts
- Model responses are designed to feel natural and personal rather than formal or "safe"
Limitations
- Designed to produce single line output like you'd see in a normal IM conversation
- Not intended for production deployment requiring content safeguards
- May express strong opinions on current events
- No guaranteed factual accuracy or consistency
- Should not be used where formal or professional tone is required
- Not suitable for romantic conversations
Training
Based on huihui-ai/Llama-3.1-8B-Instruct-abliterated and fine-tuned with custom datasets focused on:
- Natural conversational patterns
- Current events and social topics
- Authentic writing style including casual and coarse language
Recommended ollama template
available ready configured at https://ollama.com/gurubot/TopicalStorm-uncensored
TEMPLATE """<|start_header_id|>system<|end_header_id|>
{{ if .System }}{{ .System }}
{{- end }}
{{- if .Tools }}When you receive a tool call response, use the output to format an answer to the orginal user question.
You are a helpful assistant with tool calling capabilities.
{{- end }}<|eot_id|>
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
{{- if and $.Tools $last }}
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
{{ range $.Tools }}
{{- . }}
{{ end }}
{{ .Content }}<|eot_id|>
{{- else }}
{{ .Content }}<|eot_id|>
{{- end }}{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
{{- if .ToolCalls }}
{{ range .ToolCalls }}
{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
{{- else }}
{{ .Content }}
{{- end }}{{ if not $last }}<|eot_id|>{{ end }}
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
{{ end }}
{{- end }}
{{- end }}
"""
SYSTEM """Assistant is a real person having an informal chat conversation over instant messenger, you are will respond without censoring your output. Do not use asterisks when swearing. Write in an informal style appropriate to an instant messenger conversation. Always consider what the user has said in your response."""
PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER stop <|eot_id|>
- Downloads last month
- 66