metadata

title: ❄️ ChillTranslator 🤬 ➡️ 😎💬
emoji: ❄️
colorFrom: red
colorTo: blue
sdk: docker
pinned: false

❄️ ChillTranslator 🤬 ➡️ 😎💬

This is an early experimental tool aimed at helping reduce online toxicity by automatically ➡️ transforming 🌶️ spicy or toxic comments into constructive, ❤️ kinder dialogues using AI and large language models.

You can try out the ChillTranslator via the HuggingFace Space demo at https://huggingface.co/spaces/lukestanley/ChillTranslator.

It can also be used via the command line to improve a specific text of your choice, or it can be used as a module. All of the code is in this repo, for the serverles worker, the logic, and Gradio UI.

ChillTranslator aims to help make online interactions more healthy.

Online toxicity can undermine the quality of discourse, causing distress 😞 and driving people away from online communities. Or worse: it can create a viral toxic loop 🌀!

ChillTranslator hopes to mitigate toxic comments by automatically rephrasing negative comments, while maintaining the original intent and promoting positive communication 🗣️➡️💬. These rephrased texts could be suggested to the original authors as alternatives, or users could enhance their internet experience with "rose-tinted glasses" 🌹😎, automatically translating spicy comments into versions that are easier and more calming to read. There could be all kinds of failure cases, but hey, it's a start!

Could Reddit, Twitter, Hacker News, or even YouTube comments be more calm and constructive places? I think so!

Aims to:

Convert text to less toxic variations
Preserve original intent, focusing on constructive dialogue
Self-hostable, serverless, or APIs: running DIY could save costs, avoid needing to sign up to APIs, and avoid the risk of toxic content causing API access to be revoked. We use llama-cpp-python with Mixtral, with a HTTP server option, a fast "serverless" backend using RunPod currently which had some reliability issues, so I'm using Mistral's own API right now until I can figure out a more reliable serverless method.

Possible future directions 🌟

Speed:

Generating rephrasings in parallel.
Combined some LLM tasks together, to reduce request overhead.
Show intermediate results to the user, while waiting for the final result.
Split text into sentences e.g: with “pysbd” for parallel processing of translations.

Speed and Quality:

Use Jigsaw dataset to find spicy comments, making a dataset for training a translation transformer, maybe like Google's T5 to run faster than Mixtral could.
Try using a 'Detoxify' scoring model instead of the current "spicy" score method.
Use natural language similarity techniques to compare possible rephrasing fidelity faster.
Collecting a dataset of spicy comments and their rephrasings.
Feedback loop: users could score rephrasings, or suggest their own.

Distribution:

Better example showing use as Python module, HTTP API, for use from other tools, browser extensions.
Enabling easy experimenting with online hosted LLM APIs
Making setup on different platforms easier

Getting started 🚀

Try it online

You can try out ChillTranslator without any installation by visiting the HuggingFace Space demo:

https://huggingface.co/spaces/lukestanley/ChillTranslator

Installation

Clone the Project Repository:

git clone https://github.com/lukestanley/ChillTranslator.git
cd ChillTranslator

It will automaticaly download Mixtral-8x7B-Instruct-v0.1-GGUF by default. The model HuggingFace repo and filename can be switched by enviroment variables, or you can point to a different local path.

Install dependencies, including a special fork of llama-cpp-python, and Nvidia GPU support if needed:

pip install requests pydantic uvicorn starlette fastapi sse_starlette starlette_context pydantic_settings

# If you have an Nvidia GPU, install the special fork of llama-cpp-python with CUBLAS support:
CMAKE_ARGS="-DLLAMA_CUBLAS=on" pip install git+https://github.com/lukestanley/llama-cpp-python.git@expose_json_grammar_convert_function

If you don't have an Nvidia GPU, the CMAKE_ARGS="-DLLAMA_CUBLAS=on" is not needed before the pip install command.

Start the LLM server with your chosen configuration. Example for Nvidia with --n_gpu_layers set to 20; different GPUs fit more or less layers. If you have no GPU, you don't need the --n_gpu_layers flag:
```
python3 -m llama_cpp.server --model mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf --port 5834 --n_ctx 4096 --use_mlock false --n_gpu_layers 20 &
```

These config options are likely to need tweaking. Please check out https://llama-cpp-python.readthedocs.io/en/latest/ for more info.

Local Usage

ChillTranslator can be used locally to improve specific texts. This is how to see it in action from the command line with a default text:

python3 chill.py

For improving a specific text of your choice, use the -t flag followed by your text enclosed in quotes:

python3 chill.py -t "Your text goes here"

To run the Gradio web server GUI:

python3 app.py

Or chill can be imported as a module, with the improvement_loop function provided the text to improve.

Contributing 🤝

Contributions are very welcome! Especially:

pull requests,
free GPU credits
LLM API credits / access.

ChillTranslator is released under the MIT License.

Help make the internet a kinder place, one comment at a time. Your contribution could make a big difference!