Spaces:
Sleeping
Sleeping
Commit
•
8c64a35
1
Parent(s):
2a25399
Readme: Note on Mistral API used, serverless backend for reliability
Browse files
README.md
CHANGED
@@ -32,12 +32,13 @@ Could Reddit, Twitter, Hacker News, or even YouTube comments be more calm and co
|
|
32 |
## Aims to:
|
33 |
- **Convert** text to less toxic variations
|
34 |
- **Preserve original intent**, focusing on constructive dialogue
|
35 |
-
- **Self-hostable, serverless, or APIs**: running DIY could save costs, avoid needing to sign up to APIs, and avoid the risk of toxic content causing API access to be revoked. We use llama-cpp-python with Mixtral, with a HTTP server option,
|
36 |
|
37 |
## Possible future directions 🌟
|
38 |
|
39 |
**Speed:**
|
40 |
- Generating rephrasings in parallel.
|
|
|
41 |
- Show intermediate results to the user, while waiting for the final result.
|
42 |
- Split text into sentences e.g: with “pysbd” for parallel processing of translations.
|
43 |
|
|
|
32 |
## Aims to:
|
33 |
- **Convert** text to less toxic variations
|
34 |
- **Preserve original intent**, focusing on constructive dialogue
|
35 |
+
- **Self-hostable, serverless, or APIs**: running DIY could save costs, avoid needing to sign up to APIs, and avoid the risk of toxic content causing API access to be revoked. We use llama-cpp-python with Mixtral, with a HTTP server option, a fast "serverless" backend using RunPod currently which had some reliability issues, so I'm using Mistral's own API right now until I can figure out a more reliable serverless method.
|
36 |
|
37 |
## Possible future directions 🌟
|
38 |
|
39 |
**Speed:**
|
40 |
- Generating rephrasings in parallel.
|
41 |
+
- Combined some LLM tasks together, to reduce request overhead.
|
42 |
- Show intermediate results to the user, while waiting for the final result.
|
43 |
- Split text into sentences e.g: with “pysbd” for parallel processing of translations.
|
44 |
|