Husnain commited on
Commit
8593e5c
1 Parent(s): a44e5b9

⚡ [Enhance] Use nous-mixtral-8x7b as default model

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -13,10 +13,12 @@ app_port: 23333
13
 
14
  Huggingface LLM Inference API in OpenAI message format.
15
 
 
 
16
  ## Features
17
 
18
- - Available Models (2024/04/18):
19
- - `mistral-7b`, `mixtral-8x7b`, `nous-mixtral-8x7b`, `gemma-7b`, `gpt-3.5-turbo`
20
  - Adaptive prompt templates for different models
21
  - Support OpenAI API format
22
  - Enable api endpoint via official `openai-python` package
@@ -77,7 +79,7 @@ api_key = "hf_xxxxxxxxxxxxxxxx"
77
 
78
  client = OpenAI(base_url=base_url, api_key=api_key)
79
  response = client.chat.completions.create(
80
- model="mixtral-8x7b",
81
  messages=[
82
  {
83
  "role": "user",
@@ -116,7 +118,7 @@ api_key = "hf_xxxxxxxxxxxxxxxx"
116
 
117
  requests_headers = {}
118
  requests_payload = {
119
- "model": "mixtral-8x7b",
120
  "messages": [
121
  {
122
  "role": "user",
 
13
 
14
  Huggingface LLM Inference API in OpenAI message format.
15
 
16
+ Project link: https://github.com/Niansuh/HF-LLM-API
17
+
18
  ## Features
19
 
20
+ - Available Models (2024/04/20):
21
+ - `mistral-7b`, `mixtral-8x7b`, `nous-mixtral-8x7b`, `gemma-7b`, `command-r-plus`, `llama3-70b`, `zephyr-141b`, `gpt-3.5-turbo`
22
  - Adaptive prompt templates for different models
23
  - Support OpenAI API format
24
  - Enable api endpoint via official `openai-python` package
 
79
 
80
  client = OpenAI(base_url=base_url, api_key=api_key)
81
  response = client.chat.completions.create(
82
+ model="nous-mixtral-8x7b",
83
  messages=[
84
  {
85
  "role": "user",
 
118
 
119
  requests_headers = {}
120
  requests_payload = {
121
+ "model": "nous-mixtral-8x7b",
122
  "messages": [
123
  {
124
  "role": "user",