HuggingFaceH4

/

zephyr-7b-beta

like 1.46k

Hi, are you still maintaining this? I tried using it but it seems to time out/not return a response. Perhaps it's overloaded or some other issue. The only endpoint I can get to successfully return a response is the GET /v1/models. Thanks in advance,

I just tested with the API doc at https://limcheekin-zephyr-7b-beta-gguf.hf.space/docs. It is working for a simple prompt.

If you use it for longer generation/response, you need to set stream=True.
Please refer to the following notebook at https://huggingface.co/spaces/limcheekin/Mistral-7B-Instruct-v0.1-GGUF/blob/main/mistral-7b-instruct.ipynb.

Hope this help.

Hi, are you still maintaining this? I tried using it but it seems to time out/not return a response. Perhaps it's overloaded or some other issue. The only endpoint I can get to successfully return a response is the GET /v1/models. Thanks in advance,

Added a jupyter notebook at https://huggingface.co/spaces/limcheekin/zephyr-7B-beta-GGUF/blob/main/zephyr-7b.ipynb

I hope this is helpful to you. :)

How can I create my own o'penai API with zephyr-7b-beta and share your code? Thank you

You just need to select "Duplicate this Space" from the drop down menu.

All codes are located at https://huggingface.co/spaces/limcheekin/zephyr-7B-beta-GGUF/tree/main.

By the way, you can use git to clone the repo of the space.

I hope this helps and clarify.

Thank you very much. I have just learned about Zephyr, and I have seen your way of using CPP. I would like to directly use Zephyr-7b-beta, while also implementing stream output.
I find ohter coder create api by using pipeline in the github,i want to use AutoModelForCausalLM.from_pretrained.

I didn't do that, can't help you. Appreciate your sharing if you managed to publish the api.

Thanks.

i got an error:
AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = ', or you can set the environment variable OPENAI_API_KEY=). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path =

i got an error:
AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = ', or you can set the environment variable OPENAI_API_KEY=). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path =

Set your HF token as the API key.