gpt-engineer/docs/open_models.md

Using with open/local models

You can integrate gpt-engineer with open-source models by leveraging an OpenAI-compatible API. One such API is provided by the text-generator-ui extension openai.

Setup

To get started, first set up the API with the Runpod template, as per the instructions.

Running the Example

Once the API is set up, you can find the host and the exposed TCP port by checking your Runpod dashboard.

Then, you can use the port and host to run the following example using WizardCoder-Python-34B hosted on Runpod:

  OPENAI_API_BASE=http://<host>:<port>/v1 python -m gpt_engineer.cli.main benchmark/pomodoro_timer --steps benchmark TheBloke_WizardCoder-Python-34B-V1.0-GPTQ

Using Azure models

You set your Azure OpenAI key:

• export OPENAI_API_KEY=[your api key]

Then you call gpt-engineer with your service endpoint --azure https://aoi-resource-name.openai.azure.com and set your deployment name (which you created in the Azure AI Studio) as the model name (last gpt-engineer argument).

Example: gpt-engineer --azure https://myairesource.openai.azure.com ./projects/example/ my-gpt4-project-name