Package: ollama-python
Ollama Python LibraryThe ollama python library provides the easiest way to integrate your python project withOllamaGetting StartedThis requires a python version of 3.9 or higherpipinstallollama-pythonThe python package splits the functionality into three core endpointsModel Management Endpoints: This includes the ability to create, delete, pull, push and list models amongst othersGenerate Endpoint: This includes the generate and chat endpoints in OllamaEmbedding Endpoint: This includes the ability to generate embeddings for a given textPydantic is used to verify user input and Responses from the server are parsed into pydantic modelsExample UsageGenerate EndpointCompletions (Generate)Without Streamingfromollama_python.endpointsimportGenerateAPIapi=GenerateAPI(base_url="http://localhost:8000",model="mistral")result=api.generate(prompt="Hello World",options=dict(num_tokens=10),format="json")With Streamingfromollama_python.endpointsimportGenerateAPIapi=GenerateAPI(base_url="http://localhost:8000",model="mistral")forresinapi.generate(prompt="Hello World",options=dict(num_tokens=10),format="json",stream=True):print(res.response)Chat CompletionsWithout Streamingfromollama_python.endpointsimportGenerateAPIapi=GenerateAPI(base_url="http://localhost:8000",model="mistral")messages=[{'role':'user','content':'Why is the sky blue?'}]result=api.generate_chat_completion(messages=messages,options=dict(num_tokens=10),format="json")With Streamingfromollama_python.endpointsimportGenerateAPIapi=GenerateAPI(base_url="http://localhost:8000",model="mistral")messages=[{'role':'user','content':'Why is the sky blue?'}]forresinapi.generate_chat_completion(messages=messages,options=dict(num_tokens=10),format="json",stream=True):print(res.message)Chat request with imagesfromollama_python.endpointsimportGenerateAPIapi=GenerateAPI(base_url="http://localhost:8000",model="llava")messages=[{'role':'user','content':'What is in this image','image':'iVBORw0KGgoAAAANSUhEUgAAAG0AAABmCAYAAADBPx+VAAAACXBIWXMAAAsTAAALEwEAmp'}]result=api.generate_chat_completion(messages=messages,options=dict(num_tokens=10),format="json")print(result.message)Embeddings EndpointGenerate Embeddingsfromollama_python.endpointsimportEmbeddingAPIapi=EmbeddingAPI(base_url="http://localhost:8000",model="mistral")result=api.get_embedding(prompt="Hello World",options=dict(seed=10))Model Management EndpointsCreate a modelWithout Streamingfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.create(name="test_model",model_file="random model_file")With Streamingfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")forresinapi.create(name="test_model",model_file="random model_file",stream=True):print(res.status)Check if a blob existsfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.check_blob_exists(digest="sha256:29fdb92e57cf0827ded04ae6461b5931d01fa595843f55d36f5b275a52087dd2")Create a blobfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.create_blob(digest="sha256:29fdb92e57cf0827ded04ae6461b5931d01fa595843f55d36f5b275a52087dd2")List local modelsfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.list_local_models()print(result.models)Show model informationfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.show(name="mistral")print(result.details)Copy a modelfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.copy(source="mistral",destination="mistral_copy")Delete a modelfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")api.delete(name="mistral_copy")Pull a modelWithout Streamingfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.pull(name="mistral")print(result.status)With Streamingfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")forresinapi.pull(name="mistral",stream=True):print(res.status)Push a modelWithout Streamingfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")result=api.push(name="mistral")print(result.status)With Streamingfromollama_python.endpointsimportModelManagementAPIapi=ModelManagementAPI(base_url="http://localhost:8000")forresinapi.push(name="mistral",stream=True):print(res.status)Valid Options/ParametersParameterDescriptionValue TypeExample UsagemirostatEnable Mirostat sampling for controlling perplexity. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)intmirostat 0mirostat_etaInfluences how quickly the algorithm responds to feedback from the generated text. A lower learning rate will result in slower adjustments, while a higher learning rate will make the algorithm more responsive. (Default: 0.1)floatmirostat_eta 0.1mirostat_tauControls the balance between coherence and diversity of the output. A lower value will result in more focused and coherent text. (Default: 5.0)floatmirostat_tau 5.0num_ctxSets the size of the context window used to generate the next token. (Default: 2048)intnum_ctx 4096num_gqaThe number of GQA groups in the transformer layer. Required for some models, for example it is 8 for llama2:70bintnum_gqa 1num_gpuThe number of layers to send to the GPU(s). On macOS it defaults to 1 to enable metal support, 0 to disable.intnum_gpu 50num_threadSets the number of threads to use during computation. By default, Ollama will detect this for optimal performance. It is recommended to set this value to the number of physical CPU cores your system has (as opposed to the logical number of cores).intnum_thread 8repeat_last_nSets how far back for the model to look back to prevent repetition. (Default: 64, 0 = disabled, -1 = num_ctx)intrepeat_last_n 64repeat_penaltySets how strongly to penalize repetitions. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. (Default: 1.1)floatrepeat_penalty 1.1temperatureThe temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)floattemperature 0.7seedSets the random number seed to use for generation. Setting this to a specific number will make the model generate the same text for the same prompt. (Default: 0)intseed 42stopSets the stop sequences to use. When this pattern is encountered the LLM will stop generating text and return. Multiple stop patterns may be set by specifying multiple separatestopparameters in a modelfile.stringstop "AI assistant:"tfs_zTail free sampling is used to reduce the impact of less probable tokens from the output. A higher value (e.g., 2.0) will reduce the impact more, while a value of 1.0 disables this setting. (default: 1)floattfs_z 1num_predictMaximum number of tokens to predict when generating text. (Default: 128, -1 = infinite generation, -2 = fill context)intnum_predict 42top_kReduces the probability of generating nonsense. A higher value (e.g. 100) will give more diverse answers, while a lower value (e.g. 10) will be more conservative. (Default: 40)inttop_k 40top_pWorks together with top-k. A higher value (e.g., 0.95) will lead to more diverse text, while a lower value (e.g., 0.5) will generate more focused and conservative text. (Default: 0.9)floattop_p 0.9TodoAdd support for Asynchronous version of the libraryTo ContributeClone the repoRunpoetry installRunpre-commit installThen you're ready to contribute to the repo