Repository: openvinotoolkit/openvino_contrib
Pull request #966: The NPU output length is set to 2048 by default.
Author: login: zhaohb
Labels: name: category: CI
description: OpenVINO public CI
Created: 2025-05-14T08:36:39Z | Merged: 2025-05-21T07:06:42Z

Upgrade to GenAI version 2025.2.0.0.dev20250513 for the latest features and improvements.


Diff:
diff --git a/.github/workflows/ollama_openvino_build_and_test.yml b/.github/workflows/ollama_openvino_build_and_test.yml
index decbe9beba..e5fe94f772 100644
--- a/.github/workflows/ollama_openvino_build_and_test.yml
+++ b/.github/workflows/ollama_openvino_build_and_test.yml
@@ -9,8 +9,8 @@ on:
 permissions: read-all
 
 jobs:
-  test_ubuntu20:
-    runs-on: ubuntu-20.04
+  test_ubuntu22:
+    runs-on: ubuntu-22.04
     steps:
       - name: Download repo
         uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4.1.6
diff --git a/modules/ollama_openvino/Dockerfile_genai b/modules/ollama_openvino/Dockerfile_genai
index a496f16896..d97807ed77 100644
--- a/modules/ollama_openvino/Dockerfile_genai
+++ b/modules/ollama_openvino/Dockerfile_genai
@@ -22,9 +22,9 @@ ARG GOVERSION=1.24.1
 RUN curl -fsSL https://golang.org/dl/go${GOVERSION}.linux-$(case $(uname -m) in x86_64) echo amd64 ;; aarch64) echo arm64 ;; esac).tar.gz | tar xz -C /usr/local
 ENV PATH=/usr/local/go/bin:$PATH
 
-RUN wget https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu20_2025.2.0.0.dev20250320_x86_64.tar.gz
-RUN tar -xzf openvino_genai_ubuntu20_2025.2.0.0.dev20250320_x86_64.tar.gz
-ENV GENAI_DIR=/home/ollama_ov_server/openvino_genai_ubuntu20_2025.2.0.0.dev20250320_x86_64
+RUN wget https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_ubuntu22_2025.2.0.0.dev20250513_x86_64.tar.gz
+RUN tar -xzf openvino_genai_ubuntu20_2025.2.0.0.dev20250513_x86_64.tar.gz
+ENV GENAI_DIR=/home/ollama_ov_server/openvino_genai_ubuntu20_2025.2.0.0.dev20250513_x86_64
 
 ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GENAI_DIR/runtime/3rdparty/tbb/lib:$GENAI_DIR/runtime/lib/intel64
 
@@ -35,8 +35,8 @@ ENV CGO_LDFLAGS=-L$GENAI_DIR/runtime/lib/intel64
 ENV CGO_CFLAGS=-I$GENAI_DIR/runtime/include
 
 WORKDIR /home/ollama_ov_server
-RUN git clone https://github.com/zhaohb/ollama_ov.git
-WORKDIR /home/ollama_ov_server/ollama_ov
+RUN git clone https://github.com/openvinotoolkit/openvino_contrib.git
+WORKDIR /home/ollama_ov_server/openvino_contrib/modules/ollama_openvino
 
 RUN go build -o /bin/ollama .
 
diff --git a/modules/ollama_openvino/README.md b/modules/ollama_openvino/README.md
index e5d3bd265f..c25002327f 100644
--- a/modules/ollama_openvino/README.md
+++ b/modules/ollama_openvino/README.md
@@ -584,17 +584,17 @@ Getting started with large language models and using the [GenAI](https://github.
 We provide two ways to download the executable file of Ollama, one is to download it from Google Drive, and the other is to download it from Baidu Drive:
 ## Google Driver
 ### Windows
-[Download exe](https://drive.google.com/file/d/1Sep1IdGn7mJaE8PCXKYxp_aj1ljiPvpN/view?usp=sharing) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip)
+[Download exe](https://drive.google.com/file/d/1Xo3ohbfC852KtJy_4xtn_YrYaH4Y_507/view?usp=sharing) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_windows_2025.2.0.0.dev20250513_x86_64.zip)
 
 ### Linux(Ubuntu 22.04)
-[Download](https://drive.google.com/file/d/1DdBoEGp_eoyJPbpMGVbEivihYSKrCMGt/view?usp=sharing) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz)
+[Download](https://drive.google.com/file/d/1_P7CQqFUqeyx4q5y5bQ-xQsb10T9gzJD/view?usp=sharing) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_ubuntu22_2025.2.0.0.dev20250513_x86_64.tar.gz)
 
 ## 百度云盘
 ### Windows
-[Download exe](https://pan.baidu.com/s/1TCH7rYSPr8jQDHLvCeXdLw?pwd=6bk9) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip)
+[Download exe](https://pan.baidu.com/s/1uIUjji7Mxf594CJy1vbrVw?pwd=36mq) + [Download OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_windows_2025.2.0.0.dev20250513_x86_64.zip)
 
 ### Linux(Ubuntu 22.04)
-[Download](https://pan.baidu.com/s/1UVO0ZK4DFTjTwfarQ8LUIw?pwd=pxkd) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz)
+[Download](https://pan.baidu.com/s/1OCq3aKJBiCrtjLKa7kXbMw?pwd=exhz) + [Donwload OpenVINO GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_ubuntu22_2025.2.0.0.dev20250513_x86_64.tar.gz)
 
 ## Docker
 ### Linux
@@ -715,7 +715,7 @@ Let's take [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://hf-mirror.com/deeps
     ```
 
 3. Create a file named `Modelfile`, with a `FROM` instruction with the local filepath to the model you want to import.
-   For convenience, we have put the model file of DeepSeek-R1-Distill-Qwen-7B-int4-ov model under example dir: [Modelfile for DeepSeek](https://github.com/zhaohb/ollama_ov/blob/main/examples/modelfile/deepseek_r1_distill_qwen/Modelfile), we can use it directly.
+   For convenience, we have put the model file of DeepSeek-R1-Distill-Qwen-7B-int4-ov model under example dir: [Modelfile for DeepSeek](https://github.com/openvinotoolkit/openvino_contrib/blob/master/modules/ollama_openvino/examples/modelfile/deepseek_r1_distill_qwen/Modelfile), we can use it directly.
 
    Note:
 
@@ -725,7 +725,7 @@ Let's take [deepseek-ai/DeepSeek-R1-Distill-Qwen-7B](https://hf-mirror.com/deeps
       
 4. Unzip OpenVINO GenAI package and set environment
    ```shell
-   cd openvino_genai_windows_2025.2.0.0.dev20250320_x86_64
+   cd openvino_genai_windows_2025.2.0.0.dev20250513_x86_64
    setupvars.bat
    ```
 
@@ -792,7 +792,8 @@ Then build and run Ollama from the root directory of the repository:
 1. clone repo
    ```shell
    git lfs install
-   git lfs clone https://github.com/zhaohb/ollama_ov.git
+   git clone https://github.com/openvinotoolkit/openvino_contrib.git
+   cd openvino_contrib/modules/ollama_openvino
    ```
 
 2. Enable CGO:
@@ -802,9 +803,9 @@ Then build and run Ollama from the root directory of the repository:
 
 3. Initialize the GenAI environment
 
-   Download GenAI runtime from [GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_windows_2025.2.0.0.dev20250320_x86_64.zip), then extract it to a directory openvino_genai_windows_2025.2.0.0.dev20250320_x86_64 .
+   Download GenAI runtime from [GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_windows_2025.2.0.0.dev20250513_x86_64.zip), then extract it to a directory openvino_genai_windows_2025.2.0.0.dev20250513_x86_64.
    ```shell
-   cd openvino_genai_windows_2025.2.0.0.dev20250320_x86_64
+   cd openvino_genai_windows_2025.2.0.0.dev20250513_x86_64
    setupvars.bat
    ```
   
@@ -819,7 +820,7 @@ Then build and run Ollama from the root directory of the repository:
    go build -o ollama.exe
    ```
 
-6. If you don't want to recompile ollama, you can choose to directly use the compiled executable file, and then initialize the genai environment in `step 3` to run ollama directly [ollama](https://drive.google.com/file/d/1iizO9iLhSJGFUu6BgY3EwOchrCyzImUN/view?usp=drive_link).
+6. If you don't want to recompile ollama, you can choose to directly use the compiled executable file, and then initialize the genai environment in `step 3` to run ollama directly.
    
    But if you encounter the error when executing ollama.exe, it is recommended that you recompile from source code.
    ```shell
@@ -830,7 +831,8 @@ Then build and run Ollama from the root directory of the repository:
 1. clone repo
    ```shell
    git lfs install
-   git lfs clone https://github.com/zhaohb/ollama_ov.git
+   git clone https://github.com/openvinotoolkit/openvino_contrib.git
+   cd openvino_contrib/modules/ollama_openvino
    ```
 
 2. Enable CGO:
@@ -840,9 +842,9 @@ Then build and run Ollama from the root directory of the repository:
 
 3. Initialize the GenAI environment
 
-   Download GenAI runtime from [GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250320/openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.tar.gz), then extract it to a directory openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64.
+   Download GenAI runtime from [GenAI](https://storage.openvinotoolkit.org/repositories/openvino_genai/packages/nightly/2025.2.0.0.dev20250513/openvino_genai_ubuntu22_2025.2.0.0.dev20250513_x86_64.tar.gz), then extract it to a directory openvino_genai_ubuntu22_2025.2.0.0.dev20250513_x86_64.
    ```shell
-   cd openvino_genai_ubuntu22_2025.2.0.0.dev20250320_x86_64 
+   cd openvino_genai_ubuntu22_2025.2.0.0.dev20250513_x86_64
    source setupvars.sh
    ```
   
diff --git a/modules/ollama_openvino/genai/genai.go b/modules/ollama_openvino/genai/genai.go
index a4bb9df78b..a5ae079c96 100644
--- a/modules/ollama_openvino/genai/genai.go
+++ b/modules/ollama_openvino/genai/genai.go
@@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
+																  const char* device,
+                                                                  ov_genai_llm_pipeline** pipe) {
+	return 	ov_genai_llm_pipeline_create(models_path, "NPU", 4, pipe, "MAX_PROMPT_LEN", "2048", "MIN_RESPONSE_LEN", "256");
+}
+
+static ov_status_e ov_genai_llm_pipeline_create_cgo(const char* models_path,
+																  const char* device,
+                                                                  ov_genai_llm_pipeline** pipe) {
+	return 	ov_genai_llm_pipeline_create(models_path, device, 0, pipe);
+}
+
 */
 import "C"
 
@@ -111,7 +124,12 @@ func CreatePipeline(modelsPath string, device string) *C.ov_genai_llm_pipeline {
 	defer C.free(unsafe.Pointer(cModelsPath))
 	defer C.free(unsafe.Pointer(cDevice))
 
-	C.ov_genai_llm_pipeline_create(cModelsPath, cDevice, &pipeline)
+	// C.ov_genai_llm_pipeline_create(cModelsPath, cDevice, &pipeline)
+	if device == "NPU" {
+		C.ov_genai_llm_pipeline_create_npu_output_2048(cModelsPath, cDevice, &pipeline)
+	} else {
+		C.ov_genai_llm_pipeline_create_cgo(cModelsPath, cDevice, &pipeline)
+	}
 	return pipeline
 }
 
diff --git a/modules/ollama_openvino/llm/genai/genaiserver.go b/modules/ollama_openvino/llm/genai/genaiserver.go
index 05b373d10b..0ab133c296 100644
--- a/modules/ollama_openvino/llm/genai/genaiserver.go
+++ b/modules/ollama_openvino/llm/genai/genaiserver.go
@@ -108,6 +108,36 @@ func SelectDevice(device string, supportedDevices []string) string {
 	return device
 }
 
+func addIndexToDuplicates(input []string) []string {
+	// output := make([]string, 0, len(input))
+	var output []string
+	counters := make(map[string]int)    // Used to record the occurrence count of each value
+	duplicates := make(map[string]bool) // Used to mark which values are duplicates
+
+	// First pass: Count the occurrences of each value and mark duplicates
+	for _, item := range input {
+		counters[item]++
+		if counters[item] > 1 {
+			duplicates[item] = true
+		}
+	}
+
+	// Second pass: Add an index to duplicate values
+	for _, item := range input {
+		if duplicates[item] { // If it's a duplicate
+			output = append(output, fmt.Sprintf("%s:%d", item, counters[item]-1))
+			counters[item]-- // Update the counter
+		} else { // If it's not a duplicate
+			output = append(output, item)
+		}
+	}
+	if ContainsInSlice(input, "GPU") {
+		output = append(output, "GPU")
+	}
+
+	return output
+}
+
 // NewGenaiServer will run a server
 func NewGenaiServer(gpus discover.GpuInfoList, model string, modelname string, inferdevice string, f *ggml.GGML, adapters, projectors []string, opts api.Options, numParallel int) (GenaiServer, error) {
 	systemInfo := discover.GetSystemInfo()
@@ -121,6 +151,7 @@ func NewGenaiServer(gpus discover.GpuInfoList, model string, modelname string, i
 	for i := 0; i < int(len(genai_device)); i++ {
 		genai_device_list = append(genai_device_list, genai_device[i]["device_name"])
 	}
+	genai_device_list = addIndexToDuplicates(genai_device_list)
 	inferdevice = SelectDevice(inferdevice, genai_device_list)
 
 	params := []string{


Reviews:
user: login: gblong1
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T14:51:42Z
user: login: zhaohb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T01:43:53Z
user: login: zhaohb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T01:51:01Z
user: login: gblong1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-15T22:59:37Z
user: login: zhaohb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-16T02:01:51Z
user: login: gblong1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-19T22:55:35Z
user: login: zhaohb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T02:00:58Z
user: login: zhaohb
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T14:19:28Z
user: login: gblong1
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T15:18:34Z
user: login: gblong1
body: Approving with the expectation that the max length will be addressed as soon as possible. 
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-20T15:19:35Z
user: login: FionaZZ92
state: APPROVED
author_association: NONE
submitted_at: 2025-05-20T15:24:42Z

Review comments:
diff_hunk: @@ -108,6 +108,36 @@ func SelectDevice(device string, supportedDevices []string) string {
 	return device
 }
 
+func addIndexToDuplicates(input []string) []string {
+	// output := make([]string, 0, len(input))
+	var output []string
+	counters := make(map[string]int)    // 用于记录每个值的出现次数
path: modules/ollama_openvino/llm/genai/genaiserver.go
user: login: gblong1
body: Please update comments to be in English language. 
created_at: 2025-05-14T14:44:40Z
author_association: CONTRIBUTOR
diff_hunk: @@ -121,6 +151,7 @@ func NewGenaiServer(gpus discover.GpuInfoList, model string, modelname string, i
 	for i := 0; i < int(len(genai_device)); i++ {
 		genai_device_list = append(genai_device_list, genai_device[i]["device_name"])
 	}
+	genai_device_list = addIndexToDuplicates(genai_device_list)
path: modules/ollama_openvino/llm/genai/genaiserver.go
user: login: gblong1
body: Why is this function needed? It seems like GetGenaiAvailableDevices is not returning the correct list? 
created_at: 2025-05-14T14:48:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: gblong1
body: This should not be hardcoded to 2048. It should updated to take a variable max_prompt_len and min_response_len so that larger range of sizes can be supported. 
created_at: 2025-05-14T14:50:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: zhaohb
body: Hi @gblong1 
To my knowledge, the default output length for the NPU is 1024, which is not dynamically adjustable. This can be referenced in the following link:
[genai npu default output len](https://github.com/openvinotoolkit/openvino.genai/blob/master/src/cpp/src/utils.cpp#L477)
The decision to set the default to 2048 was made because during testing, it was observed that some responses were being truncated, which is not the intended behavior. This adjustment ensures that responses remain complete and aligned with expectations.

thank you.
created_at: 2025-05-15T01:43:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -121,6 +151,7 @@ func NewGenaiServer(gpus discover.GpuInfoList, model string, modelname string, i
 	for i := 0; i < int(len(genai_device)); i++ {
 		genai_device_list = append(genai_device_list, genai_device[i]["device_name"])
 	}
+	genai_device_list = addIndexToDuplicates(genai_device_list)
path: modules/ollama_openvino/llm/genai/genaiserver.go
user: login: zhaohb
body: The GetGenaiAvailableDevices function returns an array like [CPU, GPU, GPU, GPU, NPU] in scenarios with multiple GPU devices. However, this format cannot be used to validate whether the user's input for inferdevice is valid, as GenAI, in the case of multiple GPUs, only accepts inputs in the format GPU:0, GPU:1, GPU:2, etc.
To address this, we add a new API that generates all valid device names:
[CPU, GPU, GPU, GPU, NPU] → [CPU, GPU:0, GPU:1, GPU:2, NPU, GPU]
This ensures compatibility with GenAI's expected input format and simplifies validation of user-provided device names.
Thank you very much.
created_at: 2025-05-15T01:51:01Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: gblong1
body: I agree that it should be made larger, however, it should not be hard coded, it should be configurable so that it doesn't have to be re-hard coded to a bigger number later, and so that some models which need even larger than 2048 (like deepseek) will also work well. 
created_at: 2025-05-15T22:59:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: zhaohb
body: Yes, I agree that having a dynamically adjustable output length would be ideal. However, the behavior of the NPU differs from that of the CPU/GPU. While the output length for the CPU/GPU can be set during generation, the NPU requires the output length to be specified at the time of model loading. Once the model is loaded, the output length cannot be modified without reloading the model.
At this stage, making the NPU's output length dynamically adjustable would necessitate reloading the model and involve significant changes. Therefore, I believe setting the output length to 2048 is a practical solution that should meet most common use cases.
created_at: 2025-05-16T02:01:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: gblong1
body: Hi @zhaohb - What I mean is that the length should be configurable so that it can be set at the time of model loading. if you see https://github.com/openvinotoolkit/openvino.genai/blob/master/src/c/src/llm_pipeline.cpp#L71 you will see that the device properties, like MAX_PROMPT_LEN, can be set at pipeline creation time rather than hard-coded to 2048. Even with models like DeepSeek, we have seen that larger than 2048 is necessary. The real fix here is to create the API such that the properties are passed as parameters and not hard coded. 
created_at: 2025-05-19T22:55:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: zhaohb
body: Yes, I understand your point. However, in Ollama's architecture, when loading the model (i.e., when calling create_pipeline), the parameter for output length is not passed. The generation length parameter is only passed during the generation phase. This is related to Ollama's implementation, which assumes that the output length can be set during the generation phase in all cases. However, our NPU currently does not meet this requirement.
Therefore, I mentioned that implementing dynamic output length setting in ollama_openvino for NPU would require significant effort. If the NPU could support setting the generation length during the generation phase, Ollama's current logic would be compatible.
The key question: When will the NPU support dynamic output length configuration during the generation phase?
created_at: 2025-05-20T02:00:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: zhaohb
body: Hi @gblong1 ,
I would like this PR to be merged as soon as possible, as it includes an update to the executable file's download URL. We are already receiving feedback from customers who are unable to download.
Regarding the output length being set to 2048, this can be addressed in a subsequent PR, as it would require modifications to the Ollama implementation.
created_at: 2025-05-20T14:19:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -22,6 +22,19 @@ package genai
 typedef int (*callback_function)(const char*, void*);
 
 extern int goCallbackBridge(char* input, void* ptr);
+
+static ov_status_e ov_genai_llm_pipeline_create_npu_output_2048(const char* models_path,
path: modules/ollama_openvino/genai/genai.go
user: login: gblong1
body: I will approve with the understanding that the hardcoded length will addressed in a future PR. Please address this post merge as soon as possible. 
created_at: 2025-05-20T15:18:34Z
author_association: CONTRIBUTOR

PR comments:
user: login: FionaZZ92
created_at: 2025-05-21T06:42:36Z
body: @andrei-kochin Shall we merge this PR for Ollama_OV? Checked CI blocked on Nv plugin w/ Ubuntu20.04 which not relevant to Ollama module and I think someone can help to remove CI testing of Nv plugin w/ 20.04?
author_association: NONE
user: login: sammysun0711
created_at: 2025-05-21T07:06:34Z
body: @andrei-kochin @akashchi I fixed CI ubuntu runner: https://github.com/openvinotoolkit/openvino_contrib/pull/970, could you please first merge https://github.com/openvinotoolkit/openvino_contrib/pull/970, then we can rebase on master, thanks!

author_association: NONE