Repository: ollama
Issue: No proper response when IPEX-LLM setup with Ollama for intel cpu/gpu
Labels: bug

### What is the issue?

After setup of IPEX-LLM to work with ollama I see that output is wrong.

Example:
```
❯ ./ollama run phi3
>>> hi
































```

Maybe I had setup something wrong...

When I serve ollama I have such logs:

```❯ ./ollama serve
2024/06/07 01:00:48 routes.go:1028: INFO server config env="map[OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST: OLLAMA_KEEP_ALIVE: OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS: OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]"
time=2024-06-07T01:00:48.471+02:00 level=INFO source=images.go:729 msg="total blobs: 38"
time=2024-06-07T01:00:48.471+02:00 level=INFO source=images.go:736 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /api/pull                 --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] GET    /api/ps                   --> github.com/ollama/ollama/server.(*Server).ProcessHandler-fm (5 handlers)
[GIN-debug] POST   /v1/chat/completions      --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
[GIN-debug] GET    /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
time=2024-06-07T01:00:48.471+02:00 level=INFO source=routes.go:1074 msg="Listening on 127.0.0.1:11434 (version 0.0.0)"
time=2024-06-07T01:00:48.472+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama607982765/runners
time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz
time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz
time=2024-06-07T01:00:48.472+02:00 level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu_avx
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama607982765/runners/cpu_avx2
time=2024-06-07T01:00:48.523+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2]"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=sched.go:90 msg="starting llm scheduler"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:122 msg="Detecting GPUs"
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcuda.so*
time=2024-06-07T01:00:48.523+02:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/opt/intel/oneapi/mkl/2024.0/lib/libcuda.so** /opt/intel/oneapi/compiler/2024.0/lib/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[]
time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:261 msg="Searching for GPU library" name=libcudart.so*
time=2024-06-07T01:00:50.186+02:00 level=DEBUG source=gpu.go:280 msg="gpu library search" globs="[/opt/intel/oneapi/mkl/2024.0/lib/libcudart.so** /opt/intel/oneapi/compiler/2024.0/lib/libcudart.so** /tmp/ollama607982765/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=gpu.go:313 msg="discovered GPU libraries" paths=[/usr/local/cuda/lib64/libcudart.so.12.5.39]
cudaSetDevice err: 35
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=gpu.go:325 msg="Unable to load cudart" library=/usr/local/cuda/lib64/libcudart.so.12.5.39 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-06-07T01:00:51.690+02:00 level=DEBUG source=amd_linux.go:322 msg="amdgpu driver not detected /sys/module/amdgpu"
time=2024-06-07T01:00:51.690+02:00 level=INFO source=types.go:71 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="7.6 GiB" available="3.6 GiB"```


### OS

WSL2

### GPU

Intel

### CPU

Intel

### Ollama version

0.1.39