Repository: ollama/ollama
Pull request: build: fix arm container image

Fix a rebase glitch from the old C++ runner build model

Verified by inspection after building locally: (confirming the jetpack specific cuda libs are present, as well as the runners)
```
% docker run --rm -it --entrypoint bash dhiltgen/ollama:0.4.2-rc0-0-g4efb98c-dirty
root@1cc199483147:/# find lib/ollama/
lib/ollama/
lib/ollama/runners
lib/ollama/runners/cuda_v11
lib/ollama/runners/cuda_v11/ollama_llama_server
lib/ollama/runners/cuda_v12
lib/ollama/runners/cuda_v12/ollama_llama_server
lib/ollama/runners/cpu
lib/ollama/runners/cpu/ollama_llama_server
lib/ollama/runners/cuda_jetpack6
lib/ollama/runners/cuda_jetpack6/ollama_llama_server
lib/ollama/runners/cuda_jetpack5
lib/ollama/runners/cuda_jetpack5/ollama_llama_server
lib/ollama/libcublas.so.12.4.5.8
lib/ollama/libcudart.so.12.4.127
lib/ollama/libcublasLt.so.11
lib/ollama/libcudart.so.11.0
lib/ollama/libcublasLt.so.12.4.5.8
lib/ollama/libggml_cuda_v12.so
lib/ollama/libcudart.so.11.3.109
lib/ollama/libcublasLt.so
lib/ollama/libcublas.so.11.5.1.109
lib/ollama/libggml_cuda_v11.so
lib/ollama/libcudart.so
lib/ollama/libcublas.so
lib/ollama/libcublas.so.12
lib/ollama/libcublasLt.so.12
lib/ollama/libcudart.so.12
lib/ollama/libcublasLt.so.11.5.1.109
lib/ollama/libcublas.so.11
lib/ollama/cuda_jetpack6
lib/ollama/cuda_jetpack6/libcublasLt.so.12.2.5.6
lib/ollama/cuda_jetpack6/libcudart.so.12.2.140
lib/ollama/cuda_jetpack6/libcublasLt.so
lib/ollama/cuda_jetpack6/libcublas.so.12.2.5.6
lib/ollama/cuda_jetpack6/libcudart.so
lib/ollama/cuda_jetpack6/libcublas.so
lib/ollama/cuda_jetpack6/libcublas.so.12
lib/ollama/cuda_jetpack6/libcublasLt.so.12
lib/ollama/cuda_jetpack6/libcudart.so.12
lib/ollama/libggml_cuda_jetpack6.so
lib/ollama/cuda_jetpack5
lib/ollama/cuda_jetpack5/libcublasLt.so.11
lib/ollama/cuda_jetpack5/libcudart.so.11.0
lib/ollama/cuda_jetpack5/libcublas.so.11.6.6.84
lib/ollama/cuda_jetpack5/libcublasLt.so
lib/ollama/cuda_jetpack5/libcudart.so
lib/ollama/cuda_jetpack5/libcublas.so
lib/ollama/cuda_jetpack5/libcublasLt.so.11.6.6.84
lib/ollama/cuda_jetpack5/libcudart.so.11.4.298
lib/ollama/cuda_jetpack5/libcublas.so.11
lib/ollama/libggml_cuda_jetpack5.so
```

Also verified the image on a JetPack 5 Orin:
```
docker run --rm -it --runtime=nvidia -e OLLAMA_DEBUG=1 -e JETSON_JETPACK dhiltgen/ollama:0.4.2-rc0-1-gca3785d-dirty
```

```
# ollama run orca-mini --verbose hello
 Hello there! How can I assist you today?

total duration:       3.786969368s
load duration:        1.771817151s
prompt eval count:    42 token(s)
prompt eval duration: 1.245s
prompt eval rate:     33.73 tokens/s
eval count:           11 token(s)
eval duration:        767ms
eval rate:            14.34 tokens/s
# ollama ps
NAME                ID              SIZE      PROCESSOR    UNTIL
orca-mini:latest    2dbd9f439647    5.9 GB    100% GPU     4 minutes from now
```

Changed files:
filename: Dockerfile
patch: @@ -234,17 +234,13 @@ COPY --from=container-build-amd64 /go/src/github.com/ollama/ollama/dist/linux-am
 COPY --from=runners-cuda-amd64 /go/src/github.com/ollama/ollama/dist/linux-amd64/lib/ /lib/
 
 FROM --platform=linux/arm64 ubuntu:22.04 AS runtime-arm64
-COPY --from=build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64-jetpack5/lib/ /lib/
-COPY --from=build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64-jetpack6/lib/ /lib/
 RUN apt-get update && \
     apt-get install -y ca-certificates && \
     apt-get clean && rm -rf /var/lib/apt/lists/*
 COPY --from=container-build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/bin/ /bin/
-COPY --from=cpu-build-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
-COPY --from=cuda-11-build-runner-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
-COPY --from=cuda-12-build-runner-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
-COPY --from=cuda-build-jetpack5-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
-COPY --from=cuda-build-jetpack6-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
+COPY --from=runners-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64/lib/ /lib/
+COPY --from=runners-jetpack5-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64-jetpack5/lib/ /lib/
+COPY --from=runners-jetpack6-arm64 /go/src/github.com/ollama/ollama/dist/linux-arm64-jetpack6/lib/ /lib/
 
 
 # ROCm libraries larger so we keep it distinct from the CPU/CUDA image

filename: docs/docker.md
patch: @@ -50,6 +50,9 @@ sudo systemctl restart docker
 docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
 ```
 
+> [!NOTE]  
+> If you're running on an NVIDIA JetPack system, Ollama can't automatically discover the correct JetPack version. Pass the environment variable JETSON_JETPACK=5 or JETSON_JETPACK=6 to the container to select version 5 or 6.
+
 ### AMD GPU
 
 To run Ollama using Docker with AMD GPUs, use the `rocm` tag and the following command: