Repository: ollama
Issue: Ollama ps says 22 GB, but nvidia-smi says 16GB with flash attention enabled
Labels: bug,memory

### What is the issue?

Ollama indicates the model is utilizing 22GB, but nvidia-smi says it's utilizing 16GB.
The model was fully loaded and generating responses when I ran nvidia-smi.

Here's the log:
```
time=2024-08-04T12:51:05.930Z level=INFO source=server.go:384 msg="starting llama server" cmd="/tmp/ollama1680167563/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-d36aafdc1d822f932f3fd3ddc18296628764c5e43f153e9c02b29f5c4525cf2a --ctx-size 65536 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --flash-attn --parallel 32 --port 34551"

ollama ps                                                                                                                                                                                                               
 NAME                           ID           SIZE  PROCESSOR UNTIL                                                                                                                                                                            
 llama3.1:8b-instruct-q8_0      9b90f0f552e7 22 GB 100% GPU  27 seconds from now                                                                                                                                                              
 root@272875ddc015:~# nvidia-smi                                                                                                                                                                                                              
 Sun Aug  4 12:57:31 2024                                                                                                                                                                                                                     
 +-----------------------------------------------------------------------------------------+                                                                                                                                                  
 | NVIDIA-SMI 555.42.02              Driver Version: 555.42.02      CUDA Version: 12.5     |                                                                                                                                                  
 |-----------------------------------------+------------------------+----------------------+                                                                                                                                                  
 | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |                                                                                                                                                  
 | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |                                                                                                                                                  
 |                                         |                        |               MIG M. |                                                                                                                                                  
 |=========================================+========================+======================|                                                                                                                                                  
 |   0  NVIDIA GeForce RTX 4090        On  |   00000000:41:00.0 Off |                  Off |                                                                                                                                                  
 | 74%   68C    P0            353W /  450W |   16560MiB /  24564MiB |    100%      Default |                                                                                                                                                  
 |                                         |                        |                  N/A |                                                                                                                                                  
 +-----------------------------------------+------------------------+----------------------+                                                                                                                                                  
                                                                                                                                                                                                                                              
 +-----------------------------------------------------------------------------------------+                                                                                                                                                  
 | Processes:                                                                              |                                                                                                                                                  
 |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |                                                                                                                                                  
 |        ID   ID                                                               Usage      |                                                                                                                                                  
 |=========================================================================================|                                                                                                                                                  
 +-----------------------------------------------------------------------------------------+                                                                                                                                                  
```

### OS

Linux

### GPU

Nvidia

### CPU

Intel

### Ollama version

0.3.3