ramalama 0.17.1
RamaLama is a command line tool for working with AI LLM models.
License: MIT
Requires Python: >=3.10
Keywords: ramalama, llama, ai, llm, large-language-model, inference, containers, podman, docker, gpu, rag

<p align="center">
<img src="https://github.com/user-attachments/assets/1a338ecf-dc84-4495-8c70-16882955da47" width=50%>
</p>

<p align="center">
  <a href="https://github.com/containers/ramalama/issues">
    <img src="https://img.shields.io/github/issues/containers/ramalama?style=flat-square" height="22" alt="Open Issues"/>
  </a>
    <a href="https://github.com/containers/ramalama/blob/main/LICENSE"><img src="https://img.shields.io/github/license/containers/ramalama?style=flat-square" height="22" alt="License"/>
  </a>
  <a href="https://github.com/containers/ramalama">
    <img src="https://img.shields.io/github/languages/top/containers/ramalama?style=flat-square" height="22" alt="Top language"/>
  </a>
  <a href="https://pypi.org/project/ramalama/">
    <img src="https://img.shields.io/pypi/v/ramalama?style=flat-square" height="22" alt="PyPI version"/>
  </a>
  <a href="https://pypi.org/project/ramalama/">
    <img src="https://img.shields.io/pypi/pyversions/ramalama?style=flat-square" height="22" alt="Supported Python versions"/>
  </a>
  <a href="https://discord.gg/MkCXuTRBUn">
    <img src="https://img.shields.io/discord/1388199960503128165?style=flat-square&logo=discord&label=discord" height="22" alt="Join Discord"/>
  </a>
  <a href="https://matrix.to/#/#ramalama:fedoraproject.org">
    <img src="https://img.shields.io/badge/matrix-join-blue?style=flat-square&logo=matrix" height="22" alt="Join Matrix"/>
  </a>
</p>


<!-- <h3 align="center">
  <a href="https://docs.ramalama.com/">Docs</a> &bull;
  <a href="https://github.com/containers/ramalama/discussions" title="Community Page">Discussion</a> &bull;
  <a href="https://blog.ramalama.com" title="RamaLama Blog">Blog</a>
</h3> -->



[RamaLama](https://ramalama.ai) strives to make working with AI simple, straightforward, and familiar by using OCI containers.
<br>
<br>

## Description
RamaLama is an open-source tool that simplifies the local use and serving of AI models for inference from any source through the familiar approach of containers. It allows engineers to use container-centric development patterns and benefits to extend to AI use cases.

RamaLama eliminates the need to configure the host system by instead pulling a container image specific to the GPUs discovered on the host system, and allowing you to work with various models and platforms.

- Eliminates the complexity for users to configure the host system for AI.
- Detects and pulls an [accelerated container image](#accelerated-images) specific to the GPUs on the host system, handling dependencies and hardware optimization.
- RamaLama supports multiple [AI model registries](#transports), including OCI Container Registries.
- Models are treated similarly to how Podman and Docker treat container images.
- Use common [container commands](#commands) to work with AI models.
- Run AI models [securely](#security) in rootless containers, isolating the model from the underlying host.
- Keep data secure by defaulting to no network access and removing all temporary data on application exits.
- Interact with models via REST API or as a chatbot.
<br>

## Install
### Install on macOS (Self-Contained Installer)
Download the self-contained macOS installer that includes Python and all dependencies:

1. Download the latest `.pkg` installer from [Releases](https://github.com/containers/ramalama/releases)
2. Double-click to install, or run: `sudo installer -pkg RamaLama-*-macOS-Installer.pkg -target /`

See [macOS Installation Guide](docs/MACOS_INSTALL.md) for detailed instructions.

### Install on Fedora
RamaLama is available in [Fedora](https://fedoraproject.org/) and later. To install it, run:
```
sudo dnf install ramalama
```

### Install via PyPI
RamaLama is available via PyPI at [https://pypi.org/project/ramalama](https://pypi.org/project/ramalama)
```
pip install ramalama
```

### Install script (Linux and macOS)
Install RamaLama by running:
```
curl -fsSL https://ramalama.ai/install.sh | bash
```

### Install on Windows
RamaLama supports Windows with Docker Desktop or Podman Desktop:
```powershell
pip install ramalama
```

**Requirements:**
- Python 3.10 or later
- Docker Desktop or Podman Desktop with WSL2 backend
- For GPU support, see [NVIDIA GPU Setup for WSL2](docs/readme/wsl2-docker-cuda.md)

**Note:** Windows support requires running containers via Docker/Podman. The model store uses hardlinks (no admin required) or falls back to file copies if hardlinks are unavailable.

## Uninstall

### Uninstall via pip
If you installed RamaLama using pip, you can uninstall it with:
```bash
pip uninstall ramalama
```

### Uninstall on Fedora
If you installed RamaLama using DNF:
```bash
sudo dnf remove ramalama
```

### Uninstall on macOS (Self-Contained Installer)
To remove RamaLama installed via the `.pkg` installer:
```bash
# Remove the executable
sudo rm /usr/local/bin/ramalama

# Remove configuration and data files
sudo rm -rf /usr/local/share/ramalama

# Remove man pages (optional)
sudo rm /usr/local/share/man/man1/ramalama*.1
sudo rm /usr/local/share/man/man5/ramalama*.5
sudo rm /usr/local/share/man/man7/ramalama*.7

# Remove shell completions (optional)
sudo rm /usr/local/share/bash-completion/completions/ramalama
sudo rm /usr/local/share/fish/vendor_completions.d/ramalama.fish
sudo rm /usr/local/share/zsh/site-functions/_ramalama
```

See the [macOS Installation Guide](docs/MACOS_INSTALL.md) for more details.

### Remove User Data and Configuration
After uninstalling RamaLama using any method above, you may want to remove downloaded models and configuration files from your home directory:

```bash
# Remove downloaded models and data (can be large)
rm -rf ~/.local/share/ramalama

# Remove configuration files
rm -rf ~/.config/ramalama

# If you ran RamaLama as root, also remove:
sudo rm -rf /var/lib/ramalama
```

**Note:** The model data directory (`~/.local/share/ramalama`) can be quite large depending on how many models you've downloaded. Make sure you want to remove these files before running the commands above.

## Accelerated images

| Accelerator                       | Image                      |
| :---------------------------------| :------------------------- |
|  GGML_VK_VISIBLE_DEVICES (or CPU) | quay.io/ramalama/ramalama  |
|  HIP_VISIBLE_DEVICES              | quay.io/ramalama/rocm      |
|  CUDA_VISIBLE_DEVICES             | quay.io/ramalama/cuda      |
|  ASAHI_VISIBLE_DEVICES            | quay.io/ramalama/asahi     |
|  INTEL_VISIBLE_DEVICES            | quay.io/ramalama/intel-gpu |
|  ASCEND_VISIBLE_DEVICES           | quay.io/ramalama/cann      |
|  MUSA_VISIBLE_DEVICES             | quay.io/ramalama/musa      |

### GPU support inspection
On first run, RamaLama inspects your system for GPU support, falling back to CPU if none are present. RamaLama uses container engines like Podman or Docker to pull the appropriate OCI image with all necessary software to run an AI Model for your system setup.

<details>
<summary>
How does RamaLama select the right image?
</summary>
<br>

After initialization, RamaLama runs AI Models within a container based on the OCI image. RamaLama pulls container images specific to the GPUs discovered on your system. These images are tied to the minor version of RamaLama.
- For example, RamaLama version 1.2.3 on an NVIDIA system pulls quay.io/ramalama/cuda:1.2. To override the default image, use the `--image` option.

RamaLama then pulls AI Models from model registries, starting a chatbot or REST API service from a simple single command. Models are treated similarly to how Podman and Docker treat container images.
</details>
<br>

## Hardware Support

| Hardware                           | Enabled                     |
| :--------------------------------- | :-------------------------: |
| CPU                                | &check;                     |
| Apple Silicon GPU (Linux / Asahi)  | &check;                     |
| Apple Silicon GPU (macOS)          | &check; llama.cpp or MLX    |
| Apple Silicon GPU (podman-machine) | &check;                     |
| Nvidia GPU (cuda)                  | &check; See note below      |
| AMD GPU (rocm, vulkan)             | &check;                     |
| Ascend NPU (Linux)                 | &check;                     |
| Intel ARC GPUs (Linux)             | &check; See note below      |
| Intel GPUs (vulkan / Linux)        | &check;                     |
| Moore Threads GPU (musa / Linux)   | &check; See note below      |
| Windows (with Docker/Podman)       | &check; Requires WSL2       |

### Nvidia GPUs
On systems with NVIDIA GPUs, see [ramalama-cuda](docs/ramalama-cuda.7.md) documentation for the correct host system configuration.

### Intel GPUs
The following Intel GPUs are auto-detected by RamaLama:

| GPU ID  | Description                        |
| :------ | :--------------------------------- |
|`0xe20b` | Intel® Arc™ B580 Graphics          |
|`0xe20c` | Intel® Arc™ B570 Graphics          |
|`0x7d51` | Intel® Graphics - Arrow Lake-H     |
|`0x7dd5` | Intel® Graphics - Meteor Lake      |
|`0x7d55` | Intel® Arc™ Graphics - Meteor Lake |

See the [Intel hardware table](https://dgpu-docs.intel.com/devices/hardware-table.html) for more information.
<br>
<br>

### Moore Threads GPUs
On systems with Moore Threads GPUs, see [ramalama-musa](docs/ramalama-musa.7.md) documentation for the correct host system configuration.

### MLX Runtime (macOS only)
The MLX runtime provides optimized inference for Apple Silicon Macs. MLX requires:
- macOS operating system
- Apple Silicon hardware (M1, M2, M3, or later)
- Usage with `--nocontainer` option (containers are not supported)
- The `mlx-lm` uv package installed on the host system as a uv tool

To install and run Phi-4 on MLX, use `uv`. If `uv` is not installed, you can install it with `curl -LsSf https://astral.sh/uv/install.sh | sh`:
```bash
uv tool install mlx-lm
# or upgrade to the latest version:
uv tool upgrade mlx-lm

ramalama --runtime=mlx serve hf://mlx-community/Unsloth-Phi-4-4bit
```

#### Default Container Engine
When both Podman and Docker are installed, RamaLama defaults to Podman. The `RAMALAMA_CONTAINER_ENGINE=docker` environment variable can override this behaviour. When neither are installed, RamaLama will attempt to run the model with software on the local system.
<br>
<br>

## Security

### Test and run your models more securely
Because RamaLama defaults to running AI models inside rootless containers using Podman or Docker, these containers isolate the AI models from information on the underlying host.  With RamaLama containers, the AI model is mounted as a volume into the container in read-only mode.

This results in the process running the model (llama.cpp or vLLM) being isolated from the host. Additionally, since `ramalama run` uses the `--network=none` option, the container cannot reach the network and leak any information out of the system. Finally, containers are run with the `--rm` option, which means any content written during container execution is deleted when the application exits.

### Here’s how RamaLama delivers a robust security footprint:
- **Container Isolation** – AI models run within isolated containers, preventing direct access to the host system.
- **Read-Only Volume Mounts** – The AI model is mounted in read-only mode, which means that processes inside the container cannot modify the host files.
- **No Network Access** – ramalama run is executed with `--network=none`, meaning the model has no outbound connectivity for which information can be leaked.
- **Auto-Cleanup** – Containers run with `--rm`, wiping out any temporary data once the session ends.
- **Drop All Linux Capabilities** – No access to Linux capabilities to attack the underlying host.
- **No New Privileges** – Linux Kernel feature that disables container processes from gaining additional privileges.
<br>


## Transports
RamaLama supports multiple AI model registries types called transports.

### Supported transports

| Transports               |  Web Site                                            |
| :-------------           | :--------------------------------------------------- |
| HuggingFace              | [`huggingface.co`](https://www.huggingface.co)       |
| ModelScope               | [`modelscope.cn`](https://www.modelscope.cn)         |
| Ollama                   | [`ollama.com`](https://www.ollama.com)               |
| RamaLama Labs Container Registry | [`ramalama.com`](https://registry.ramalama.com) |
| OCI Container Registries | [`opencontainers.org`](https://opencontainers.org)   |
|                          |Examples: [`quay.io`](https://quay.io),  [`Docker Hub`](https://docker.io), [`Pulp`](https://pulpproject.org), and [`Artifactory`](https://jfrog.com/artifactory/)|

### Default Transport
RamaLama uses the Ollama registry transport by default

<details>
<summary>
How to change transports.
</summary>
<br>

Use the RAMALAMA_TRANSPORT environment variable to modify the default. `export RAMALAMA_TRANSPORT=huggingface` Changes RamaLama to use huggingface transport.

Individual model transports can be modified when specifying a model via the `huggingface://`, `oci://`, `modelscope://`, `ollama://`, or `rlcr://` prefix.

Example:
```
ramalama pull huggingface://afrideva/Tiny-Vicuna-1B-GGUF/tiny-vicuna-1b.q2_k.gguf
```
</details>

### Transport shortnames
To make it easier for users, RamaLama uses shortname files, which contain alias names for fully specified AI Models, allowing users to refer to models using shorter names.

<details>
<summary>
More information on shortnames.
</summary>
<br>

RamaLama reads shortnames.conf files if they exist. These files contain a list of name-value pairs that specify the model. The following table specifies the order in which RamaLama reads the files. Any duplicate names that exist override previously defined shortnames.
<br>
| Shortnames type    | Path                                                       |
| :----------------  | :-------------------------------------------               |
| Development        | ./shortnames.conf                                          |
| User (Config)      | $HOME/.config/ramalama/shortnames.conf                     |
| User (Local Share) | $HOME/.local/share/ramalama/shortnames.conf                |
| Administrators     | /etc/ramalama/shortnames.conf                              |
| Distribution       | /usr/share/ramalama/shortnames.conf                        |
| Local Distribution | /usr/local/share/ramalama/shortnames.conf                  |
<br>

```
$ cat /usr/share/ramalama/shortnames.conf
[shortnames]
  "tiny" = "ollama://tinyllama"
  "granite" = "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf"
  "granite:7b" = "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf"
  "ibm/granite" = "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf"
  "merlinite" = "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf"
  "merlinite:7b" = "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf"
  ...
```
</details>
<br>
<br>


## Commands

### [`ramalama-bench`](https://github.com/containers/ramalama/blob/main/docs/ramalama-bench.1.md)
#### Benchmark specified AI Model.
- <details>
    <summary>
        Benchmark specified AI Model
    </summary>
    <br>

    ```
    $ ramalama bench granite3-moe
    ```
</details>

### [`ramalama-containers`](https://github.com/containers/ramalama/blob/main/docs/ramalama-containers.1.md)
#### List all RamaLama containers.
- <details>
    <summary>
        List all containers running AI Models
    </summary>
    <br>

    ```
    $ ramalama containers
    ```
    Returns for example:
    ```
    CONTAINER ID  IMAGE                             COMMAND               CREATED        STATUS                    PORTS                   NAMES
    85ad75ecf866  quay.io/ramalama/ramalama:latest  /usr/bin/ramalama...  5 hours ago    Up 5 hours                0.0.0.0:8080->8080/tcp  ramalama_s3Oh6oDfOP
    85ad75ecf866  quay.io/ramalama/ramalama:latest  /usr/bin/ramalama...  4 minutes ago  Exited (0) 4 minutes ago                          granite-server
    ```
</details>

- <details>
    <summary>
        List all containers in a particular format
    </summary>
    <br>

    ```
    $ ramalama ps --noheading --format "{{ .Names }}"
    ```
    Returns for example:

    ```
    ramalama_s3Oh6oDfOP
    granite-server
    ```
</details>

### [`ramalama-convert`](https://github.com/containers/ramalama/blob/main/docs/ramalama-convert.1.md)
#### Convert AI Model from local storage to OCI Image.
- <details>
    <summary>
        Generate an oci model out of an Ollama model.
    </summary>
    <br>

    ```
    $ ramalama convert ollama://tinyllama:latest oci://quay.io/rhatdan/tiny:latest
    ```
    Returns for example:
    ```
    Building quay.io/rhatdan/tiny:latest...
    STEP 1/2: FROM scratch
    STEP 2/2: COPY sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 /model
    --> Using cache 69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
    COMMIT quay.io/rhatdan/tiny:latest
    --> 69db4a10191c
    Successfully tagged quay.io/rhatdan/tiny:latest
    69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
    ```
</details>

- <details>
    <summary>
        Generate and run an OCI model with a quantized GGUF converted from Safetensors.
    </summary>
    <br>

    Generate OCI model
    ```
    $ ramalama --image quay.io/ramalama/ramalama-rag convert --gguf Q4_K_M hf://ibm-granite/granite-3.2-2b-instruct oci://quay.io/kugupta/granite-3.2-q4-k-m:latest
    ```

    Returns for example:
    ```
    Converting /Users/kugupta/.local/share/ramalama/models/huggingface/ibm-granite/granite-3.2-2b-instruct to quay.io/kugupta/granite-3.2-q4-k-m:latest...
    Building quay.io/kugupta/granite-3.2-q4-k-m:latest...
    ```

    Run the generated model
    ```
    $ ramalama run oci://quay.io/kugupta/granite-3.2-q4-k-m:latest
    ```
</details>

### [`ramalama-info`](https://github.com/containers/ramalama/blob/main/docs/ramalama-info.1.md)
#### Display RamaLama configuration information.
- <details>
    <summary>
        Info with no container engine.
    </summary>
    <br>

    ```
    $ ramalama info
    ```
    Returns for example:
    ```
    {
        "Accelerator": "cuda",
        "Engine": {
        "Name": ""
        },
        "Image": "quay.io/ramalama/cuda:0.7",
        "Inference": {
            "Default": "llama.cpp",
            "Engines": {
                "llama.cpp": "/usr/share/ramalama/inference-spec/engines/llama.cpp.yaml",
                "mlx": "/usr/share/ramalama/inference-spec/engines/mlx.yaml",
                "vllm": "/usr/share/ramalama/inference-spec/engines/vllm.yaml"
            },
            "Schema": {
                "1-0-0": "/usr/share/ramalama/inference-spec/schema/schema.1-0-0.json"
            }
        },
        "Shortnames": {
        "Names": {
            "cerebrum": "huggingface://froggeric/Cerebrum-1.0-7b-GGUF/Cerebrum-1.0-7b-Q4_KS.gguf",
            "deepseek": "ollama://deepseek-r1",
            "dragon": "huggingface://llmware/dragon-mistral-7b-v0/dragon-mistral-7b-q4_k_m.gguf",
            "gemma3": "hf://bartowski/google_gemma-3-4b-it-GGUF/google_gemma-3-4b-it-IQ2_M.gguf",
            "gemma3:12b": "hf://bartowski/google_gemma-3-12b-it-GGUF/google_gemma-3-12b-it-IQ2_M.gguf",
            "gemma3:1b": "hf://bartowski/google_gemma-3-1b-it-GGUF/google_gemma-3-1b-it-IQ2_M.gguf",
            "gemma3:27b": "hf://bartowski/google_gemma-3-27b-it-GGUF/google_gemma-3-27b-it-IQ2_M.gguf",
            "gemma3:4b": "hf://bartowski/google_gemma-3-4b-it-GGUF/google_gemma-3-4b-it-IQ2_M.gguf",
            "granite": "ollama://granite3.1-dense",
            "granite-code": "hf://ibm-granite/granite-3b-code-base-2k-GGUF/granite-3b-code-base.Q4_K_M.gguf",
            "granite-code:20b": "hf://ibm-granite/granite-20b-code-base-8k-GGUF/granite-20b-code-base.Q4_K_M.gguf",
            "granite-code:34b": "hf://ibm-granite/granite-34b-code-base-8k-GGUF/granite-34b-code-base.Q4_K_M.gguf",
            "granite-code:3b": "hf://ibm-granite/granite-3b-code-base-2k-GGUF/granite-3b-code-base.Q4_K_M.gguf",
            "granite-code:8b": "hf://ibm-granite/granite-8b-code-base-4k-GGUF/granite-8b-code-base.Q4_K_M.gguf",
            "granite-lab-7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "granite-lab-8b": "huggingface://ibm-granite/granite-8b-code-base-GGUF/granite-8b-code-base.Q4_K_M.gguf",
            "granite-lab:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "granite:2b": "ollama://granite3.1-dense:2b",
            "granite:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "granite:8b": "ollama://granite3.1-dense:8b",
            "hermes": "huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q4_K_M.gguf",
            "ibm/granite": "ollama://granite3.1-dense:8b",
            "ibm/granite:2b": "ollama://granite3.1-dense:2b",
            "ibm/granite:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
            "ibm/granite:8b": "ollama://granite3.1-dense:8b",
            "merlinite": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "merlinite-lab-7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "merlinite-lab:7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "merlinite:7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
            "mistral": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
            "mistral:7b": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
            "mistral:7b-v1": "huggingface://TheBloke/Mistral-7B-Instruct-v0.1-GGUF/mistral-7b-instruct-v0.1.Q5_K_M.gguf",
            "mistral:7b-v2": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
            "mistral:7b-v3": "huggingface://MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf",
            "mistral_code_16k": "huggingface://TheBloke/Mistral-7B-Code-16K-qlora-GGUF/mistral-7b-code-16k-qlora.Q4_K_M.gguf",
            "mistral_codealpaca": "huggingface://TheBloke/Mistral-7B-codealpaca-lora-GGUF/mistral-7b-codealpaca-lora.Q4_K_M.gguf",
            "mixtao": "huggingface://MaziyarPanahi/MixTAO-7Bx2-MoE-Instruct-v7.0-GGUF/MixTAO-7Bx2-MoE-Instruct-v7.0.Q4_K_M.gguf",
            "openchat": "huggingface://TheBloke/openchat-3.5-0106-GGUF/openchat-3.5-0106.Q4_K_M.gguf",
            "openorca": "huggingface://TheBloke/Mistral-7B-OpenOrca-GGUF/mistral-7b-openorca.Q4_K_M.gguf",
            "phi2": "huggingface://MaziyarPanahi/phi-2-GGUF/phi-2.Q4_K_M.gguf",
            "smollm:135m": "ollama://smollm:135m",
            "tiny": "ollama://tinyllama"
        },
        "Files": [
            "/usr/share/ramalama/shortnames.conf",
            "/home/dwalsh/.config/ramalama/shortnames.conf",
        ]
        },
        "Store": "/usr/share/ramalama",
        "UseContainer": true,
        "Version": "0.7.5"
    }
    ```
</details>

- <details>
    <summary>
        Info with Podman engine.
    </summary>
    <br>

    ```
    $ ramalama info
    ```
    Returns for example:
    ```
    {
        "Accelerator": "cuda",
        "Engine": {
        "Info": {
            "host": {
            "arch": "amd64",
            "buildahVersion": "1.39.4",
            "cgroupControllers": [
                "cpu",
                "io",
                "memory",
                "pids"
            ],
            "cgroupManager": "systemd",
            "cgroupVersion": "v2",
            "conmon": {
                "package": "conmon-2.1.13-1.fc42.x86_64",
                "path": "/usr/bin/conmon",
                "version": "conmon version 2.1.13, commit: "
            },
            "cpuUtilization": {
                "idlePercent": 97.36,
                "systemPercent": 0.64,
                "userPercent": 2
            },
            "cpus": 32,
            "databaseBackend": "sqlite",
            "distribution": {
                "distribution": "fedora",
                "variant": "workstation",
                "version": "42"
            },
            "eventLogger": "journald",
            "freeLocks": 2043,
            "hostname": "danslaptop",
            "idMappings": {
                "gidmap": [
                {
                    "container_id": 0,
                    "host_id": 3267,
                    "size": 1
                },
                {
                    "container_id": 1,
                    "host_id": 524288,
                    "size": 65536
                }
                ],
                "uidmap": [
                {
                    "container_id": 0,
                    "host_id": 3267,
                    "size": 1
                },
                {
                    "container_id": 1,
                    "host_id": 524288,
                    "size": 65536
                }
                ]
            },
            "kernel": "6.14.2-300.fc42.x86_64",
            "linkmode": "dynamic",
            "logDriver": "journald",
            "memFree": 65281908736,
            "memTotal": 134690979840,
            "networkBackend": "netavark",
            "networkBackendInfo": {
                "backend": "netavark",
                "dns": {
                "package": "aardvark-dns-1.14.0-1.fc42.x86_64",
                "path": "/usr/libexec/podman/aardvark-dns",
                "version": "aardvark-dns 1.14.0"
                },
                "package": "netavark-1.14.1-1.fc42.x86_64",
                "path": "/usr/libexec/podman/netavark",
                "version": "netavark 1.14.1"
            },
            "ociRuntime": {
                "name": "crun",
                "package": "crun-1.21-1.fc42.x86_64",
                "path": "/usr/bin/crun",
                "version": "crun version 1.21\ncommit: 10269840aa07fb7e6b7e1acff6198692d8ff5c88\nrundir: /run/user/3267/crun\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL"
            },
            "os": "linux",
            "pasta": {
                "executable": "/bin/pasta",
                "package": "passt-0^20250415.g2340bbf-1.fc42.x86_64",
                "version": ""
            },
            "remoteSocket": {
                "exists": true,
                "path": "/run/user/3267/podman/podman.sock"
            },
            "rootlessNetworkCmd": "pasta",
            "security": {
                "apparmorEnabled": false,
                "capabilities": "CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT",
                "rootless": true,
                "seccompEnabled": true,
                "seccompProfilePath": "/usr/share/containers/seccomp.json",
                "selinuxEnabled": true
            },
            "serviceIsRemote": false,
            "slirp4netns": {
                "executable": "/bin/slirp4netns",
                "package": "slirp4netns-1.3.1-2.fc42.x86_64",
                "version": "slirp4netns version 1.3.1\ncommit: e5e368c4f5db6ae75c2fce786e31eef9da6bf236\nlibslirp: 4.8.0\nSLIRP_CONFIG_VERSION_MAX: 5\nlibseccomp: 2.5.5"
            },
            "swapFree": 8589930496,
            "swapTotal": 8589930496,
            "uptime": "116h 35m 40.00s (Approximately 4.83 days)",
            "variant": ""
            },
            "plugins": {
            "authorization": null,
            "log": [
                "k8s-file",
                "none",
                "passthrough",
                "journald"
            ],
            "network": [
                "bridge",
                "macvlan",
                "ipvlan"
            ],
            "volume": [
                "local"
            ]
            },
            "registries": {
            "search": [
                "registry.fedoraproject.org",
                "registry.access.redhat.com",
                "docker.io"
            ]
            },
            "store": {
            "configFile": "/home/dwalsh/.config/containers/storage.conf",
            "containerStore": {
                "number": 5,
                "paused": 0,
                "running": 0,
                "stopped": 5
            },
            "graphDriverName": "overlay",
            "graphOptions": {},
            "graphRoot": "/usr/share/containers/storage",
            "graphRootAllocated": 2046687182848,
            "graphRootUsed": 399990419456,
            "graphStatus": {
                "Backing Filesystem": "btrfs",
                "Native Overlay Diff": "true",
                "Supports d_type": "true",
                "Supports shifting": "false",
                "Supports volatile": "true",
                "Using metacopy": "false"
            },
            "imageCopyTmpDir": "/var/tmp",
            "imageStore": {
                "number": 297
            },
            "runRoot": "/run/user/3267/containers",
            "transientStore": false,
            "volumePath": "/usr/share/containers/storage/volumes"
            },
            "version": {
            "APIVersion": "5.4.2",
            "BuildOrigin": "Fedora Project",
            "Built": 1743552000,
            "BuiltTime": "Tue Apr  1 19:00:00 2025",
            "GitCommit": "be85287fcf4590961614ee37be65eeb315e5d9ff",
            "GoVersion": "go1.24.1",
            "Os": "linux",
            "OsArch": "linux/amd64",
            "Version": "5.4.2"
            }
        },
        "Name": "podman"
        },
        "Image": "quay.io/ramalama/cuda:0.7",
        "Inference": {
            "Default": "llama.cpp",
            "Engines": {
                "llama.cpp": "/usr/share/ramalama/inference-spec/engines/llama.cpp.yaml",
                "mlx": "/usr/share/ramalama/inference-spec/engines/mlx.yaml",
                "vllm": "/usr/share/ramalama/inference-spec/engines/vllm.yaml"
            },
            "Schema": {
                "1-0-0": "/usr/share/ramalama/inference-spec/schema/schema.1-0-0.json"
            }
        },
        "Shortnames": {
            "Names": {
                "cerebrum": "huggingface://froggeric/Cerebrum-1.0-7b-GGUF/Cerebrum-1.0-7b-Q4_KS.gguf",
                "deepseek": "ollama://deepseek-r1",
                "dragon": "huggingface://llmware/dragon-mistral-7b-v0/dragon-mistral-7b-q4_k_m.gguf",
                "gemma3": "hf://bartowski/google_gemma-3-4b-it-GGUF/google_gemma-3-4b-it-IQ2_M.gguf",
                "gemma3:12b": "hf://bartowski/google_gemma-3-12b-it-GGUF/google_gemma-3-12b-it-IQ2_M.gguf",
                "gemma3:1b": "hf://bartowski/google_gemma-3-1b-it-GGUF/google_gemma-3-1b-it-IQ2_M.gguf",
                "gemma3:27b": "hf://bartowski/google_gemma-3-27b-it-GGUF/google_gemma-3-27b-it-IQ2_M.gguf",
                "gemma3:4b": "hf://bartowski/google_gemma-3-4b-it-GGUF/google_gemma-3-4b-it-IQ2_M.gguf",
                "granite": "ollama://granite3.1-dense",
                "granite-code": "hf://ibm-granite/granite-3b-code-base-2k-GGUF/granite-3b-code-base.Q4_K_M.gguf",
                "granite-code:20b": "hf://ibm-granite/granite-20b-code-base-8k-GGUF/granite-20b-code-base.Q4_K_M.gguf",
                "granite-code:34b": "hf://ibm-granite/granite-34b-code-base-8k-GGUF/granite-34b-code-base.Q4_K_M.gguf",
                "granite-code:3b": "hf://ibm-granite/granite-3b-code-base-2k-GGUF/granite-3b-code-base.Q4_K_M.gguf",
                "granite-code:8b": "hf://ibm-granite/granite-8b-code-base-4k-GGUF/granite-8b-code-base.Q4_K_M.gguf",
                "granite-lab-7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
                "granite-lab-8b": "huggingface://ibm-granite/granite-8b-code-base-GGUF/granite-8b-code-base.Q4_K_M.gguf",
                "granite-lab:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
                "granite:2b": "ollama://granite3.1-dense:2b",
                "granite:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
                "granite:8b": "ollama://granite3.1-dense:8b",
                "hermes": "huggingface://NousResearch/Hermes-2-Pro-Mistral-7B-GGUF/Hermes-2-Pro-Mistral-7B.Q4_K_M.gguf",
                "ibm/granite": "ollama://granite3.1-dense:8b",
                "ibm/granite:2b": "ollama://granite3.1-dense:2b",
                "ibm/granite:7b": "huggingface://instructlab/granite-7b-lab-GGUF/granite-7b-lab-Q4_K_M.gguf",
                "ibm/granite:8b": "ollama://granite3.1-dense:8b",
                "merlinite": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
                "merlinite-lab-7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
                "merlinite-lab:7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
                "merlinite:7b": "huggingface://instructlab/merlinite-7b-lab-GGUF/merlinite-7b-lab-Q4_K_M.gguf",
                "mistral": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
                "mistral:7b": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
                "mistral:7b-v1": "huggingface://TheBloke/Mistral-7B-Instruct-v0.1-GGUF/mistral-7b-instruct-v0.1.Q5_K_M.gguf",
                "mistral:7b-v2": "huggingface://TheBloke/Mistral-7B-Instruct-v0.2-GGUF/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
                "mistral:7b-v3": "huggingface://MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf",
                "mistral_code_16k": "huggingface://TheBloke/Mistral-7B-Code-16K-qlora-GGUF/mistral-7b-code-16k-qlora.Q4_K_M.gguf",
                "mistral_codealpaca": "huggingface://TheBloke/Mistral-7B-codealpaca-lora-GGUF/mistral-7b-codealpaca-lora.Q4_K_M.gguf",
                "mixtao": "huggingface://MaziyarPanahi/MixTAO-7Bx2-MoE-Instruct-v7.0-GGUF/MixTAO-7Bx2-MoE-Instruct-v7.0.Q4_K_M.gguf",
                "openchat": "huggingface://TheBloke/openchat-3.5-0106-GGUF/openchat-3.5-0106.Q4_K_M.gguf",
                "openorca": "huggingface://TheBloke/Mistral-7B-OpenOrca-GGUF/mistral-7b-openorca.Q4_K_M.gguf",
                "phi2": "huggingface://MaziyarPanahi/phi-2-GGUF/phi-2.Q4_K_M.gguf",
                "smollm:135m": "ollama://smollm:135m",
                "tiny": "ollama://tinyllama"
            },
            "Files": [
                "/usr/share/ramalama/shortnames.conf",
                "/home/dwalsh/.config/ramalama/shortnames.conf",
            ]
        },
        "Store": "/usr/share/ramalama",
        "UseContainer": true,
        "Version": "0.7.5"
    }
    ```
</details>

- <details>
    <summary>
        Using jq to print specific `ramalama info` content.
    </summary>
    <br>

    ```
    $ ramalama info |  jq .Shortnames.Names.mixtao
    ```
    Returns for example:
    ```
   "huggingface://MaziyarPanahi/MixTAO-7Bx2-MoE-Instruct-v7.0-GGUF/MixTAO-7Bx2-MoE-Instruct-v7.0.Q4_K_M.gguf"
    ```
</details>

### [`ramalama-inspect`](https://github.com/containers/ramalama/blob/main/docs/ramalama-inspect.1.md)
#### Inspect the specified AI Model.
- <details>
    <summary>
        Inspect the smollm:135m model for basic information.
    </summary>
    <br>

    ```
    $ ramalama inspect smollm:135m
    ```
    Returns for example:
    ```
    smollm:135m
       Path: /var/lib/ramalama/models/ollama/smollm:135m
       Registry: ollama
       Format: GGUF
       Version: 3
       Endianness: little
       Metadata: 39 entries
       Tensors: 272 entries
    ```
</details>

- <details>
    <summary>
        Inspect the smollm:135m model for all information in json format.
    </summary>
    <br>

    ```
    $ ramalama inspect smollm:135m --all --json
    ```
    Returns for example:
    ```
    {
        "Name": "smollm:135m",
        "Path": "/home/mengel/.local/share/ramalama/models/ollama/smollm:135m",
        "Registry": "ollama",
        "Format": "GGUF",
        "Version": 3,
        "LittleEndian": true,
        "Metadata": {
        "general.architecture": "llama",
        "general.base_model.0.name": "SmolLM 135M",
        "general.base_model.0.organization": "HuggingFaceTB",
        "general.base_model.0.repo_url": "https://huggingface.co/HuggingFaceTB/SmolLM-135M",
        ...
        },
        "Tensors": [
        {
            "dimensions": [
            576,
            49152
            ],
            "n_dimensions": 2,
            "name": "token_embd.weight",
            "offset": 0,
            "type": 8
        },
        ...
        ]
    }
    ```
</details>

### [`ramalama-list`](https://github.com/containers/ramalama/blob/main/docs/ramalama-list.1.md)
#### List all downloaded AI Models.
- <details>
    <summary>
        You can `list` all models pulled into local storage.
    </summary>
    <br>

    ```
    $ ramalama list
    ```
    Returns for example:
    ```
    NAME                                                                    MODIFIED      SIZE
    ollama://smollm:135m                                                    16 hours ago  5.5M
    huggingface://afrideva/Tiny-Vicuna-1B-GGUF/tiny-vicuna-1b.q2_k.gguf     14 hours ago  460M
    ollama://moondream:latest                                               6 days ago    791M
    ollama://phi4:latest                                                    6 days ago    8.43 GB
    ollama://tinyllama:latest                                               1 week ago    608.16 MB
    ollama://granite3-moe:3b                                                1 week ago    1.92 GB
    ollama://granite3-moe:latest                                            3 months ago  1.92 GB
    ollama://llama3.1:8b                                                    2 months ago  4.34 GB
    ollama://llama3.1:latest                                                2 months ago  4.34 GB
    ```
</details>

### [`ramalama-login`](https://github.com/containers/ramalama/blob/main/docs/ramalama-login.1.md)
#### Log in to a remote registry.
- <details>
    <summary>
        Log in to quay.io/username oci registry
    </summary>
    <br>

    ```
    $ export RAMALAMA_TRANSPORT=quay.io/username
    $ ramalama login -u username
    ```
</details>

- <details>
    <summary>
        Log in to Ollama registry
    </summary>
    <br>

    ```
    $ export RAMALAMA_TRANSPORT=ollama
    $ ramalama login
    ```
</details>

- <details>
    <summary>
        Log in to huggingface registry
    </summary>
    <br>

    ```
    $ export RAMALAMA_TRANSPORT=huggingface
    $ ramalama login --token=XYZ
    ```

    Logging in to Hugging Face requires the `hf tool`. For installation and usage instructions, see the documentation of the [Hugging Face command line interface](https://huggingface.co/docs/huggingface_hub/en/guides/cli).
</details>

### [`ramalama-logout`](https://github.com/containers/ramalama/blob/main/docs/ramalama-logout.1.md)
#### Log out of a remote registry.
- <details>
    <summary>
        Log out from quay.io/username oci repository
    </summary>
    <br>

    ```
    $ ramalama logout quay.io/username
    ```
</details>

- <details>
    <summary>
        Log out from Ollama registry
    </summary>
    <br>

    ```
    $ ramalama logout ollama
    ```
</details>

- <details>
    <summary>
        Log out from huggingface
    </summary>
    <br>

    ```
    $ ramalama logout huggingface
    ```