Repository: dezoito/ollama-grid-search
Issue: Connecting to Ollama servers has stopped working
Root cause: Validation Gap

Hello 👋 , I noticed that recently connecting to remote Ollama servers stopped working, while localhost remains fine.

Just wondering if something has perhaps changed in the Ollama API or the way OGS handles network connectivity that could be causing this?

When providing a valid Ollama url (e.g. https://ollama.my.internet.network) and saving the settings - the list of models remains loading forever:

<img width="473" alt="image" src="https://github.com/user-attachments/assets/92eb6ea3-9072-4f1e-b3a8-2e91ac259f1a">

- Other Ollama clients are all working fine with the same host/url.
- Using localhost:11434 seems fine
- I've tried building from source with updated node packages but that didn't help.

Ollama server (and local) are both built from the latest Ollama source.


@sammcj , thank you for writing the detailed post.

I can't currently reproduce the issue, but will try again in a different environment later.

This project uses [Ollama-rs](https://github.com/pepperoni21/ollama-rs/) to handle all http calls (with the exception of getting the current version of Ollama running on the server).

I just checked there and could not find any issues related to network connectivity, but since your other clients work I agree that there might be an issue either here or on that upstream dependency.

- If you call `ollama-grid-search` from the command line, can you spot any errors or relevant debug information?

- If you use your remote server's IP address instead of the host name, is there any change in behavior?

Sorry Dezoito, I've been away for a few days, I'll follow up on this properly tomorrow - but from a quick look it seems to be hanging getting the ollama version:

```
thread 'tokio-runtime-worker' panicked at src/lib.rs:115:25:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Fetching ollama version from https://ollama.my.internal.domain/api/version
[src/commands.rs:54:5] &response = Response {
    url: "https://ollama.my.internal.domain/api/version",
    status: 200,
    headers: {
        "alt-svc": "h3=\":443\"; ma=2592000",
        "content-length": "19",
        "content-type": "application/json; charset=utf-8",
        "date": "Mon, 26 Aug 2024 12:08:10 GMT",
    },
}
```

This returns correctly for me:

```
curl https://ollama.my.internal.domain/api/version
{"version":"0.0.0"}
```

(0.0.0 is "correct" behaviour as it's Ollama returns whenever you build it yourself)

>  If you use your remote server's IP address instead of the host name, is there any change in behavior?

I can't use an IP address directly Ollama is running behind a reverse proxy (for HTTPS / containerisation).

Awesome, thank you. 

This is gives me a starting point. 

Thank you, I really appreciate it. I mentioned OGS in a talk I gave last month actually: https://smcleod.net/2024/07/code-chaos-and-copilots-ai/llm-talk-july-2024/ (slide 69 I think) and someone that was there pinged me to ask if I had started facing issue and I realised I did - so that's how this report came about!

That presentation looks amazing... I seriously appreciate the effort you must have put into it.

I'm sharing it with a group of AI people @ work.

@sammcj (and anyone else who know what they are doing, really),

I just getting started in trying to debug this and noticed two things, comparing the response output you posted with the one on my machine:

```
Fetching models from http://localhost
Fetching ollama version from http://localhost:11434/api/version
[src/commands.rs:54:5] &response = Response {
    url: Url {
        scheme: "http",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "localhost",
            ),
        ),
        port: Some(
            11434,
        ),
        path: "/api/version",
        query: None,
        fragment: None,
    },
    status: 200,
    headers: {
        "content-type": "application/json; charset=utf-8",
        "date": "Tue, 27 Aug 2024 11:43:09 GMT",
        "content-length": "19",
    },
}
```

First, the line `Fetching ollama version from http://localhost:11434/api/version` shows the port for the Ollama server, but I don't see that information on your post. Is the ollama server configured to a default port for your remote server (like 443, since you are using https?).

If you manually add port 443 to the server URL in settings, does anything change?

Secondly, my response includes a lot of additional information regarding the URL, and judging by the error

```
thread 'tokio-runtime-worker' panicked at src/lib.rs:115:25:
called `Option::unwrap()` on a `None` value
```

The absence of some of those keys may have caused the issue (I'm writing this more as an output of internal monologue and in the hopes that other people can help ;) )

I'm thinking about rewriting this in a way that this call returns "version not available" if there are any errors (using an `unwrap_or()` for the Rust nerds out there).



@sammcj,

Just (finally) released a new minor version.
Hopefully this solves the issue that prevented the UI not rendering, but if not, please let me know.



Hey Dezoito 👋 ,

Just tried the latest build but it's the same:

<img width="1704" alt="image" src="https://github.com/user-attachments/assets/7102ef8f-5997-415d-b13f-311ec170b434">


```
 ./ollama-grid-search
thread 'tokio-runtime-worker' panicked at src/lib.rs:115:25:
called `Option::unwrap()` on a `None` value
Fetching ollama version from https://ollama.internal.domain/api/version
stack backtrace:
   0:        0x102988678 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h243268f17d714c7f
   1:        0x1029a7b4c - core::fmt::write::hb3cfb8a30e72d7ff
   2:        0x102985904 - std::io::Write::write_fmt::hfb2314975de9ecf1
   3:        0x102989754 - std::panicking::default_hook::{{closure}}::h14c7718ccf39d316
   4:        0x102989378 - std::panicking::default_hook::hc62e60da3be2f352
   5:        0x10298a218 - std::panicking::rust_panic_with_hook::h09e8a656f11e82b2
   6:        0x102989b40 - std::panicking::begin_panic_handler::{{closure}}::h1230eb3cc91b241c
   7:        0x102988b04 - std::sys::backtrace::__rust_end_short_backtrace::hc3491307aceda2c2
   8:        0x102989830 - _rust_begin_unwind
   9:        0x1029caad0 - core::panicking::panic_fmt::ha4b80a05b9fff47a
  10:        0x1029cab3c - core::panicking::panic::h298549a7412a7069
  11:        0x1029caa58 - core::option::unwrap_failed::hb7af631ec4f78cd6
  12:        0x10267587c - grid_search_desktop::split_host_port::hf4b8dc5b50bb0a9a
  13:        0x10261bd40 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h20a22abc505ce30c
  14:        0x10261d850 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::hd3b3c5e10256b7f2
  15:        0x1025ced7c - tauri::hooks::InvokeResolver<R>::respond_async_serialized::{{closure}}::h9b71474746cb288e
  16:        0x1025d4d40 - tokio::runtime::task::core::Core<T,S>::poll::he288c8b53dee3a05
  17:        0x1025f8368 - tokio::runtime::task::harness::Harness<T,S>::poll::h479dc18a34c6a26b
  18:        0x10286630c - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h37103666849dad32
  19:        0x1028657b0 - tokio::runtime::scheduler::multi_thread::worker::Context::run::h786d1530061b2df0
  20:        0x102869800 - tokio::runtime::context::scoped::Scoped<T>::set::h94cc4ac376a7991f
  21:        0x102873820 - tokio::runtime::context::runtime::enter_runtime::hcc81e48ef7b32ff7
  22:        0x102865600 - tokio::runtime::scheduler::multi_thread::worker::run::h942ad243713a1476
  23:        0x10286c20c - tokio::runtime::task::core::Core<T,S>::poll::he5e1a65201261146
  24:        0x1028586e4 - tokio::runtime::task::harness::Harness<T,S>::poll::h07433706a2e54d05
  25:        0x10285ada8 - tokio::runtime::blocking::pool::Inner::run::hb443c8f5e869d718
  26:        0x10286ac84 - std::sys::backtrace::__rust_begin_short_backtrace::h17efc0b2c2a02c97
  27:        0x10286b448 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h85c88631f453f70f
  28:        0x10298e8e4 - std::sys::pal::unix::thread::Thread::new::thread_start::h1bd1b9c95010bf71
  29:        0x18c9d9f94 - __pthread_joiner_wake
[src/commands.rs:55:5] &response = Response {
    url: "https://ollama.internal.domain/api/version",
    status: 200,
    headers: {
        "alt-svc": "h3=\":443\"; ma=2592000",
        "content-length": "19",
        "content-type": "application/json; charset=utf-8",
        "date": "Sun, 15 Sep 2024 23:08:43 GMT",
    },
}
```


According to AI (🤣):

This crash log indicates a panic in a Rust program, specifically in a project that seems to be interacting with Ollama (an open-source large language model runner). Let's break down the key points:

1. The panic occurred in the `tokio-runtime-worker` thread, which suggests the program is using the Tokio asynchronous runtime.

2. The specific error is:
   ```
   called `Option::unwrap()` on a `None` value
   ```
   This means the code tried to unwrap an `Option` that was `None`, likely assuming it would always have a value.

3. The error happened in the `split_host_port` function in the `grid_search_desktop` module:
   ```
   grid_search_desktop::split_host_port::hf4b8dc5b50bb0a9a
   ```

4. The program was trying to fetch the Ollama version from `https://ollama.internal.domain/api/version`.

5. Interestingly, despite the panic, the program seems to have successfully made the HTTP request and received a 200 OK response, as evidenced by the debug output at the end of the log.

The crash likely occurred because the code assumed it could always split the host and port from the URL, but in this case, it couldn't. This could happen if:

1. The URL doesn't explicitly specify a port.
2. There's an unexpected format in the URL.
3. There's a bug in the `split_host_port` function that doesn't handle certain edge cases.

To fix this, the developers should:

1. Review the `split_host_port` function and ensure it handles cases where a port might not be present.
2. Use `Option::map` or pattern matching instead of `unwrap` to safely handle the case where splitting fails.
3. Ensure the URL format being passed to this function is always what the function expects.

It's worth noting that the program successfully made the HTTP request before crashing, so the issue is likely in post-processing of the URL rather than in the network request itself.

Thank  you again, @sammcj.


 Can you try to explicitly define the host port for your Ollama server?

`https://ollama.internal.domain:<your port>`

I'll tackle the lack of a defined port, but want to check if this could still work.

UPDATE:
Version 0.5.3 **_should_** handle server URLs that use default ports.
If the port is not defined by the user, it will use 443 or 80, according to the URL's protocol.

YES!!! It works! 🎉 

Well done @dezoito! 

<img width="2197" alt="image" src="https://github.com/user-attachments/assets/7566baa5-7d22-4975-8865-eaee4f55075b">


Thanks so much for your efforts on this, truly appreciated!