Repository: containers/ramalama
Pull request #1345: Use CLI ramalama serve here
Author: login: ericcurtin
Created: 2025-05-05T12:28:50Z | Merged: 2025-05-06T12:51:20Z

It's easier to debug, we can do ps -ef, etc. Easier to code also.

## Summary by Sourcery

Enhancements:
- Improve debugging capabilities by switching to CLI serve approach


Diff:
diff --git a/libexec/ramalama/ramalama-client-core b/libexec/ramalama/ramalama-client-core
index 0a02d48f4..a4e47f694 100755
--- a/libexec/ramalama/ramalama-client-core
+++ b/libexec/ramalama/ramalama-client-core
@@ -52,7 +52,7 @@ def res(response, color):
     return assistant_response
 
 
-def req(conversation_history, url, color):
+def req(conversation_history, url, parsed_args):
     data = construct_request_data(conversation_history)
     json_data = json.dumps(data).encode("utf-8")
     headers = {
@@ -64,10 +64,8 @@ def req(conversation_history, url, color):
 
     # Send request
     i = 0.01
+    response = None
     for c in itertools.cycle(['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏']):
-        if i > 10:
-            break
-
         try:
             response = urllib.request.urlopen(request)
             break
@@ -75,11 +73,20 @@ def req(conversation_history, url, color):
             if sys.stdout.isatty():
                 print(f"\r{c}", end="", flush=True)
 
+            if i > 32:
+                break
+
             time.sleep(i)
             i *= 2
 
-    return res(response, color)
+    if response:
+        return res(response, parsed_args.color)
+
+    from ramalama.common import perror
+    perror(f"\rError: could not connect to: {url}")
+    do_kills(parsed_args)
 
+    return None
 
 class RamaLamaShell(cmd.Cmd):
     def __init__(self, parsed_args):
@@ -104,8 +111,12 @@ class RamaLamaShell(cmd.Cmd):
 
         self.conversation_history.append({"role": "user", "content": user_content})
         self.request_in_process = True
+        response = req(self.conversation_history, self.url, self.parsed_args)
+        if not response:
+            return True
+
         self.conversation_history.append(
-            {"role": "assistant", "content": req(self.conversation_history, self.url, self.parsed_args.color)}
+            {"role": "assistant", "content": response}
         )
         self.request_in_process = False
 
@@ -167,6 +178,8 @@ def run_shell_loop(ramalama_shell):
         break
 
 def main(args):
+    sys.path.append('./')
+
     parsed_args = parse_arguments(args)
     ramalama_shell = RamaLamaShell(parsed_args)
     if handle_args(parsed_args, ramalama_shell):
diff --git a/libexec/ramalama/ramalama-run-core b/libexec/ramalama/ramalama-run-core
index f09966a9e..3cb53fe01 100755
--- a/libexec/ramalama/ramalama-run-core
+++ b/libexec/ramalama/ramalama-run-core
@@ -65,22 +65,8 @@ def main(args):
     pid = os.fork()
     if pid == 0:
         signal.pthread_sigmask(signal.SIG_BLOCK, {signal.SIGINT})
-        with open(os.devnull, 'w') as devnull:
-            os.dup2(devnull.fileno(), sys.stderr.fileno())
-
-        args = Namespace(
-            container=False, dryrun=False, engine=None, podman_keep_groups=False,
-            image=None, runtime='llama.cpp',
-            store=os.path.expanduser("~/.local/share/ramalama"), use_model_store=False,
-            quiet=False, debug=False, subcommand='serve', ngl=parsed_args.ngl, threads=parsed_args.threads,
-            temp=parsed_args.temp, authfile=None, env=[], device=None, name=None,
-            oci_runtime=None, privileged=False, pull='newer', seed=None,
-            tlsverify=True, context=parsed_args.context, runtime_args=[], network=None,
-            detach=False, host='127.0.0.1', webui='on', generate=None, port=str(port),
-            rag=None, MODEL=parsed_args.MODEL, func=lambda: None,
-            UNRESOLVED_MODEL=parsed_args.MODEL
-        )
-        serve_cli(args)
+        # at this point we are already in a container so, --nocontainer always makes sense
+        exec_cmd(["ramalama", "--nocontainer", "serve", "--threads", str(parsed_args.threads), "--temp", parsed_args.temp, "--port", str(port), parsed_args.MODEL], stdout2null=True, stderr2null=True)
 
         return 0
 
diff --git a/ramalama/common.py b/ramalama/common.py
index 0cc4a0b92..36d743823 100644
--- a/ramalama/common.py
+++ b/ramalama/common.py
@@ -133,10 +133,14 @@ def quoted(arr):
     return " ".join(['"' + element + '"' if ' ' in element else element for element in arr])
 
 
-def exec_cmd(args, debug=False, stderr2null=False):
+def exec_cmd(args, debug=False, stdout2null=False, stderr2null=False):
     if debug:
         perror("exec_cmd: ", quoted(args))
 
+    if stdout2null:
+        with open(os.devnull, 'w') as devnull:
+            os.dup2(devnull.fileno(), sys.stdout.fileno())
+
     if stderr2null:
         with open(os.devnull, 'w') as devnull:
             os.dup2(devnull.fileno(), sys.stderr.fileno())
diff --git a/test/system/030-run.bats b/test/system/030-run.bats
index 96574c425..a808cbc33 100755
--- a/test/system/030-run.bats
+++ b/test/system/030-run.bats
@@ -2,7 +2,7 @@
 
 load helpers
 
-MODEL=smollm:1.7b
+MODEL=smollm:135m
 
 @test "ramalama --dryrun run basic output" {
     image=m_$(safename)


Reviews:
user: login: sourcery-ai[bot]
body: Hey @ericcurtin - I've reviewed your changes and they look great!

<details>
<summary>Here's what I looked at during the review</summary>

- 🟢 **General issues**: all looks good
- 🟢 **Security**: all looks good
- 🟢 **Testing**: all looks good
- 🟢 **Complexity**: all looks good
- 🟢 **Documentation**: all looks good
</details>

***

<details>
<summary>Sourcery is free for open source - if you like our reviews please consider sharing them ✨</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https://sourcery.ai)
- [Facebook](https://www.facebook.com/sharer/sharer.php?u=https://sourcery.ai)

</details>

<sub>
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
</sub>
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-05T12:29:04Z
user: login: rhatdan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T13:39:35Z
user: login: ericcurtin
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T13:45:52Z
user: login: rhatdan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T14:47:32Z
user: login: ericcurtin
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T15:07:52Z
user: login: rhatdan
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T11:51:33Z

Review comments:
diff_hunk: @@ -68,19 +68,11 @@ def main(args):
         with open(os.devnull, 'w') as devnull:
             os.dup2(devnull.fileno(), sys.stderr.fileno())
 
-        args = Namespace(
-            container=False, dryrun=False, engine=None, podman_keep_groups=False,
-            image=None, runtime='llama.cpp',
-            store=os.path.expanduser("~/.local/share/ramalama"), use_model_store=False,
-            quiet=False, debug=False, subcommand='serve', ngl=parsed_args.ngl, threads=parsed_args.threads,
-            temp=parsed_args.temp, authfile=None, env=[], device=None, name=None,
-            oci_runtime=None, privileged=False, pull='newer', seed=None,
-            tlsverify=True, context=parsed_args.context, runtime_args=[], network=None,
-            detach=False, host='127.0.0.1', webui='on', generate=None, port=str(port),
-            rag=None, MODEL=parsed_args.MODEL, func=lambda: None,
-            UNRESOLVED_MODEL=parsed_args.MODEL
-        )
-        serve_cli(args)
+        with open(os.devnull, 'w') as devnull:
path: libexec/ramalama/ramalama-run-core
user: login: rhatdan
body: Looks like  you duplicated this code?
created_at: 2025-05-05T13:39:34Z
author_association: MEMBER
diff_hunk: @@ -68,19 +68,11 @@ def main(args):
         with open(os.devnull, 'w') as devnull:
             os.dup2(devnull.fileno(), sys.stderr.fileno())
 
-        args = Namespace(
-            container=False, dryrun=False, engine=None, podman_keep_groups=False,
-            image=None, runtime='llama.cpp',
-            store=os.path.expanduser("~/.local/share/ramalama"), use_model_store=False,
-            quiet=False, debug=False, subcommand='serve', ngl=parsed_args.ngl, threads=parsed_args.threads,
-            temp=parsed_args.temp, authfile=None, env=[], device=None, name=None,
-            oci_runtime=None, privileged=False, pull='newer', seed=None,
-            tlsverify=True, context=parsed_args.context, runtime_args=[], network=None,
-            detach=False, host='127.0.0.1', webui='on', generate=None, port=str(port),
-            rag=None, MODEL=parsed_args.MODEL, func=lambda: None,
-            UNRESOLVED_MODEL=parsed_args.MODEL
-        )
-        serve_cli(args)
+        with open(os.devnull, 'w') as devnull:
path: libexec/ramalama/ramalama-run-core
user: login: ericcurtin
body: It's deliberate, one redirects stderr to null, the other redirects stdout to null.

Could create a function for this with a parameter for stderr/stdout seems I just didn't because it seemed like overkill for something that's 2 lines and called twice

created_at: 2025-05-05T13:45:52Z
author_association: MEMBER
diff_hunk: @@ -68,19 +68,11 @@ def main(args):
         with open(os.devnull, 'w') as devnull:
             os.dup2(devnull.fileno(), sys.stderr.fileno())
 
-        args = Namespace(
-            container=False, dryrun=False, engine=None, podman_keep_groups=False,
-            image=None, runtime='llama.cpp',
-            store=os.path.expanduser("~/.local/share/ramalama"), use_model_store=False,
-            quiet=False, debug=False, subcommand='serve', ngl=parsed_args.ngl, threads=parsed_args.threads,
-            temp=parsed_args.temp, authfile=None, env=[], device=None, name=None,
-            oci_runtime=None, privileged=False, pull='newer', seed=None,
-            tlsverify=True, context=parsed_args.context, runtime_args=[], network=None,
-            detach=False, host='127.0.0.1', webui='on', generate=None, port=str(port),
-            rag=None, MODEL=parsed_args.MODEL, func=lambda: None,
-            UNRESOLVED_MODEL=parsed_args.MODEL
-        )
-        serve_cli(args)
+        with open(os.devnull, 'w') as devnull:
path: libexec/ramalama/ramalama-run-core
user: login: rhatdan
body: Ok I missed the subtle difference.
created_at: 2025-05-05T14:47:32Z
author_association: MEMBER
diff_hunk: @@ -68,19 +68,11 @@ def main(args):
         with open(os.devnull, 'w') as devnull:
             os.dup2(devnull.fileno(), sys.stderr.fileno())
 
-        args = Namespace(
-            container=False, dryrun=False, engine=None, podman_keep_groups=False,
-            image=None, runtime='llama.cpp',
-            store=os.path.expanduser("~/.local/share/ramalama"), use_model_store=False,
-            quiet=False, debug=False, subcommand='serve', ngl=parsed_args.ngl, threads=parsed_args.threads,
-            temp=parsed_args.temp, authfile=None, env=[], device=None, name=None,
-            oci_runtime=None, privileged=False, pull='newer', seed=None,
-            tlsverify=True, context=parsed_args.context, runtime_args=[], network=None,
-            detach=False, host='127.0.0.1', webui='on', generate=None, port=str(port),
-            rag=None, MODEL=parsed_args.MODEL, func=lambda: None,
-            UNRESOLVED_MODEL=parsed_args.MODEL
-        )
-        serve_cli(args)
+        with open(os.devnull, 'w') as devnull:
path: libexec/ramalama/ramalama-run-core
user: login: ericcurtin
body: Changed it anyway to try and make it more clear
created_at: 2025-05-05T15:07:52Z
author_association: MEMBER
diff_hunk: @@ -64,22 +64,28 @@ def req(conversation_history, url, color):
 
     # Send request
     i = 0.01
+    response = None
     for c in itertools.cycle(['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏']):
-        if i > 10:
-            break
-
         try:
             response = urllib.request.urlopen(request)
             break
         except Exception:
             if sys.stdout.isatty():
                 print(f"\r{c}", end="", flush=True)
 
+            if i > 32:
+                break
+
             time.sleep(i)
             i *= 2
 
-    return res(response, color)
+    if response:
+        return res(response, parsed_args.color)
 
+    print(f"\rError: could not connect to: {url}")
path: libexec/ramalama/ramalama-client-core
user: login: rhatdan
body: perror

created_at: 2025-05-06T11:51:33Z
author_association: MEMBER

PR comments:
user: login: sourcery-ai[bot]
created_at: 2025-05-05T12:28:54Z
body: <!-- Generated by sourcery-ai[bot]: start review_guide -->

## Reviewer's Guide

This pull request modifies the `ramalama-run-core` script to start the core process using the `ramalama serve` command-line interface, replacing the previous execution method.

### File-Level Changes

| Change | Details | Files |
| ------ | ------- | ----- |
| Replaced the core process execution logic with a call to `ramalama serve`. | <ul><li>Removed the previous method of starting the ramalama core process.</li><li>Added execution of the `ramalama serve` command.</li></ul> | `libexec/ramalama/ramalama-run-core` |

---

<details>
<summary>Tips and commands</summary>

#### Interacting with Sourcery

- **Trigger a new review:** Comment `@sourcery-ai review` on the pull request.
- **Continue discussions:** Reply directly to Sourcery's review comments.
- **Generate a GitHub issue from a review comment:** Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with `@sourcery-ai issue` to create an issue from it.
- **Generate a pull request title:** Write `@sourcery-ai` anywhere in the pull
  request title to generate a title at any time. You can also comment
  `@sourcery-ai title` on the pull request to (re-)generate the title at any time.
- **Generate a pull request summary:** Write `@sourcery-ai summary` anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment `@sourcery-ai summary` on the pull request to
  (re-)generate the summary at any time.
- **Generate reviewer's guide:** Comment `@sourcery-ai guide` on the pull
  request to (re-)generate the reviewer's guide at any time.
- **Resolve all Sourcery comments:** Comment `@sourcery-ai resolve` on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
- **Dismiss all Sourcery reviews:** Comment `@sourcery-ai dismiss` on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  `@sourcery-ai review` to trigger a new review!

#### Customizing Your Experience

Access your [dashboard](https://app.sourcery.ai) to:
- Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
- Change the review language.
- Add, remove or edit custom review instructions.
- Adjust other review settings.

#### Getting Help

- [Contact our support team](mailto:support@sourcery.ai) for questions or feedback.
- Visit our [documentation](https://docs.sourcery.ai) for detailed guides and information.
- Keep in touch with the Sourcery team by following us on [X/Twitter](https://x.com/SourceryAI), [LinkedIn](https://www.linkedin.com/company/sourcery-ai/) or [GitHub](https://github.com/sourcery-ai).

</details>

<!-- Generated by sourcery-ai[bot]: end review_guide -->
author_association: CONTRIBUTOR
user: login: ericcurtin
created_at: 2025-05-05T13:47:26Z
body: This PR is to help me figure out the broken test, I don't have a way of fixing --nocontainer tests locally on Linux right now:

https://github.com/containers/ramalama/pull/1347
author_association: MEMBER
user: login: rhatdan
created_at: 2025-05-05T14:48:17Z
body: Is the failure a flake?
author_association: MEMBER
user: login: ericcurtin
created_at: 2025-05-05T14:56:21Z
body: > Is the failure a flake?

Sadly I think not, it's occurred multiple times 😢 
author_association: MEMBER
user: login: ericcurtin
created_at: 2025-05-05T14:56:39Z
body: Trying to re-produce outside CI
author_association: MEMBER
user: login: ericcurtin
created_at: 2025-05-05T15:04:48Z
body: I think I figured it out, missed passing the port
author_association: MEMBER
user: login: ericcurtin
created_at: 2025-05-05T15:33:38Z
body: I'm not sure but I think we must just increase the timeouts now
author_association: MEMBER
user: login: rhatdan
created_at: 2025-05-06T11:52:12Z
body: One small change and then LGTM
author_association: MEMBER
user: login: ericcurtin
created_at: 2025-05-06T12:39:48Z
body: It's green @rhatdan 
author_association: MEMBER