ronantakizawa/codereview-bench huggingface dataset:
Repository: ErikBjare/gptme
File: gptme/tools/shell.py (Python)

Code before:

def is_allowlisted(cmd: str) -> bool:
    for match in cmd_regex.finditer(cmd):
        for group in match.groups():
            if group and group not in allowlist_commands:
                return False
    return True


def execute_shell_impl(
    cmd: str, _: Path | None, confirm: ConfirmFunc
) -> Generator[Message, None, None]:
    """Execute shell command and format output."""
    shell = get_shell()
    allowlisted = is_allowlisted(cmd)

    try:
        returncode, stdout, stderr = shell.run(cmd)
    except Exception as e:
        raise ValueError(f"Shell error: {e}") from None

    stdout = _shorten_stdout(stdout.strip(), pre_tokens=2000, post_tokens=8000)
    stderr = _shorten_stdout(stderr.strip(), pre_tokens=2000, post_tokens=2000)
    msg = (
    _format_block_smart(
        f"{'Failed to run' if returncode else 'Ran'} "
        f"{'allowlisted ' if allowlisted else ''}command",
        cmd,
        lang="bash"
    )
    + "\n\n"
)
    if stdout:
        msg += _format_block_smart("", stdout, "stdout") + "\n\n"
    if stderr:
        msg += _format_block_smart("", stderr, "stderr") + "\n\n"
    if not stdout and not stderr:
        msg += "No output\n"
    if returncode:
        msg += f"Return code: {returncode}"

    yield Message("system", msg)


def get_path_fn(*args, **kwargs) -> Path | None:
    return None


def execute_shell(
    code: str | None,
    args: list[str] | None,

Review comment (bug):
I think "failed to run" might be bad prompting. We should be careful to not infer too much from a non-zero exit code I think. I'd just keep the "Ran" and not try to infer a success/error status for shell commands, for now.

Code after:

def is_allowlisted(cmd: str) -> bool:
    for match in cmd_regex.finditer(cmd):
        for group in match.groups():
            if group and group not in allowlist_commands:
                return False
    return True


def execute_shell_impl(
    cmd: str, _: Path | None, confirm: ConfirmFunc
) -> Generator[Message, None, None]:
    """Execute shell command and format output."""
    shell = get_shell()
    allowlisted = is_allowlisted(cmd)

    try:
        returncode, stdout, stderr = shell.run(cmd)
    except Exception as e:
        raise ValueError(f"Shell error: {e}") from None

    stdout = _shorten_stdout(stdout.strip(), pre_tokens=2000, post_tokens=8000)
    stderr = _shorten_stdout(stderr.strip(), pre_tokens=2000, post_tokens=2000)

    msg = (
        _format_block_smart(
            f"Ran {'allowlisted ' if allowlisted else ''}command", cmd, lang="bash"
        )
        + "\n\n"
    )
    if stdout:
        msg += _format_block_smart("", stdout, "stdout") + "\n\n"
    if stderr:
        msg += _format_block_smart("", stderr, "stderr") + "\n\n"
    if not stdout and not stderr:
        msg += "No output\n"
    if returncode:
        msg += f"Return code: {returncode}"

    yield Message("system", msg)


def get_path_fn(*args, **kwargs) -> Path | None:
    return None


def execute_shell(
    code: str | None,
    args: list[str] | None,
    kwargs: dict[str, str] | None,
    confirm: ConfirmFunc,
Repository: ErikBjare/gptme
File: gptme/tools/rag.py (Python)

Code before:
"""
RAG (Retrieval-Augmented Generation) tool for context-aware assistance.

The RAG tool provides context-aware assistance by indexing and searching project documentation.

.. rubric:: Installation

The RAG tool requires the ``gptme-rag`` CLI to be installed::

    pipx install gptme-rag

.. rubric:: Configuration

Configure RAG in your ``gptme.toml``::

    [rag]
    enabled = true
    post_process = false # Whether to post-process the context with an LLM to extract the most relevant information
    post_process_model = "openai/gpt-4-turbo" # Which model to use for post-processing
    post_process_prompt = "" # Optional prompt to use for post-processing (overrides default prompt)
    workspace_only = true # Whether to only search in the workspace directory, or the whole RAG index
    paths = [] # List of paths to include in the RAG index. Has no effect if workspace_only is true.

.. rubric:: Features

1. Manual Search and Indexing
   - Index project documentation with ``rag_index``
   - Search indexed documents with ``rag_search``
   - Check index status with ``rag_status``

2. Automatic Context Enhancement
   - Retrieves semantically similar documents
   - Preserves conversation flow with hidden context messages
"""

import logging
import shutil
import subprocess
import time
from dataclasses import replace
from functools import lru_cache
from pathlib import Path

from ..config import RagConfig, get_project_config

Review comment (suggestion):
Better default plz (obviously AI generated lol)
`gpt-4o-mini` should do
```suggestion
    post_process_model = "openai/gpt-4o-mini" # Which model to use for post-processing
```

Code after:
"""
RAG (Retrieval-Augmented Generation) tool for context-aware assistance.

The RAG tool provides context-aware assistance by indexing and searching project documentation.

.. rubric:: Installation

The RAG tool requires the ``gptme-rag`` CLI to be installed::

    pipx install gptme-rag

.. rubric:: Configuration

Configure RAG in your ``gptme.toml``::

    [rag]
    enabled = true
    post_process = false # Whether to post-process the context with an LLM to extract the most relevant information
    post_process_model = "openai/gpt-4o-mini" # Which model to use for post-processing
    post_process_prompt = "" # Optional prompt to use for post-processing (overrides default prompt)
    workspace_only = true # Whether to only search in the workspace directory, or the whole RAG index
    paths = [] # List of paths to include in the RAG index. Has no effect if workspace_only is true.

.. rubric:: Features

1. Manual Search and Indexing
   - Index project documentation with ``rag_index``
   - Search indexed documents with ``rag_search``
   - Check index status with ``rag_status``

2. Automatic Context Enhancement
   - Retrieves semantically similar documents
   - Preserves conversation flow with hidden context messages
"""

import logging
import shutil
import subprocess
import time
from dataclasses import replace
from functools import lru_cache
from pathlib import Path

from ..config import RagConfig, get_project_config
Repository: ErikBjare/gptme
File: gptme/tools/screenshot.py (Python)

Code before:

    if os.uname().sysname == "Darwin":  # macOS
        subprocess.run(["screencapture", str(path)], check=True)
    elif os.name == "posix":  # Linux
        # TODO: add support for specifying window/fullscreen?
        if IS_WAYLAND and shutil.which("gnome-screenshot"):
            subprocess.run(["gnome-screenshot", "-f", str(path)], check=True)
            return path
        elif shutil.which("scrot"):
            subprocess.run(["scrot", "--overwrite", str(path)], check=True)  # --focused
            return path
        else:
            raise NotImplementedError("No supported screenshot method available")
    else:
        raise NotImplementedError(
            "Screenshot functionality is only available on macOS and Linux."
        )

    return path


tool = ToolSpec(
    name="screenshot",
    desc="Take a screenshot",
    # TODO: check for this instead of prompting the llm
    instructions="If all you see is a wallpaper, the user may have to allow screen capture in `System Preferences -> Security & Privacy -> Screen Recording`."
    if platform.system() == "Darwin"
    else "",
    functions=[screenshot],
)


Review comment (suggestion):
```suggestion
    assert path.exists()
    return path
```

Code after:

    if path is None:
        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        path = OUTPUT_DIR / f"screenshot_{timestamp}.png"

    path.parent.mkdir(parents=True, exist_ok=True)
    path = path.resolve()

    if IS_MACOS:
        subprocess.run(["screencapture", str(path)], check=True)
        return path
    elif os.name == "posix":
        # TODO: add support for specifying window/fullscreen?
        if shutil.which("gnome-screenshot"):
            subprocess.run(["gnome-screenshot", "-f", str(path)], check=True)
            return path
        elif not IS_WAYLAND and shutil.which("scrot"):
            subprocess.run(["scrot", "--overwrite", str(path)], check=True)
            return path
        else:
            raise NotImplementedError("No supported screenshot method available")
    else:
        raise NotImplementedError(
            "Screenshot functionality is only available on macOS and Linux."
        )


tool = ToolSpec(
    name="screenshot",
    desc="Take a screenshot",
    instructions=INSTRUCTIONS,
    functions=[screenshot],
)
Repository: ErikBjare/gptme
File: tests/test_auto_naming.py (Python)

Code before:
    )

    assert wait_for_event(event_listener, "message_added")

    requests.post(
        f"http://localhost:{port}/api/v2/conversations/{conversation_id}/step",
        json={"session_id": session_id},
    )

    assert wait_for_event(event_listener, "generation_started")

    # Wait for config_changed event
    assert wait_for_event(event_listener, "config_changed", timeout=20)

    # Check that the generated name is contextually relevant
    logdir = get_logs_dir() / conversation_id
    chat_config = ChatConfig.from_logdir(logdir)

    assert chat_config.name is not None, "Expected auto-generated name but got None"

    # Clean thinking tags that some models (e.g., Claude Haiku) output in names
    name = chat_config.name
    name = re.sub(r"</?think>", "", name).strip()
    name = name.lower()

    print(f"Auto-generated display name: '{chat_config.name}' (cleaned: '{name}')")

    # Should contain relevant keywords (being flexible since LLM might use different terms)
    relevant_keywords = [
        "css",
        "div",
        "center",
        "centering",
        "layout",
        "styling",
        "web",
    ]
    has_relevant_content = any(keyword in name for keyword in relevant_keywords)
    assert has_relevant_content, f"Generated name '{chat_config.name}' (cleaned: '{name}') doesn't seem contextually relevant. Expected keywords: {relevant_keywords}"


Review comment (suggestion):
```suggestion
    name = chat_config.name.lower()
    print(f"Auto-generated display name: '{chat_config.name}'")
```

Code after:

    assert wait_for_event(event_listener, "generation_started")

    # Wait for config_changed event
    assert wait_for_event(event_listener, "config_changed", timeout=20)

    # Check that the generated name is contextually relevant
    logdir = get_logs_dir() / conversation_id
    chat_config = ChatConfig.from_logdir(logdir)

    assert chat_config.name is not None, "Expected auto-generated name but got None"
    name = chat_config.name.lower()
    print(f"Auto-generated display name: '{chat_config.name}'")

    # Should contain relevant keywords (being flexible since LLM might use different terms)
    relevant_keywords = [
        "css",
        "div",
        "center",
        "centering",
        "layout",
        "styling",
        "web",
    ]
    has_relevant_content = any(keyword in name for keyword in relevant_keywords)
    assert has_relevant_content, f"Generated name '{chat_config.name}' doesn't seem contextually relevant. Expected keywords: {relevant_keywords}"
Repository: ErikBjare/gptme
File: tests/test_tools_shell.py (Python)

Code before:
        assert matched_cmd is not None, f"Should have matched command: {cmd[:50]}..."


def test_pipe_with_stdin_consuming_command(shell):
    """Test that piping commands that consume stdin doesn't hang (issue #684).

    Commands like gptme that try to read from stdin can cause deadlocks in pipelines
    if stdin isn't properly redirected to /dev/null for the first command.
    """
    # This used to hang before the fix
    ret_code, stdout, stderr = shell.run(
        "echo 'test' | grep test", output=False, timeout=5.0
    )

    assert ret_code == 0
    assert "test" in stdout

    # More complex case: command that would block on stdin if not redirected
    ret_code, stdout, stderr = shell.run(
        "python3 -c 'import sys; print(\"ready\")' | grep ready",
        output=False,
        timeout=5.0,
    )

    assert ret_code == 0
    assert "ready" in stdout


Review comment (bug):
These both actually pass on master already, it's specifically `gptme "/shell gptme '/exit' | grep Assistant"` that fails.

Code after:
        assert matched_cmd is not None, f"Should have matched command: {cmd[:50]}..."


def test_heredoc_in_compound_command(shell):
    """Test that heredocs work correctly in compound commands with &&."""
    # Issue #703: This should not get stuck
    ret, out, err = shell.run(
        """echo "test" && python3 <<'EOF'
print('0')
EOF"""
    )
    assert ret == 0
    assert "test" in out
    assert "0" in out
    # Commented out due to weird error in CI:
    # pytest-cov: Failed to setup subprocess coverage. Environ: {'COV_CORE_DATAFILE': ...} Exception: FileNotFoundError(2, 'No such file or directory')"
    # assert err.strip() == ""


def test_pipe_with_stdin_consuming_command(shell):
    """Test that piping commands that consume stdin doesn't hang (issue #684).

    Reproduces the specific failing case from Erik's comment:
    gptme "/shell gptme '/exit' | grep Assistant"

    The script simulates gptme's behavior:
    - Without stdin redirection: blocks reading from pipe stdin
    - With stdin redirected to /dev/null: prints "Assistant" immediately

    This test would hang without the fix that redirects stdin for the first
    command in a pipeline.
    """
    # Create test script that simulates gptme's stdin behavior
    test_script = """#!/usr/bin/env python3
import sys
import os

# Check if stdin is /dev/null
try:
    stdin_stat = os.fstat(sys.stdin.fileno())
    devnull_stat = os.stat('/dev/null')
    is_devnull = (stdin_stat.st_dev == devnull_stat.st_dev and
                  stdin_stat.st_ino == devnull_stat.st_ino)
except:
    is_devnull = False

if not is_devnull and not sys.stdin.isatty():
    # stdin is a pipe (not /dev/null, not terminal)
    # This would block forever without stdin redirection
    sys.stdin.read(1)
    print("blocked")
Repository: ErikBjare/gptme
File: gptme/util/output_storage.py (Python)

Code before:
    Returns:
        Summary message with reference to saved file
    """
    # Try to extract command info from content
    lines = content.split("\n")
    command_info = None

    for line in lines[:10]:  # Check first 10 lines
        if (
            line.startswith("Ran command:")
            or line.startswith("Executed:")
            or line.startswith("Command:")
        ):
            command_info = line.strip()
            break

    # Detect status from content
    status = "completed"
    if any(
        word in content.lower()
        for word in ["error", "failed", "exception", "traceback"]
    ):
        status = "failed"

    # Create base message
    base_msg = f"[Large tool output removed - {original_tokens} tokens]: Tool execution {status}"

    if command_info:
        base_msg += f" ({command_info})"

    # If no logdir, return simple message
    if not logdir:
        base_msg += "."
        return f"{base_msg} Output was automatically removed due to size to allow conversation continuation."

    # Save the output
    _, saved_path = save_large_output(
        content=content,
        logdir=logdir,
        output_type=tool_name,
        command_info=command_info,
        original_tokens=original_tokens,
    )

    # Return message with file reference
    return f"{base_msg}. Full output saved to: {saved_path}\nYou can read or grep this file if needed."


Review comment (suggestion):
```suggestion
    base_msg = f"[Large tool output removed - {original_tokens} tokens]"
```

Code after:
    Returns:
        Summary message with reference to saved file
    """
    # Try to extract command info from content
    lines = content.split("\n")
    command_info = None

    for line in lines[:10]:  # Check first 10 lines
        if (
            line.startswith("Ran command:")
            or line.startswith("Executed:")
            or line.startswith("Command:")
        ):
            command_info = line.strip()
            break

    # Create base message without status inference
    base_msg = f"[Large tool output removed - {original_tokens} tokens]"

    if command_info:
        base_msg += f" ({command_info})"

    # If no logdir, return simple message
    if not logdir:
        base_msg += "."
        return f"{base_msg} Output was automatically removed due to size to allow conversation continuation."

    # Save the output
    _, saved_path = save_large_output(
        content=content,
        logdir=logdir,
        output_type=tool_name,
        command_info=command_info,
        original_tokens=original_tokens,
    )

    # Return message with file reference
    return f"{base_msg}. Full output saved to: {saved_path}\nYou can read or grep this file if needed."
Repository: ErikBjare/gptme
File: gptme/util/output_storage.py (Python)

Code before:
    Returns:
        Summary message with reference to saved file
    """
    # Try to extract command info from content
    lines = content.split("\n")
    command_info = None

    for line in lines[:10]:  # Check first 10 lines
        if (
            line.startswith("Ran command:")
            or line.startswith("Executed:")
            or line.startswith("Command:")
        ):
            command_info = line.strip()
            break

    # Detect status from content
    status = "completed"
    if any(
        word in content.lower()
        for word in ["error", "failed", "exception", "traceback"]
    ):
        status = "failed"

    # Create base message with status
    base_msg = f"[Large tool output removed - {original_tokens} tokens] ({status})"

    if command_info:
        base_msg += f" ({command_info})"

    # If no logdir, return simple message
    if not logdir:
        base_msg += "."
        return f"{base_msg} Output was automatically removed due to size to allow conversation continuation."

    # Save the output
    _, saved_path = save_large_output(
        content=content,
        logdir=logdir,
        output_type=tool_name,
        command_info=command_info,
        original_tokens=original_tokens,
    )

    # Return message with file reference
    return f"{base_msg}. Full output saved to: {saved_path}\nYou can read or grep this file if needed."


Review comment (suggestion):
```suggestion

    # Create base message with status
    base_msg = f"[Large tool output removed - {original_tokens} tokens]"
```

Code after:
    Returns:
        Summary message with reference to saved file
    """
    # Try to extract command info from content
    lines = content.split("\n")
    command_info = None

    for line in lines[:10]:  # Check first 10 lines
        if (
            line.startswith("Ran command:")
            or line.startswith("Executed:")
            or line.startswith("Command:")
        ):
            command_info = line.strip()
            break

    # Create base message without status inference
    base_msg = f"[Large tool output removed - {original_tokens} tokens]"

    if command_info:
        base_msg += f" ({command_info})"

    # If no logdir, return simple message
    if not logdir:
        base_msg += "."
        return f"{base_msg} Output was automatically removed due to size to allow conversation continuation."

    # Save the output
    _, saved_path = save_large_output(
        content=content,
        logdir=logdir,
        output_type=tool_name,
        command_info=command_info,
        original_tokens=original_tokens,
    )

    # Return message with file reference
    return f"{base_msg}. Full output saved to: {saved_path}\nYou can read or grep this file if needed."
Repository: FlowiseAI/Flowise
File: packages/components/credentials/GoogleSearchApi.credential.ts (TypeScript)

Code before:
import { INodeParams, INodeCredential } from '../src/Interface'

class GoogleSearchApi implements INodeCredential {
    label: string
    name: string
    version: number
    description: string
    inputs: INodeParams[]

    constructor() {
        this.label = 'Google Custom Search API'
        this.name = 'googleCustomSearchApi'
        this.version = 1.0
        this.inputs = [
            {
                label: 'Google Custom Search Api Key',
                name: 'googleCustomSearchApiKey',
                type: 'password'
            },
            {
                label: 'Programmable Search Engine ID',
                name: 'googleCustomSearchApiId',
                type: 'string'
            }
        ]
    }
}

module.exports = { credClass: GoogleSearchApi }


Review comment (security):
can we add a description like [this](https://github.com/FlowiseAI/Flowise/blob/main/packages/components/credentials/ZepMemoryApi.credential.ts#L14) to let user know how and where to get these credentials?

Code after:
import { INodeParams, INodeCredential } from '../src/Interface'

class GoogleSearchApi implements INodeCredential {
    label: string
    name: string
    version: number
    description: string
    inputs: INodeParams[]

    constructor() {
        this.label = 'Google Custom Search API'
        this.name = 'googleCustomSearchApi'
        this.version = 1.0
        this.description =
            'Please refer to the <a target="_blank" href="https://console.cloud.google.com/apis/credentials">Google Cloud Console</a> for instructions on how to create an API key, and visit the <a target="_blank" href="https://programmablesearchengine.google.com/controlpanel/create">Search Engine Creation page</a> to learn how to generate your Search Engine ID.'
        this.inputs = [
            {
                label: 'Google Custom Search Api Key',
                name: 'googleCustomSearchApiKey',
                type: 'password'
            },
            {
                label: 'Programmable Search Engine ID',
                name: 'googleCustomSearchApiId',
                type: 'string'
            }
        ]
    }
}

module.exports = { credClass: GoogleSearchApi }