OpenHandsにUSBデバイスを触らせる



# 環境
- Ubuntu22.04

# USBデバイス追加

コンフィグを書き換える必要があるのでcloneします．

```bash
git clone git@github.com:All-Hands-AI/OpenHands.git
```

```OpenHands/openhands/core/config/sandbox_config.py```でサンドボックスコンテナの設定ができます

```docker_runtime_kwargs```に欲しいオプションを追加していく
```OpenHands/openhands/core/config/sandbox_config.py
    docker_runtime_kwargs: dict | None = Field(default=None)
```

例えば/dev/ttyACM0を追加したければ以下のように書き換える．
```OpenHands/openhands/core/config/sandbox_config.py
    # docker_runtime_kwargs: dict | None = Field(default=None)
    docker_runtime_kwargs: dict = {
        "devices": ["/dev/ttyACM0:/dev/ttyACM0"]
    }
```


実行
```bash
cd OpenHands
docker compose up
```

認識しています．
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/260065/7011234a-b80c-489b-9e45-654c381630ed.png)


# GUIアプリを実行させる

```environment```や```volumes```は別で定義されているため，なにか追加したい場合は以下のように```OpenHands/openhands/runtime/impl/docker/docker_runtime.py```を編集する必要があります．

self.docker_client.containers.runの前に追加しましょう

```OpenHands/openhands/runtime/impl/docker/docker_runtime.py
        import os
        environment['DISPLAY'] = os.environ.get('DISPLAY', ':0')
        volumes = volumes or {}
        volumes['/tmp/.X11-unix'] = {'bind': '/tmp/.X11-unix', 'mode': 'rw'}

        try:
            self.container = self.docker_client.containers.run(
                self.runtime_container_image,
                command=command,
                network_mode=network_mode,
                ports=port_mapping,
                working_dir='/openhands/code/',  # do not change this!
                name=self.container_name,
                detach=True,
                environment=environment,
                volumes=volumes,
                device_requests=(
                    [docker.types.DeviceRequest(capabilities=[['gpu']], count=-1)]
                    if self.config.sandbox.enable_gpu
                    else None
                ),
                **(self.config.sandbox.docker_runtime_kwargs or {}),
            )
```

docker-composeにも```--env="DISPLAY" --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw"```を追加しましょう．

```OpenHands/docker-compose.yml
services:
  openhands:
    build:
      context: ./
      dockerfile: ./containers/app/Dockerfile
    image: openhands:latest
    container_name: openhands-app-${DATE:-}
    environment:
      - SANDBOX_RUNTIME_CONTAINER_IMAGE=${SANDBOX_RUNTIME_CONTAINER_IMAGE:-docker.all-hands.dev/all-hands-ai/runtime:0.28-nikolaik}
      #- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of openhands-state for this user
      - WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
      - DISPLAY=${DISPLAY:-:0} # 追加！
    ports:
      - "3000:3000"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ~/.openhands-state:/.openhands-state
      - ${WORKSPACE_BASE:-$PWD/workspace}:/opt/workspace_base
      - /tmp/.X11-unix:/tmp/.X11-unix:rw # 追加！
    pull_policy: build
    stdin_open: true
    tty: true

```

実行
```bash
cd OpenHands
docker compose up
```

できました
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/260065/1f9c6be3-87cd-45e9-b73a-84bb765174ea.png)


# トラブルシューティング

環境によっては以下のように動かないかも
```bash
openhands-app-  | 12:04:28 - openhands:INFO: standalone_conversation_manager.py:299 - found_local_agent_loop:fba2c58c9a8d4e2f8ca903b3ef9494c3
openhands-app-  | 12:04:29 - openhands:INFO: docker_runtime.py:155 - [runtime fba2c58c9a8d4e2f8ca903b3ef9494c3] Waiting for client to become ready at http://host.docker.internal:30239...
```

environmentに```SANDBOX_RUNTIME_BINDING_ADDRESS="0.0.0.0"```を追加すると動く
[参考] https://github.com/All-Hands-AI/OpenHands/issues/7152#issuecomment-2707578503

```OpenHands/docker-compose.yml
services:
  openhands:
    build:
      context: ./
      dockerfile: ./containers/app/Dockerfile
    image: openhands:latest
    container_name: openhands-app-${DATE:-}
    environment:
      - SANDBOX_RUNTIME_BINDING_ADDRESS="0.0.0.0"  #追加！
      - SANDBOX_RUNTIME_CONTAINER_IMAGE=${SANDBOX_RUNTIME_CONTAINER_IMAGE:-docker.all-hands.dev/all-hands-ai/runtime:0.28-nikolaik}
      #- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of openhands-state for this user
      - WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
    ports:
      - "3000:3000"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ~/.openhands-state:/.openhands-state
      - ${WORKSPACE_BASE:-$PWD/workspace}:/opt/workspace_base
    pull_policy: build
    stdin_open: true
    tty: true

```