Repository: AcademySoftwareFoundation/OpenCue
Pull request #1751: Improve sandbox install script, pyproject metadata, and update documentation
Author: login: ramonfigueiredo
Created: 2025-05-22T22:06:15Z | Merged: 2025-05-22T23:06:01Z

- Updated `sandbox/install-client-sources.sh` to:
  - Add comments and safety checks
  - Automatically tag the version based on VERSION.in
  - Support local macOS builds using grpcio~=1.71.0 and grpcio-tools~=1.71.0
- Updated `cuesubmit/pyproject.toml` to fix script entry point via [project.scripts]
  - Required to build cuesubmit and launch it via `cuesubmit &`
- Revised `requirements.txt` to use grpcio~=1.71.0 and grpcio-tools~=1.71.0 to fix RuntimeError
  - grpc version 1.69.0 is too old; upgrade to grpcio~=1.71.0
  - Fix RuntimeError: The grpc package installed is at version 1.69.0, but the generated code in `comment_pb2_grpc.py` depends on `grpcio>=1.71.0`
- Rewrote `sandbox/README.md` with clearer step-by-step setup instructions, including use of `sandbox/install-client-sources.sh`
- Updated main `README.md` to reflect improvements in sandbox usage and mention quick install and testing instructions

These changes make it easier to locally build and test OpenCue Python clients from source, especially on macOS and with virtual environments.

**Link the Issue(s) this Pull Request is related to.**
- #1749 
- #1750 


Diff:
diff --git a/README.md b/README.md
index ab668edc6..d11749ca1 100644
--- a/README.md
+++ b/README.md
@@ -42,6 +42,23 @@ For more information on installing, using, and administering OpenCue, visit
 
 Watch YouTube videos on the [OpenCue Playlist](https://www.youtube.com/playlist?list=PL9dZxafYCWmzSBEwVT2AQinmZolYqBzdp) of the Academy Software Foundation (ASWF) to learn more.
 
+# Quick installation and tests
+
+Read the [OpenCue sandbox documentation](https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/sandbox/README.md) 
+to learn how to set up a local OpenCue environment.
+
+- The sandbox environment offers an easy way to run a test OpenCue deployment locally, with all components running in 
+separate Docker containers or Python virtual environments.
+- It is ideal for small tests, development work, and for those new to OpenCue who want a simple setup for 
+experimentation and learning.
+
+To learn how to run the sandbox environment, see https://www.opencue.io/docs/quick-starts/.
+
+# OpenCue full installation
+
+Guides for system admins deploying OpenCue components and installing dependencies are available in the 
+[OpenCue documentation](https://www.opencue.io/docs/getting-started/).
+
 # Meeting notes
 
 Starting from May 2024, all Opencue meeting notes are stored on the [Opencue Confluence page](http://wiki.aswf.io/display/OPENCUE/OpenCue+Home).
diff --git a/cuesubmit/pyproject.toml b/cuesubmit/pyproject.toml
index 01217fc72..570f20b45 100644
--- a/cuesubmit/pyproject.toml
+++ b/cuesubmit/pyproject.toml
@@ -24,7 +24,7 @@ default-version = "0.0.0"
 [tool.hatch.build.targets.wheel]
 packages = ["cuesubmit"]
 
-[console_scripts]
+[project.scripts]
 cuesubmit = "cuesubmit.__main__:main"
 
 # --- Pytest configuration ---
diff --git a/requirements.txt b/requirements.txt
index cc32a28bc..87686ffa0 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,8 +1,8 @@
 future==1.0.0
 grpcio==1.48.2;python_version<"3.7"
 grpcio-tools==1.48.2;python_version<"3.7"
-grpcio==1.69.0;python_version>="3.7"
-grpcio-tools==1.69.0;python_version>="3.7"
+grpcio~=1.71.0;python_version>="3.7"
+grpcio-tools~=1.71.0;python_version>="3.7"
 mock==2.0.0
 packaging==20.9;python_version<"3.7"
 packaging==24.1;python_version>="3.7"
diff --git a/sandbox/README.md b/sandbox/README.md
index e2cadb902..b8fe7b601 100644
--- a/sandbox/README.md
+++ b/sandbox/README.md
@@ -1,16 +1,165 @@
 # OpenCue sandbox environment
 
-The sandbox environment provides a way to run a test OpenCue deployment. You can use the sandbox
-environment to run small tests or development work. The sandbox environment runs OpenCue components
-in separate Docker containers on your local machine.
+The sandbox environment offers an easy way to run a test OpenCue deployment locally, with all components running in 
+separate Docker containers or Python virtual environments. It is ideal for small tests, development work, and for 
+those new to OpenCue who want a simple setup for experimentation and learning.
 
 To learn how to run the sandbox environment, see
 https://www.opencue.io/docs/quick-starts/.
 
+## Usage example
+
+If you don’t already have a recent local copy of the OpenCue source code, you must do one of the following:
+
+- Download and unzip the [OpenCue source code ZIP file](https://github.com/AcademySoftwareFoundation/OpenCue/archive/master.zip).
+- If you have the git command installed on your machine, you can clone the repository:
+
+```bash
+git clone https://github.com/AcademySoftwareFoundation/OpenCue.git
+```
+
+For developers, you can also use the following commands to set up a local copy of the OpenCue source code:
+
+- Fork the [Opencue repository](https://github.com/AcademySoftwareFoundation/OpenCue) using your GitHub account.
+- Clone your forked repository:
+```bash
+git clone https://github.com/<username>/OpenCue.git
+```
+
+### 1. Deploying the OpenCue Sandbox Environment
+
+- Run the services: DB, Flyway, Cuebot, and RQD
+  - A PostgreSQL database
+  - A Flyway database migration tool
+  - A Cuebot server
+  - An RQD rendering server
+
+In one terminal, run the following commands:
+
+- Create required directories for RQD logs and shots:
+
+```bash
+mkdir -p /tmp/rqd/logs /tmp/rqd/shots
+```
+
+- Change to the root of the OpenCue source code directory
+
+```bash
+cd OpenCue
+```
+
+- Build the Cuebot container from source
+
+**Note:** Make sure [Docker](https://www.docker.com/) is installed and running on your machine.
+
+```bash
+docker build -t opencue/cuebot -f cuebot/Dockerfile .
+```
+
+- Deploy the sandbox environment
+  - This command will start the services (db, flyway, cuebot, rqd) in the background and create a network for them to 
+  communicate with each other.
+
+```bash
+docker compose up
+```
+
+**Notes:** 
+- Use `docker compose up -d` to run the services in detached mode.
+- Use `docker compose down` to stop the services and remove the network.
+- Use `docker compose logs` to view the logs of the services.
+- Use `docker compose ps` to view the status of the services.
+- Use `docker compose exec <service> bash` to open a shell in the container of the specified service.
+- Use `docker compose exec <service> <command>` to run a command in the container of the specified service.
+- Use `docker compose down --volumes` to remove the volumes created by the services.
+- Use `docker compose down --rmi all` to remove all images created by the services. 
+
+### 2. Installing the OpenCue Client Packages
+
+In a second terminal, run the following commands:
+
+- Make sure you are in the root of the OpenCue source code directory
+
+```bash
+cd OpenCue
+```
+
+- Create a virtual environment for the Python packages
+
+```bash
+python3 -m venv sandbox-venv
+```
+
+- Activate the virtual environment
+
+```bash
+source sandbox-venv/bin/activate
+```
+
+- Upgrade pip to the latest version
+
+```bash
+pip install --upgrade pip
+```
+
+- Install the required Python packages
+
+```bash
+pip install -r requirements.txt
+pip install -r requirements_gui.txt
+```
+
+- Install the OpenCue Python client libraries from source
+  - This option is mostly used by developers that contribute to the OpenCue project.
+  - It is recommended if you want to test the latest changes in the OpenCue source code.
+  - To install Opencue from source, run:
+
+```bash
+./sandbox/install-client-sources.sh
+```
+
+**Note:** The latest version of the OpenCue source code might include changes that are incompatible with the prebuilt 
+OpenCue images of Cuebot and RQD on Docker Hub used in the sandbox environment.
+
+Alternatively, you can use the script `./sandbox/install-client-archive.sh` to download, extract, and install specified 
+OpenCue client packages for a given release version from GitHub. To learn how to run the sandbox environment, see
+https://www.opencue.io/docs/quick-starts/.
+- To install the latest versions of the OpenCue client packages, you must configure the installation script with the 
+version number. 
+- The script `sandbox/get-latest-release-tag.sh` will automatically fetch this for you, but you can also look up the 
+version numbers for OpenCue releases on GitHub.
+
+```bash
+export VERSION=$(sandbox/get-latest-release-tag.sh)
+``` 
+
+### 3. Testing the Sandbox Environment
+
+To test the sandbox environment, run the following commands inside the Python virtual environment:
+
+- To verify the successful installation and connection between the client packages and sandbox, list the hosts in the 
+sandbox environment:
+
+```bash
+cueadmin -lh
+```
+
+- Launch the CueGUI (Cuetopia/CueCommander) app for monitoring and controlling jobs:
+
+```bash
+cuegui &
+```
+
+- Launch the CueSubmit app for submitting jobs:
+
+```bash
+cuesubmit &
+```
+
 ## Monitoring
 
 To get started with monitoring there is also an additional Docker compose file which sets up
 monitoring for key services.
 
 To learn how to run the sandbox environment with monitoring,
-see https://www.opencue.io/docs/other-guides/monitoring-with-prometheus-loki-and-grafana/.
+see https://www.opencue.io/docs/other-guides/monitoring-with-prometheus-loki-and-grafana/.
\ No newline at end of file
diff --git a/sandbox/install-client-sources.sh b/sandbox/install-client-sources.sh
index 603aa5c34..3b165ca3d 100755
--- a/sandbox/install-client-sources.sh
+++ b/sandbox/install-client-sources.sh
@@ -1,5 +1,12 @@
 #!/bin/bash
 
+# Installs OpenCue Python client libraries from source.
+#
+# Read the [OpenCue sandbox documentation](https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/sandbox/README.md)
+# to learn how to set up a local OpenCue environment.
+
+# This script should be run from the root of the OpenCue repository.
+
 set -e
 
 # Install all client packages.
@@ -8,6 +15,15 @@ then
   echo "Installing pre-built cuebot package"
   pip install ${OPENCUE_PROTO_PACKAGE_PATH}
 else
+  VERSION_TAG="$(cat VERSION.in | tr -d '[:space:]')"
+  # Check if the tag already exists
+  if git tag --list "$VERSION_TAG" | grep -q "^$VERSION_TAG$"; then
+    echo "Tag $VERSION_TAG already exists."
+  else
+    echo "Creating tag $VERSION_TAG"
+    git tag "$VERSION_TAG"
+  fi
+  # Build the proto package
   pip install proto/
 fi
 pip install pycue/ pyoutline/ cueadmin/ cuesubmit/ cuegui/


Reviews:
user: login: DiegoTavares
body: Pre-approved with minor changes
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-22T22:41:24Z
user: login: ramonfigueiredo
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T22:58:28Z

Review comments:
diff_hunk: @@ -1,8 +1,8 @@
 future==1.0.0
 grpcio==1.48.2;python_version<"3.7"
 grpcio-tools==1.48.2;python_version<"3.7"
-grpcio==1.69.0;python_version>="3.7"
-grpcio-tools==1.69.0;python_version>="3.7"
+grpcio>=1.71.0;python_version>="3.7"
+grpcio-tools>=1.71.0;python_version>="3.7"
path: requirements.txt
user: login: DiegoTavares
body: This can be dangerous. If grpc released and incompatible version 2.0 for example, it would be picked up and crash.
I suggest Something like `~=1.69`, that will accept anything above 1.69 and bellow 2.0.
created_at: 2025-05-22T22:40:43Z
author_association: COLLABORATOR
diff_hunk: @@ -1,8 +1,8 @@
 future==1.0.0
 grpcio==1.48.2;python_version<"3.7"
 grpcio-tools==1.48.2;python_version<"3.7"
-grpcio==1.69.0;python_version>="3.7"
-grpcio-tools==1.69.0;python_version>="3.7"
+grpcio>=1.71.0;python_version>="3.7"
+grpcio-tools>=1.71.0;python_version>="3.7"
path: requirements.txt
user: login: ramonfigueiredo
body: Done. The correct is: 

```yaml
grpcio~=1.71.0;python_version>="3.7"
grpcio-tools~=1.71.0;python_version>="3.7"
```

To prevent the error: 

```bash
RuntimeError: The grpc package installed is at version 1.69.0, but the generated code in comment_pb2_grpc.py depends on grpcio>=1.71.0. Please upgrade your grpc module to grpcio>=1.71.0 or downgrade your generated code using grpcio-tools<=1.69.0.
````

Update grpcio and grpcio-tools to ~1.71.0 for Python >=3.7
created_at: 2025-05-22T22:58:28Z
author_association: COLLABORATOR

PR comments:
user: login: ramonfigueiredo
created_at: 2025-05-22T22:17:35Z
body: @DiegoTavares @lithorus 

FYI ...

This PR pull request: 
- Fix the script `sandbox/install-client-sources.sh` (Opencue sandbox installation)
- Fix the Cuesubmit binary installation to open the Cuesubmit using `cuesubmit &`. 
   - It was't working because the pyproject.toml was using the `[console_scripts]` instead of `[project.scripts]`
- Rewrote `sandbox/README.md` with clearer step-by-step setup instructions, including use of `sandbox/install-client-sources.sh`
- Updated main README.md to reflect improvements in sandbox usage and mention quick install and testing instructions 
author_association: COLLABORATOR
user: login: ramonfigueiredo
created_at: 2025-05-22T23:10:22Z
body: To whom it may concern (Opencue developers, for example):

This sandbox/README.md file shows how to use the OpenCue sandbox environment: https://github.com/AcademySoftwareFoundation/OpenCue/tree/master/sandbox

Steps:
1. Deploying the OpenCue Sandbox Environment
2. Installing the OpenCue Client Packages
3. Testing the Sandbox Environment

- The sandbox environment offers an easy way to run a test OpenCue deployment locally, with all components running in separate Docker containers or Python virtual environments.
- It is ideal for small tests, development work, and for those new to OpenCue who want a simple setup for experimentation and learning.
author_association: COLLABORATOR