Repository: gradio-app/gradio

Issue:
Install `gradio_client` locally in CI
I think we should install `gradio_client` in editable mode in CI, so the local version of `gradio_client` is used by `gradio` for Ci checks.

Currently `gradio` relies on the version of `gradio_client` available on pypi, which is correct. However, if we make changes to `gradio_client` that impact `gradio` CI checks can fail giving as those changes won't be used by the gradio version used in CI.

Currently this is annoying but understandable because the release processes for `gradio` and `gradio_client` are independent, we can release changes to one without the other even if said changes exist on main. When we move to a unified release process for the whole repository (#4851) any changes in gradio + gradio client will be released together (along with any other changes on main). This will mean that they can't get out of sync and an editable install of all python packages is safe (and desired because tests should reflect what will happen when the libraries are published as closely as possible).

The way the release process will work is that we look for dependency changes, if `gradio_client` is bumped and `gradio` depends on it, then the version of `gradio_client` in the requirements file will be bumped (if the current version doesn't satisfy the semver range of the dependency in the requirements file). This is also how JS dependencies will work in the future (if we have libraries that depend on libraries, which we will shortly).

Would love to get people's thoughts on this, and any context for the original decision to _not_ install the client in editable mode.

cc @freddyaboulton @abidlabs 


Discussion:
Can you explain how this works in practice:

> if gradio_client is bumped and gradio depends on it, then the version of gradio_client in the requirements file will be bumped (if the current version doesn't satisfy the semver range of the dependency in the requirements file)

If we do `gradio_client>=0.2.7` in gradio's requirements.txt then the version of `gradio_client` will always match the semver range of the `gradio` requirements file.
The reason for doing this was exactly what you said here: 

> Currently this is annoying but understandable because the release processes for gradio and gradio_client are independent, we can release changes to one without the other even if said changes exist on main

The risk being we release a version of `gradio` without releasing a corresponding version of `gradio_client`. So `gradio` passes all of the checks on CI but is actually broken if someone was to download from pypi.

As far as I understand, the proposed release process does not fix this. Its still possible for us to make changes to `gradio_client` that are merged into `main` (but NOT released on pypi). 

We could then make a release of `gradio` that depends on these changes so would be broken if downloaded from pypi
Unless the proposed changes also mean that every release of `gradio` includes a release of `gradio_client`? 

If we could do the following workflow, then I think it would be okay. Say we want to do a release of `gradio` and there have been some changes to the `gradio_client` on `main` that have not been deployed to pypi, then the following steps MUST happen:

1. `gradio_client` should be released first
2. In the `requirements.txt` of `gradio`, the version of `gradio_client` dependency should be incremented
3. Then, `gradio` should be released

Okay @abidlabs and I caught up and discussed some of the concerns/ confusions. I'll summarise here:

Firstly, with the incoming unification of release, it will not be possible to release an individual library if there are changes in main that include a changeset. The whole repo (every public package in it) gets released together.

> Unless the proposed changes also mean that every release of gradio includes a release of gradio_client?

So the answer to this is yes. Assuming there were both changes to `gradio_client` and `gradio` they will be released together. If there are no changes to `gradio_client` but there are change to `gradio` then only gradio will be bumped and released.

I'll go through some scenarios to explain the flow more clearly and why this is 'safe'.

**We update `gradio` and not `gradio_client`**

The current `gradio` has a dependency on `gradio_client>=0.2.9`

- We make PRs as normal and instead of adding a changelog entry we create a changeset file detailing the changes + version.
- When PRs containing a changeset file is merged to main one of two things will happen
  - if there is no 'release new version' PR, one is created and the version of gradio is bumped according to the version in the changeset and a changelog is generated.
  - if there is already a 'release new version' PR then the PR is updated adding the contents of the additional changeset(s) to the changelog and bumping the version further if needed. If one change is `patch`, and seven others are `minor` gradio will be bumped by a single `minor` version lets say from `3.50` -> `3.51`.
- The `gradio` `requirements.txt` will be left as it is (reflecting the current version of `gradio_client`) as there are no changes to `gradio_client`.
- When we merge the 'release new version' PR into `main` the new version of `gradio` is released on pypi.

**We update `gradio` _and_ `gradio_client`**

The current `gradio` has a dependency on `gradio_client>=0.2.9`

- We make PRs as normal and instead of adding a changelog entry we create a changeset file detailing the changes + version.
- When PRs containing a changeset file is merged to main, the principles are the same as above.
- In this scenario we make changes to both `gradio` and `gradio_client` with the appropriate changeset files.
- Since the version of `gradio_client` has been incremented and all packages are release in lockstep, we need to increment the version of `gradio_client` that `gradio` depends upon in `gradio`'s `requirements.txt`. This is true for any local package that depends on another local package. If they depend on one another then we always increment the dependent to match the new version of the dependency.
- When we merge the 'release new version' PR into `main` new versions of both `gradio_client` and `gradio` are released in topological order (dependency -> dependent), if anything fails we abort, ensuring we don't release `gradio` without the appropriate `gradio_client` version. Our graph is small and simple here so this isn't very difficult to figure out.

So if the current version of `gradio` depends on `gradio_client>=0.2.9` and we bumped `gradio_client` to `0.3.0`. Then we automatically update `gradio` to depend on `gradio_client>=0.30` (by updating the `requirements.txt`). Even though the previous semver ranged matched, we always bump the version because that is what we test against in CI, the current local version (which is this 0.3.0). There may not be breaking changes and it may be fine but we don't have any confidence of that, so we we always keep them in sync.


**We update `gradio_client` but not `gradio`**

The current `gradio` has a dependency on `gradio_client>=0.2.9`

- We make PRs as normal and instead of adding a changelog entry we create a changeset file detailing the changes + version.
- When PRs containing a changeset file is merged to main, the principles are the same as above.
- In this scenario we make changes to both `gradio_client` with the appropriate changeset files.
- this is identical to the above. We see that the `gradio_client` version has been released and we automatically bump the version in `gradio`. We do this to keep things in sync. We have pretty much 0 knowledge about mismatched version and want to avoid that just to be safe, even though this probably won't cause any issue. The version are really just an implementation detail, it is very to think about it like 'gradio depends on the local version of `gradio_client`, we just fix up the requirements file to reflect what that will mean when published. We want whatever gets published to be _exactly_ what we last ran CI on (on release), while it is unlikely to cause any issues, it is safer to keep them synced. "local gradio version" converted for publishing just means "whatever the current version for that dep is", we could even consider a hard pin `==`.


I hope this helps clear things up.
---

Now one concern that @abidlabs raised is this:

> 1. gradio_client should be released first
> 2. In the requirements.txt of gradio, the version of gradio_client dependency should be incremented
> 3. Then, gradio should be released

The reason that the version bumps are potentially problematic is mainly in the 'release new version' PR. In this PR when we install gradio (with its updated `requirements.txt` file), it will try to install a version of `gradio_client` that doesn't exist on pypi. 

However, it turns out that this isn't an issue, as long as we install `gradio_client` in editable mode _first_ and then `gradio` then the concrete version of `gradio_client` that `gradio` will grab is the version installed from source. We checked this and it works fine.

This is the only place the version could cause a potential issue because the version are not bumped when every changeset PR is merged to main, they are only bumped in the 'release new versions' PR. And when that is merged into main the version actually get released. There may be a brief window of a few seconds when someone working locally could have issues but that is about the worst thing that can happen. If they try again 2 seconds later (or just install `gradio_client` in editable mode first), they wont have an issue.

--

So to summarise, i think the above solution for installing both `gradio` and `gradio_client` in editable mode (even if the version is not available on pypi), the automatic dependency updates, and the unified release process will put us in a place where our CI is a near perfect reflection of what gradio will actually behave like _when it is published_ which I think is pretty much ideal.

---

Apologies for the wall of text but i didn't want to leave any thing to chance. I hope this was helpful and just let me know if you have any other questions or concerns.







Amazing @pngwn thanks for walking through all of the cases. LGTM 
Thank you so much for the detailed summary @pngwn ! 

> we could even consider a hard pin

Yes I think how you describe the version bumps of `gradio_client` in `gradio` are equivalent to a hard pin. 
Yeah, they are basically.

Resolving patch:
diff --git a/client/python/gradio_client/client.py b/client/python/gradio_client/client.py
--- a/client/python/gradio_client/client.py
+++ b/client/python/gradio_client/client.py
@@ -12,7 +12,7 @@
 import urllib.parse
 import uuid
 import warnings
-from concurrent.futures import Future, TimeoutError
+from concurrent.futures import Future
 from datetime import datetime
 from pathlib import Path
 from threading import Lock
@@ -283,6 +283,11 @@ def predict(
             client.predict(5, "add", 4, api_name="/predict")
             >> 9.0
         """
+        inferred_fn_index = self._infer_fn_index(api_name, fn_index)
+        if self.endpoints[inferred_fn_index].is_continuous:
+            raise ValueError(
+                "Cannot call predict on this function as it may run forever. Use submit instead."
+            )
         return self.submit(*args, api_name=api_name, fn_index=fn_index).result()
 
     def submit(
@@ -761,6 +766,7 @@ def __init__(self, client: Client, fn_index: int, dependency: dict):
         self.input_component_types = []
         self.output_component_types = []
         self.root_url = client.src + "/" if not client.src.endswith("/") else client.src
+        self.is_continuous = dependency.get("types", {}).get("continuous", False)
         try:
             # Only a real API endpoint if backend_fn is True (so not just a frontend function), serializers are valid,
             # and api_name is not False (meaning that the developer has explicitly disabled the API endpoint)
@@ -1103,7 +1109,7 @@ def result(self, timeout: float | None = None) -> Any:
         Parameters:
             timeout: The number of seconds to wait for the result if the future isn't done. If None, then there is no limit on the wait time.
         Returns:
-            The result of the call that the future represents.
+            The result of the call that the future represents. For generator functions, it will return the final iteration.
         Example:
             from gradio_client import Client
             calculator = Client(src="gradio/calculator")
@@ -1111,25 +1117,7 @@ def result(self, timeout: float | None = None) -> Any:
             job.result(timeout=5)
             >> 9
         """
-        if self.communicator:
-            timeout = timeout or float("inf")
-            if self.future._exception:  # type: ignore
-                raise self.future._exception  # type: ignore
-            with self.communicator.lock:
-                if self.communicator.job.outputs:
-                    return self.communicator.job.outputs[0]
-            start = datetime.now()
-            while True:
-                if (datetime.now() - start).seconds > timeout:
-                    raise TimeoutError()
-                if self.future._exception:  # type: ignore
-                    raise self.future._exception  # type: ignore
-                with self.communicator.lock:
-                    if self.communicator.job.outputs:
-                        return self.communicator.job.outputs[0]
-                time.sleep(0.01)
-        else:
-            return super().result(timeout=timeout)
+        return super().result(timeout=timeout)
 
     def outputs(self) -> list[tuple | Any]:
         """


Test patch:
diff --git a/client/python/test/conftest.py b/client/python/test/conftest.py
--- a/client/python/test/conftest.py
+++ b/client/python/test/conftest.py
@@ -178,6 +178,32 @@ def show(n):
     return demo.queue()
 
 
+@pytest.fixture
+def count_generator_demo_exception():
+    def count(n):
+        for i in range(int(n)):
+            time.sleep(0.1)
+            if i == 5:
+                raise ValueError("Oh no!")
+            yield i
+
+    def show(n):
+        return str(list(range(int(n))))
+
+    with gr.Blocks() as demo:
+        with gr.Column():
+            num = gr.Number(value=10)
+            with gr.Row():
+                count_btn = gr.Button("Count")
+                count_forever = gr.Button("Count forever")
+        with gr.Column():
+            out = gr.Textbox()
+
+        count_btn.click(count, num, out, api_name="count")
+        count_forever.click(show, num, out, api_name="count_forever", every=3)
+    return demo.queue()
+
+
 @pytest.fixture
 def file_io_demo():
     demo = gr.Interface(
diff --git a/client/python/test/test_client.py b/client/python/test/test_client.py
--- a/client/python/test/test_client.py
+++ b/client/python/test/test_client.py
@@ -133,6 +133,17 @@ def test_intermediate_outputs(self, count_generator_demo):
                 outputs.append(o)
             assert outputs == [str(i) for i in range(3)]
 
+    @pytest.mark.flaky
+    def test_intermediate_outputs_with_exception(self, count_generator_demo_exception):
+        with connect(count_generator_demo_exception) as client:
+            with pytest.raises(Exception):
+                client.predict(7, api_name="/count")
+
+            with pytest.raises(
+                ValueError, match="Cannot call predict on this function"
+            ):
+                client.predict(5, api_name="/count_forever")
+
     def test_break_in_loop_if_error(self, calculator_demo):
         with connect(calculator_demo) as client:
             job = client.submit("foo", "add", 4, fn_index=0)
@@ -229,8 +240,9 @@ def test_cancel_from_client_queued(self, cancel_from_client_demo):
                         job.cancel()
                         break
                     time.sleep(0.5)
-            # Result for iterative jobs is always the first result
-            assert job.result() == 0
+            # Result for iterative jobs will raise there is an exception
+            with pytest.raises(CancelledError):
+                job.result()
             # The whole prediction takes 10 seconds to run
             # and does not iterate. So this tests that we can cancel
             # halfway through a prediction
diff --git a/scripts/install_test_requirements.sh b/scripts/install_test_requirements.sh
--- a/scripts/install_test_requirements.sh
+++ b/scripts/install_test_requirements.sh
@@ -7,5 +7,5 @@ pip_required
 
 echo "Installing requirements before running tests..."
 pip install --upgrade pip
-pip install -r requirements.txt
 pip install -r test/requirements.txt
+pip install -e client/python