Repository: gradio-app/gradio
Pull request: Allow client to be duplicated from an existing Space
Adds and documents a `Client.duplicate_from()` factory method

Not sure if there's a good way to test this without revealing write-access keys which is not a good idea, Lmk if you have suggestions!

Closes: #3804 

Problem statement:
Let Python client be able to duplicate a Space easily and make it private
Public Spaces will be rate limited, which means that if someone connects a Space using the Gradio client and starts making requests, they'll quickly hit the limit, something like this:

```py
from gradio_client import Client

client = Client("abidlabs/en2fr")
client.predict("hello")
> bonjour
client.predict("hello")
> bonjour
...
client.predict("hello")
> Error: 429 Rate limit. Please duplicate the Space and make it private to avoid being rate limited.
```

However, this requires users to manually duplicate the Space, which is quite cumbersome and harder to use than other APIs-as-a-service. What if we had a "private mode" for the Client. If you do:

```py
from gradio_client import Client

client = Client("abidlabs/en2fr", hf_token="...", private=True)
> Creating a private version of this Space for your own use 🤗 
> Created "aliabid94/en2fr", running on T4 GPU with a sleep timeout of 1 hour (after 1 hour of inactivity, the Space will go to sleep). 
> Ready to make predictions! 

client.predict("hello")
> bonjour
client.predict("hello")
> bonjour
client.predict("hello")
> bonjour

# no timeout!
```

cc @aliabid94 @freddyaboulton @osanseviero @julien-c @Wauplin

Hints from issue discussion:
My thought is that running `Client("abidlabs/en2fr", hf_token="...", private=True)` should be idempotent so that a developer's app doesn't drastically change without their knowledge. But does that mean we should give them a mechanism to updating the private version of their space?

What if we keep the duplication logic in a class method that gives users more control of what version of the upstream space/hardware is used?

```python
client = Client.duplicate_from("abidlabs/en2fr", hf_token=..., private=...) # always fetch latest
client = Client.duplicate_from("abidlabs/en2fr", commit=..., hardware=....) # pin to a commit
```

Makes sense to me! I'll put together a PR
> What if we keep the duplication logic in a class method that gives users more control of what version of the upstream space/hardware is used?

I also like the idea. Just a nit but I would align some parameter if that's fine with you:

```py
# hf_token => token ?
# commit => revision (=> can be a branch name, a tag,...)

client = Client.duplicate_from("abidlabs/en2fr", token=..., private=..., revision=..., hardware=..., secrets=...)
```

It would also be useful to be able to set secrets for the space. For now it's not possible to retrieve which secrets are required when you duplicate a space via the API so it would be at user's discretion to know that. 

---

Since adding hardware to a Space can be very costly, I wonder what would be the best way to ensure users don't feel cheated if a Space is not in use anymore. A possibility could be to use the auto-sleep feature but I'm not sure how well it was working + it's not accessible via script. Another possibility is to request a free cpu once the python exit (using `atexit` for example?). But even like this it's possible that it's not called if the script is shut down abrutely.

Any thoughts on how we can handle this?
> I also like the idea. Just a nit but I would align some parameter if that's fine with you:

Totally

> commit => revision

Will make the change. Is it possible to use the revision with `duplicate_space`? I don't see it as a parameter here: https://huggingface.co/docs/huggingface_hub/v0.13.4/en/package_reference/hf_api#huggingface_hub.HfApi.duplicate_space

> hf_token => token

For this, I'd prefer to keep `hf_token`. I know that's not consistent with usage in other HF libraries, but it's consistent with usage in other parts of the gradio library, such as `gradio.themes.push_to_hub`

> It would also be useful to be able to set secrets for the space

Makes sense. Will add using `add_space_secret`:  https://huggingface.co/docs/huggingface_hub/v0.13.4/en/package_reference/hf_api#huggingface_hub.HfApi.add_space_secret
> Since adding hardware to a Space can be very costly, I wonder what would be the best way to ensure users don't feel cheated if a Space is not in use anymore. A possibility could be to use the auto-sleep feature but I'm not sure how well it was working + it's not accessible via script. Another possibility is to request a free cpu once the python exit (using atexit for example?). But even like this it's possible that it's not called if the script is shut down abrutely.

For this, I was thinking we can set a sleep timeout of 1 hour. So worst case scenario, people are charged for one hour of GPU usage. Is it possible to do this with the `huggingface_hub` library? 
>  Is it possible to use the revision with duplicate_space? 

Actually no, it's not possible. Maybe not worth investing time to add it since it's maybe more a corner case.

> For this, I'd prefer to keep hf_token. 

Yep, fine :+1: 

> Will add using `add_space_secret`

Definitely the way to go yes!

> I was thinking we can set a sleep timeout of 1 hour.
> Is it possible to do this with the huggingface_hub library?

Unfortunately not at the moment no. But the endpoint already exists server-side (see [API route](https://github.com/huggingface/moon-landing/blob/a461ec4007cf96db2f2764d767c308e984c5ff56/server/server.ts#L675) and [implementation](https://github.com/huggingface/moon-landing/blob/main/server/app/spacesHandlers.ts#L445) -private link). I can work on adding this to `huggingface_hub`. Next release `0.14` should come _soon_ (probably next week?).  


> Unfortunately not at the moment no. But the endpoint already exists server-side (see [API route](https://github.com/huggingface/moon-landing/blob/a461ec4007cf96db2f2764d767c308e984c5ff56/server/server.ts#L675) and [implementation](https://github.com/huggingface/moon-landing/blob/main/server/app/spacesHandlers.ts#L445) -private link). I can work on adding this to huggingface_hub. Next release 0.14 should come soon (probably next week?).

Sweet, that'd be awesome. If you're open to doing an alpha release, maybe we can incorporate it even earlier.
I'll let you know :)
My main concern with `client = Client.duplicate_from("abidlabs/whisper") ` is also around Hardware Upgrades and automatically giving specified hardware. I feel the command should give clear warnings if a repo has secrets and/or Hardware Upgrades

Commit message: fix tests

Patch to review:
diff --git a/client/python/README.md b/client/python/README.md
index a6fa8365e3..be2bab3917 100644
--- a/client/python/README.md
+++ b/client/python/README.md
@@ -53,6 +53,26 @@ from gradio_client import Client
 client = Client("abidlabs/my-private-space", hf_token="...") 
 ```
 
+**Duplicating a Space for private use**
+
+While you can use any public Space as an API, you may get rate limited by Hugging Face if you make too many requests. For unlimited usage of a Space, simply duplicate the Space to create a private Space,
+and then use it to make as many requests as you'd like!
+
+The `gradio_client` includes a class method: `Client.duplicate_from` to make this process simple:
+
+```python
+from gradio_client import Client
+
+client = Client.duplicate_from("abidlabs/whisper") 
+client.predict("audio_sample.wav")  
+
+>> "This is a test of the whisper speech recognition model."
+```
+
+If you have previously duplicated a Space, re-running `duplicate_from` will *not* create a new Space. Instead, the Client will attach to the previously-created Space. So it is safe to re-run the `Client.duplicate_from` method multiple times. 
+
+**Note:** if the original Space uses GPUs, your private Space will as well, and your Hugging Face account will get billed based on the price of the GPU. To minimize charges, your Space will automatically go to sleep after 1 hour of inactivity. You can also set the hardware using the `hardware` parameter of `duplicate()`.
+
 
 **Connecting a general Gradio app**
 
@@ -64,9 +84,10 @@ from gradio_client import Client
 client = Client("https://bec81a83-5b5c-471e.gradio.live")
 ```
 
+
 ### Inspecting the API endpoints
 
-Once you have connected to a Gradio app, you can view the APIs that are available to you by calling the `Client.view_api()` method. For the Whisper Space, we see the following:
+Once you have connected to a Gradio app, you can view the APIs that are available to you by calling the `.view_api()` method. For the Whisper Space, we see the following:
 
 ```
 Client.predict() Usage Info
diff --git a/client/python/gradio_client/client.py b/client/python/gradio_client/client.py
index 8f62384b1e..489ac5ba3c 100644
--- a/client/python/gradio_client/client.py
+++ b/client/python/gradio_client/client.py
@@ -17,7 +17,11 @@
 import huggingface_hub
 import requests
 import websockets
-from huggingface_hub.utils import build_hf_headers, send_telemetry
+from huggingface_hub.utils import (
+    RepositoryNotFoundError,
+    build_hf_headers,
+    send_telemetry,
+)
 from packaging import version
 from typing_extensions import Literal
 
@@ -58,7 +62,7 @@ def __init__(
         """
         Parameters:
             src: Either the name of the Hugging Face Space to load, (e.g. "abidlabs/whisper-large-v2") or the full URL (including "http" or "https") of the hosted Gradio app to load (e.g. "http://mydomain.com/app" or "https://bec81a83-5b5c-471e.gradio.live/").
-            hf_token: The Hugging Face token to use to access private Spaces. Automatically fetched if you are logged in via the Hugging Face Hub CLI.
+            hf_token: The Hugging Face token to use to access private Spaces. Automatically fetched if you are logged in via the Hugging Face Hub CLI. Obtain from: https://huggingface.co/settings/token
             max_workers: The maximum number of thread workers that can be used to make requests to the remote Gradio app simultaneously.
             verbose: Whether the client should print statements to the console.
         """
@@ -81,14 +85,20 @@ def __init__(
                 )
             self.space_id = src
         self.src = _src
-        if self.verbose:
-            print(f"Loaded as API: {self.src} ✔")
         state = self._get_space_state()
+        if state == utils.BUILDING_RUNTIME:
+            if self.verbose:
+                print("Space is still building. Please wait...")
+            while self._get_space_state() == utils.BUILDING_RUNTIME:
+                time.sleep(2)  # so we don't get rate limited by the API
+                pass
         if state in utils.INVALID_RUNTIME:
             raise ValueError(
                 f"The current space is in the invalid state: {state}. "
                 "Please contact the owner to fix this."
             )
+        if self.verbose:
+            print(f"Loaded as API: {self.src} ✔")
 
         self.api_url = urllib.parse.urljoin(self.src, utils.API_URL)
         self.ws_url = urllib.parse.urljoin(
@@ -110,6 +120,90 @@ def __init__(
         # Disable telemetry by setting the env variable HF_HUB_DISABLE_TELEMETRY=1
         threading.Thread(target=self._telemetry_thread).start()
 
+    @classmethod
+    def duplicate_from(
+        cls,
+        from_id: str,
+        to_id: str | None = None,
+        hf_token: str | None = None,
+        private: bool = True,
+        hardware: str | None = None,
+        secrets: Dict[str, str] | None = None,
+        max_workers: int = 40,
+        verbose: bool = True,
+    ):
+        """
+        Duplicates a Hugging Face Space under your account and returns a Client object
+        for the new Space. No duplication is created if the Space already exists in your
+        account (to override this, provide a new name for the new Space using `to_id`).
+        To use this method, you must provide an `hf_token` or be logged in via the Hugging
+        Face Hub CLI.
+
+        The new Space will be private by default and use the same hardware as the original
+        Space. This can be changed by using the `private` and `hardware` parameters. For
+        hardware upgrades (beyond the basic CPU tier), you may be required to provide
+        billing information on Hugging Face: https://huggingface.co/settings/billing
+
+        Parameters:
+            from_id: The name of the Hugging Face Space to duplicate, e.g. "abidlabs/whisper".
+            to_id: The name of the new Hugging Face Space to create, e.g. "whisper-duplicate". Do not add your Hugging Face username. If not provided, the new Space will be named "{you_hf_username}/{from_id}".
+            hf_token: The Hugging Face token to use to access private Spaces. Automatically fetched if you are logged in via the Hugging Face Hub CLI. Obtain from: https://huggingface.co/settings/token
+            private: Whether the new Space should be private (True) or public (False). Defaults to True.
+            hardware: The hardware tier to use for the new Space. Defaults to the same hardware tier as the original Space. Options include "cpu-basic", "cpu-upgrade", "t4-small", "t4-medium", "a10g-small", "a10g-large", "a100-large", subject to availability.
+            secrets: A dictionary of (secret key, secret value) to pass to the new Space. Defaults to None.
+            max_workers: The maximum number of thread workers that can be used to make requests to the remote Gradio app simultaneously.
+            verbose: Whether the client should print statements to the console.
+        """
+        # Duplicate a Hugging Face Space
+        try:
+            info = huggingface_hub.get_space_runtime(from_id, token=hf_token)
+        except RepositoryNotFoundError:
+            raise ValueError(
+                f"Could not find Space: {from_id}. If it is a private Space, please provide an `hf_token`."
+            )
+        if to_id:
+            space_id = huggingface_hub.get_full_repo_name(to_id, token=hf_token)
+        else:
+            space_id = huggingface_hub.get_full_repo_name(
+                from_id.split("/")[1], token=hf_token
+            )
+        try:
+            huggingface_hub.get_space_runtime(space_id, token=hf_token)
+            if verbose:
+                print(
+                    f"Using your existing Space: {utils.SPACE_URL.format(space_id)} 🤗"
+                )
+        except RepositoryNotFoundError:
+            if verbose:
+                print(f"Creating a duplicate of {from_id} for your own use... 🤗")
+            huggingface_hub.duplicate_space(
+                from_id=from_id,
+                to_id=space_id,
+                token=hf_token,
+                exist_ok=True,
+                private=private,
+            )
+            if verbose:
+                print(f"Created new Space: {utils.SPACE_URL.format(space_id)}")
+        current_info = huggingface_hub.get_space_runtime(space_id, token=hf_token)
+        current_hardware = current_info.hardware or "cpu-basic"
+        if hardware is None:
+            hardware = info.hardware
+        if not current_hardware == hardware:
+            huggingface_hub.request_space_hardware(space_id, hardware)  # type: ignore
+            print(
+                f"-------\nNOTE: this Space uses upgraded hardware: {hardware}... see billing info at https://huggingface.co/settings/billing\n-------"
+            )
+        if secrets is not None:
+            for key, value in secrets.items():
+                huggingface_hub.add_space_secret(space_id, key, value, token=hf_token)
+        if verbose:
+            print("")
+        client = cls(
+            space_id, hf_token=hf_token, max_workers=max_workers, verbose=verbose
+        )
+        return client
+
     def _get_space_state(self):
         if not self.space_id:
             return None
@@ -520,22 +614,29 @@ def _predict(*data) -> Tuple:
 
             if self.use_ws:
                 result = utils.synchronize_async(self._ws_fn, data, hash_data, helper)
-                output = result["data"]
+                if "error" in result:
+                    raise ValueError(result["error"])
             else:
                 response = requests.post(
                     self.client.api_url, headers=self.client.headers, data=data
                 )
                 result = json.loads(response.content.decode("utf-8"))
-                try:
-                    output = result["data"]
-                except KeyError:
-                    if "error" in result and "429" in result["error"]:
-                        raise utils.TooManyRequestsError(
-                            "Too many requests to the Hugging Face API"
-                        )
-                    raise KeyError(
-                        f"Could not find 'data' key in response. Response received: {result}"
+            try:
+                output = result["data"]
+            except KeyError:
+                is_public_space = (
+                    self.client.space_id
+                    and not huggingface_hub.space_info(self.client.space_id).private
+                )
+                if "error" in result and "429" in result["error"] and is_public_space:
+                    raise utils.TooManyRequestsError(
+                        f"Too many requests to the API, please try again later. To avoid being rate-limited, please duplicate the Space using Client.duplicate_from({self.client.space_id}) and pass in your Hugging Face token."
                     )
+                elif "error" in result:
+                    raise ValueError(result["error"])
+                raise KeyError(
+                    f"Could not find 'data' key in response. Response received: {result}"
+                )
             return tuple(output)
 
         return _predict
@@ -893,7 +994,7 @@ def status(self) -> StatusUpdate:
                     if self.verbose and self.space_id and eta and eta > 30:
                         print(
                             f"Due to heavy traffic on this app, the prediction will take approximately {int(eta)} seconds."
-                            f"For faster predictions without waiting in queue, you may duplicate the space: {utils.DUPLICATE_URL.format(self.space_id)}"
+                            f"For faster predictions without waiting in queue, you may duplicate the space using: Client.duplicate_from({self.space_id})"
                         )
                     return self.communicator.job.latest_status
 
diff --git a/client/python/gradio_client/utils.py b/client/python/gradio_client/utils.py
index 80efcf6b1f..bb8b93ec43 100644
--- a/client/python/gradio_client/utils.py
+++ b/client/python/gradio_client/utils.py
@@ -25,16 +25,16 @@
 WS_URL = "/queue/join"
 UPLOAD_URL = "/upload"
 RESET_URL = "/reset"
-DUPLICATE_URL = "https://huggingface.co/spaces/{}?duplicate=true"
+SPACE_URL = "https://hf.space/{}"
 STATE_COMPONENT = "state"
 INVALID_RUNTIME = [
     "NO_APP_FILE",
     "CONFIG_ERROR",
-    "BUILDING",
     "BUILD_ERROR",
     "RUNTIME_ERROR",
     "PAUSED",
 ]
+BUILDING_RUNTIME = "BUILDING"
 
 __version__ = (pkgutil.get_data(__name__, "version.txt") or b"").decode("ascii").strip()
 
diff --git a/client/python/test/test_client.py b/client/python/test/test_client.py
index dd9ea4bb22..53d671de43 100644
--- a/client/python/test/test_client.py
+++ b/client/python/test/test_client.py
@@ -100,7 +100,7 @@ def test_intermediate_outputs(
         self,
     ):
         client = Client(src="gradio/count_generator")
-        job = client.submit(3, api_name="/count")
+        job = client.submit(3, fn_index=0)
 
         while not job.done():
             time.sleep(0.1)
@@ -108,7 +108,7 @@ def test_intermediate_outputs(
         assert job.outputs() == [str(i) for i in range(3)]
 
         outputs = []
-        for o in client.submit(3, api_name="/count"):
+        for o in client.submit(3, fn_index=0):
             outputs.append(o)
         assert outputs == [str(i) for i in range(3)]
 
@@ -123,7 +123,7 @@ def test_break_in_loop_if_error(self):
     def test_timeout(self):
         with pytest.raises(TimeoutError):
             client = Client(src="gradio/count_generator")
-            job = client.submit(api_name="/sleep")
+            job = client.submit(3, fn_index=0)
             job.result(timeout=0.05)
 
     @pytest.mark.flaky


Code review comments:
diff_hunk: @@ -110,6 +120,90 @@ def __init__(
         # Disable telemetry by setting the env variable HF_HUB_DISABLE_TELEMETRY=1
         threading.Thread(target=self._telemetry_thread).start()
 
+    @classmethod
+    def duplicate_from(
+        cls,
+        from_id: str,
+        to_id: str | None = None,
+        hf_token: str | None = None,
+        private: bool = True,
+        hardware: str | None = None,
+        secrets: Dict[str, str] | None = None,
+        max_workers: int = 40,
+        verbose: bool = True,
+    ):
+        """
+        Duplicates a Hugging Face Space under your account and returns a Client object
+        for the new Space. No duplication is created if the Space already exists in your
+        account (to override this, provide a new name for the new Space using `to_id`).
+        To use this method, you must provide an `hf_token` or be logged in via the Hugging
+        Face Hub CLI.
+
+        The new Space will be private by default and use the same hardware as the original
+        Space. This can be changed by using the `private` and `hardware` parameters. For
+        hardware upgrades (beyond the basic CPU tier), you may be required to provide
+        billing information on Hugging Face: https://huggingface.co/settings/billing
+
+        Parameters:
+            from_id: The name of the Hugging Face Space to duplicate, e.g. "abidlabs/whisper".
+            to_id: The name of the new Hugging Face Space to create, e.g. "whisper-duplicate". Do not add your Hugging Face username. If not provided, the new Space will be named "{you_hf_username}/{from_id}".
path: client/python/gradio_client/client.py
text: @user1:
I got confused by the docstring here. Are you able to duplicate into to an org?

@author:
Yes, if you provide the HF_TOKEN corresponding to the org. What could make this clearer? 

@user1:
It says

> The name of the new Hugging Face Space to create, e.g. "whisper-duplicate". Do not add your Hugging Face username.

Which made me think `gradio/whisper-duplicate` was not a valid value as it seems it only expects the name after the `/`

@author:
Yes, that's correct. You should provide `whisper-duplicate`, not `gradio/whisper-duplicate`

@author:
The username is automatically inferred from the HF_TOKEN

@author:
I can change it so that you can provide one but it is ignored

@user1:
I think right now you are not able to clone to an org you are a member of?
Because the token will correspond to my personal namespace and not the one for the org I want.

Using the token for the org won't work because you need a personal token to create new repos

![image](https://user-images.githubusercontent.com/41651716/231557079-2ad4526d-c9b1-4a9e-8d50-659dbb84534c.png)


@author:
Oh interesting, was not aware of that. In that case, let me change this 

@author:
Pushed a fix

diff_hunk: @@ -110,6 +120,90 @@ def __init__(
         # Disable telemetry by setting the env variable HF_HUB_DISABLE_TELEMETRY=1
         threading.Thread(target=self._telemetry_thread).start()
 
+    @classmethod
+    def duplicate_from(
+        cls,
+        from_id: str,
+        to_id: str | None = None,
+        hf_token: str | None = None,
+        private: bool = True,
+        hardware: str | None = None,
+        secrets: Dict[str, str] | None = None,
+        max_workers: int = 40,
+        verbose: bool = True,
+    ):
+        """
+        Duplicates a Hugging Face Space under your account and returns a Client object
+        for the new Space. No duplication is created if the Space already exists in your
+        account (to override this, provide a new name for the new Space using `to_id`).
+        To use this method, you must provide an `hf_token` or be logged in via the Hugging
+        Face Hub CLI.
+
+        The new Space will be private by default and use the same hardware as the original
+        Space. This can be changed by using the `private` and `hardware` parameters. For
+        hardware upgrades (beyond the basic CPU tier), you may be required to provide
+        billing information on Hugging Face: https://huggingface.co/settings/billing
+
+        Parameters:
+            from_id: The name of the Hugging Face Space to duplicate, e.g. "abidlabs/whisper".
+            to_id: The name of the new Hugging Face Space to create, e.g. "whisper-duplicate". Do not add your Hugging Face username. If not provided, the new Space will be named "{you_hf_username}/{from_id}".
+            hf_token: The Hugging Face token to use to access private Spaces. Automatically fetched if you are logged in via the Hugging Face Hub CLI. Obtain from: https://huggingface.co/settings/token
+            private: Whether the new Space should be private (True) or public (False). Defaults to True.
+            hardware: The hardware tier to use for the new Space. Defaults to the same hardware tier as the original Space. Options include "cpu-basic", "cpu-upgrade", "t4-small", "t4-medium", "a10g-small", "a10g-large", "a100-large", subject to availability.
+            secrets: A dictionary of (secret key, secret value) to pass to the new Space. Defaults to None.
+            max_workers: The maximum number of thread workers that can be used to make requests to the remote Gradio app simultaneously.
+            verbose: Whether the client should print statements to the console.
+        """
+        # Duplicate a Hugging Face Space
path: client/python/gradio_client/client.py
text: @user1:
```suggestion
```