Repository: huggingface/huggingface_hub
Pull request #108 (2021-06-15 09:08:17)

[huggingface_hub] Support for spaces


Patch:
diff --git a/docs/inference.md b/docs/inference.md
index 39da114a..014edd7e 100644
--- a/docs/inference.md
+++ b/docs/inference.md
@@ -1,8 +1,8 @@
 ---
-title: 🤗 Hub Inference API docs
+title: Inference API docs
 ---
 
-<h1>🤗 Hub Inference API</h1>
+<h1>Inference API</h1>
 
 For detailed usage documentation, please refer to [Accelerated Inference API Documentation](https://api-inference.huggingface.co/docs/python/html/index.html).
 
diff --git a/docs/libraries.md b/docs/libraries.md
index 88ad42cf..0544fa0c 100644
--- a/docs/libraries.md
+++ b/docs/libraries.md
@@ -1,13 +1,13 @@
 ---
-title: 🤗 Hub Libraries Docs
+title: Hugging Face Hub Libraries Docs
 ---
 
-<h1>🤗 Hub Libraries</h1>
+<h1>Hugging Face Hub Libraries</h1>
 
 
 ## Do you only support 🤗 Transformers?
 
-No, the 🤗 Hub supports other libraries and we're working on expanding this support! We're happy to welcome to the 🤗 Hub a set of Open Source libraries that are pushing Machine Learning forward.
+No, the Hub supports other libraries and we're working on expanding this support! We're happy to welcome to the Hub a set of Open Source libraries that are pushing Machine Learning forward.
 
 The table below summarizes the supported libraries and how they are integrated. Find all our supported libraries [here](https://github.com/huggingface/huggingface_hub/blob/main/interfaces/Libraries.ts)! 
 
diff --git a/docs/main.md b/docs/main.md
index cca11812..d4e47509 100644
--- a/docs/main.md
+++ b/docs/main.md
@@ -1,20 +1,20 @@
 ---
-title: 🤗 Hub docs
+title: Hugging Face Hub docs
 ---
 
-<h1>🤗 Hub documentation</h1>
+<h1>Hugging Face Hub documentation</h1>
 
 
-## What's the 🤗 Hub?
+## What's the Hugging Face Hub?
 
 We are helping the community work together towards the goal of advancing Artificial Intelligence 🔥.
 
-Not one company, even the Tech Titans, will be able to “solve AI” by itself – the only way we'll achieve this is by sharing knowledge and resources. On the 🤗 Hub we are building the largest collection of models, datasets and metrics in order to democratize and advance AI for everyone 🚀. The 🤗 Hub works as a central place where anyone can share and explore models and datasets.
+Not one company, even the Tech Titans, will be able to “solve AI” by itself – the only way we'll achieve this is by sharing knowledge and resources. On the Hugging Face Hub we are building the largest collection of models, datasets and metrics in order to democratize and advance AI for everyone 🚀. The Hugging Face Hub works as a central place where anyone can share and explore models and datasets.
 
 
 ## What's a repository?
 
-The 🤗 Hub hosts Git-based repositories which are storage spaces that can contain all your files 💾.
+The Hugging Face Hub hosts Git-based repositories which are storage spaces that can contain all your files 💾.
 
 These repositories have multiple advantages over other hosting solutions:
 
@@ -22,12 +22,12 @@ These repositories have multiple advantages over other hosting solutions:
 * commit history and diffs
 * branches
 
-On top of that, 🤗 Hub repositories have many other advantages:
+On top of that, Hugging Face Hub repositories have many other advantages:
 
 * Repos provide useful [metadata](https://raw.githubusercontent.com/huggingface/huggingface_hub/main/modelcard.md) about their tasks, languages, metrics, etc.
 * Anyone can play with the model directly in the browser!
 * An API is provided to use the models in production settings.
-* [Over 10 frameworks](/docs/libraries) such as 🤗 Transformers, Asteroid and ESPnet support using models from the 🤗 Hub. 
+* [Over 10 frameworks](/docs/libraries) such as 🤗 Transformers, Asteroid and ESPnet support using models from the Hugging Face Hub. 
 
 
 ## What's a widget?
@@ -43,20 +43,20 @@ Here are some examples:
 
 ## What's the Inference API?
 
-The Inference API allows you to send HTTP requests to models in the 🤗 Hub. The Inference API is 2x to 10x faster than the widgets! ⚡⚡
+The Inference API allows you to send HTTP requests to models in the Hugging Face Hub. The Inference API is 2x to 10x faster than the widgets! ⚡⚡
 
 
-## How can I explore the 🤗 Hub?
+## How can I explore the Hugging Face Hub?
 
 [![Model Hub Video](https://img.youtube.com/vi/XvSGPZFEjDY/0.jpg)](https://www.youtube.com/watch?v=XvSGPZFEjDY)
 
 ## How is a model's type of inference API and widget determined?
 
-To determine which pipeline and widget to display (`text-classification`, `token-classification`, `translation`, etc.), we analyze information in the repo such as the metadata provided in the model card and configuration files. This information is mapped to a single `pipeline_tag`. At the moment, we expose **only one** widget per model. 
+To determine which pipeline and widget to display (`text-classification`, `token-classification`, `translation`, etc.), we analyze information in the repo such as the metadata provided in the model card and configuration files. This information is mapped to a single `pipeline_tag`. We choose to expose **only one** widget per model for simplicity.
 
 For most use cases, the model type is determined from the tags. For example, if there is `tag: text-classification` in the metadata, the inferred `pipeline_tag` will be `text-classification`.
 
-For `🤗 Transformers`, additional information is obtained from `config.json`. The architecture can be used to determine the type: for example, `AutoModelForTokenClassification` corresponds to `token-classification`. If you're really interested in this, you can see pseudo-code in [this gist](https://gist.github.com/julien-c/857ba86a6c6a895ecd90e7f7cab48046).
+For `🤗 Transformers` however, the model type is determined automatically from `config.json`. The architecture can be used to determine the type: for example, `AutoModelForTokenClassification` corresponds to `token-classification`. If you're really interested in this, you can see pseudo-code in [this gist](https://gist.github.com/julien-c/857ba86a6c6a895ecd90e7f7cab48046).
 
 **You can always manually override your pipeline type with pipeline_tag: xxx in your model card metadata.**
 
diff --git a/docs/model-repos.md b/docs/model-repos.md
index 98a586c2..d5c4338b 100644
--- a/docs/model-repos.md
+++ b/docs/model-repos.md
@@ -71,6 +71,8 @@ widget:
 - text: "Jens Peter Hansen kommer fra Danmark"
 ```
 
+We try to provide example inputs for some languages and most widget types in [this DefaultWidget.ts file](https://github.com/huggingface/huggingface_hub/blob/master/interfaces/DefaultWidget.ts). If we lack some examples, please open a PR updating this file to add them. Thanks!
+
 ## Can I specify which framework supports my model?
 
 Yes!🔥 You can specify the framework in the model card metadata section:
diff --git a/src/huggingface_hub/__init__.py b/src/huggingface_hub/__init__.py
index 29a628fa..2ad5972a 100644
--- a/src/huggingface_hub/__init__.py
+++ b/src/huggingface_hub/__init__.py
@@ -25,6 +25,7 @@ from .constants import (
     HUGGINGFACE_CO_URL_TEMPLATE,
     PYTORCH_WEIGHTS_NAME,
     REPO_TYPE_DATASET,
+    REPO_TYPE_SPACE,
     TF2_WEIGHTS_NAME,
     TF_WEIGHTS_NAME,
 )
diff --git a/src/huggingface_hub/commands/user.py b/src/huggingface_hub/commands/user.py
index 4153f3f5..1918b33e 100644
--- a/src/huggingface_hub/commands/user.py
+++ b/src/huggingface_hub/commands/user.py
@@ -18,11 +18,7 @@ from getpass import getpass
 from typing import List, Union
 
 from huggingface_hub.commands import BaseHuggingfaceCLICommand
-from huggingface_hub.constants import (
-    REPO_TYPE_DATASET,
-    REPO_TYPE_DATASET_URL_PREFIX,
-    REPO_TYPES,
-)
+from huggingface_hub.constants import REPO_TYPES, REPO_TYPES_URL_PREFIXES
 from huggingface_hub.hf_api import HfApi, HfFolder
 from requests.exceptions import HTTPError
 
@@ -67,7 +63,7 @@ class UserCommands(BaseHuggingfaceCLICommand):
         repo_create_parser.add_argument(
             "--type",
             type=str,
-            help='Optional: repo_type: set to "dataset" if creating a dataset, default is model.',
+            help='Optional: repo_type: set to "dataset" or "space" if creating a dataset or space, default is model.',
         )
         repo_create_parser.add_argument(
             "--organization", type=str, help="Optional: organization namespace."
@@ -239,8 +235,8 @@ class RepoCreateCommand(BaseUserCommand):
             print("Invalid repo --type")
             exit(1)
 
-        if self.args.type == REPO_TYPE_DATASET:
-            repo_id = REPO_TYPE_DATASET_URL_PREFIX + repo_id
+        if self.args.type in REPO_TYPES_URL_PREFIXES:
+            repo_id = REPO_TYPES_URL_PREFIXES[self.args.type] + repo_id
 
         print("You are about to create {}".format(ANSI.bold(repo_id)))
 
diff --git a/src/huggingface_hub/constants.py b/src/huggingface_hub/constants.py
index b1af0766..8155e23e 100644
--- a/src/huggingface_hub/constants.py
+++ b/src/huggingface_hub/constants.py
@@ -21,9 +21,13 @@ HUGGINGFACE_CO_URL_TEMPLATE = (
 )
 
 REPO_TYPE_DATASET = "dataset"
-REPO_TYPES = [None, REPO_TYPE_DATASET]
+REPO_TYPE_SPACE = "space"
+REPO_TYPES = [None, REPO_TYPE_DATASET, REPO_TYPE_SPACE]
 
-REPO_TYPE_DATASET_URL_PREFIX = "datasets/"
+REPO_TYPES_URL_PREFIXES = {
+    REPO_TYPE_DATASET: "datasets/",
+    REPO_TYPE_SPACE: "spaces/",
+}
 
 
 # default cache
diff --git a/src/huggingface_hub/file_download.py b/src/huggingface_hub/file_download.py
index d7b4994b..45c5ac00 100644
--- a/src/huggingface_hub/file_download.py
+++ b/src/huggingface_hub/file_download.py
@@ -23,9 +23,8 @@ from . import __version__
 from .constants import (
     HUGGINGFACE_CO_URL_TEMPLATE,
     HUGGINGFACE_HUB_CACHE,
-    REPO_TYPE_DATASET,
-    REPO_TYPE_DATASET_URL_PREFIX,
     REPO_TYPES,
+    REPO_TYPES_URL_PREFIXES,
 )
 from .hf_api import HfFolder
 
@@ -92,8 +91,8 @@ def hf_hub_url(
     if repo_type not in REPO_TYPES:
         raise ValueError("Invalid repo type")
 
-    if repo_type == REPO_TYPE_DATASET:
-        repo_id = REPO_TYPE_DATASET_URL_PREFIX + repo_id
+    if repo_type in REPO_TYPES_URL_PREFIXES:
+        repo_id = REPO_TYPES_URL_PREFIXES[repo_type] + repo_id
 
     if revision is None:
         revision = "main"
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index 385728d8..2d8d8ddf 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -22,17 +22,17 @@ from io import BufferedIOBase, RawIOBase
 from os.path import expanduser
 from typing import BinaryIO, Dict, Iterable, List, Optional, Tuple, Union
 
+import requests
+from requests.exceptions import HTTPError
+
+from .constants import REPO_TYPES, REPO_TYPES_URL_PREFIXES
+
 
 if sys.version_info >= (3, 8):
     from typing import Literal
 else:
     from typing_extensions import Literal
 
-import requests
-from requests.exceptions import HTTPError
-
-from .constants import REPO_TYPE_DATASET, REPO_TYPE_DATASET_URL_PREFIX, REPO_TYPES
-
 
 ENDPOINT = "https://huggingface.co"
 REMOTE_FILEPATH_REGEX = re.compile(r"^\w[\w\/]*(\.\w+)?$")
@@ -251,7 +251,7 @@ class HfApi:
         self, token: str, organization: Optional[str] = None
     ) -> List[RepoObj]:
         """
-        HuggingFace git-based system, used for models.
+        HuggingFace git-based system, used for models, datasets, and spaces.
 
         Call HF API to list all stored files for user (or one of their organizations).
         """
@@ -275,14 +275,14 @@ class HfApi:
         lfsmultipartthresh: Optional[int] = None,
     ) -> str:
         """
-        HuggingFace git-based system, used for models.
+        HuggingFace git-based system, used for models, datasets, and spaces.
 
         Call HF API to create a whole repo.
 
         Params:
             private: Whether the model repo should be private (requires a paid huggingface.co account)
 
-            repo_type: Set to "dataset" if creating a dataset, default is model
+            repo_type: Set to "dataset" or "space" if creating a dataset or space, default is model
 
             exist_ok: Do not raise an error if repo already exists
 
@@ -332,7 +332,7 @@ class HfApi:
         repo_type: Optional[str] = None,
     ):
         """
-        HuggingFace git-based system, used for models.
+        HuggingFace git-based system, used for models, datasets, and spaces.
 
         Call HF API to delete a whole repo.
 
@@ -374,13 +374,11 @@ class HfApi:
             namespace = organization
 
         path_prefix = "{}/api/".format(self.endpoint)
-        if repo_type == REPO_TYPE_DATASET:
-            path_prefix += REPO_TYPE_DATASET_URL_PREFIX
+        if repo_type in REPO_TYPES_URL_PREFIXES:
+            path_prefix += REPO_TYPES_URL_PREFIXES[repo_type]
 
         path = "{}{}/{}/settings".format(path_prefix, namespace, name)
         json = {"private": private}
-        if repo_type is not None:
-            json["type"] = repo_type
 
         r = requests.put(
             path,
@@ -417,7 +415,7 @@ class HfApi:
                 The repository to which the file will be uploaded, for example: :obj:`"username/custom_transformers"`
 
             repo_type (``str``, Optional):
-                Set to :obj:`"dataset"` if uploading to a dataset, :obj:`None` if uploading to a model. Default is :obj:`None`.
+                Set to :obj:`"dataset"` or :obj:`"space"` if uploading to a dataset or space, :obj:`None` if uploading to a model. Default is :obj:`None`.
 
             revision (``str``, Optional):
                 The git revision to commit from. Defaults to the :obj:`"main"` branch.
@@ -480,12 +478,8 @@ class HfApi:
                 )
             )
 
-        repo_id = "{prefix}{repo_id}".format(
-            prefix=REPO_TYPE_DATASET_URL_PREFIX
-            if (repo_type == REPO_TYPE_DATASET)
-            else "",
-            repo_id=repo_id,
-        )
+        if repo_type in REPO_TYPES_URL_PREFIXES:
+            repo_id = REPO_TYPES_URL_PREFIXES[repo_type] + repo_id
 
         revision = revision if revision is not None else "main"
 
diff --git a/src/huggingface_hub/hub_mixin.py b/src/huggingface_hub/hub_mixin.py
index a38473b1..bf28c09c 100644
--- a/src/huggingface_hub/hub_mixin.py
+++ b/src/huggingface_hub/hub_mixin.py
@@ -18,7 +18,7 @@ if is_torch_available():
 logger = logging.getLogger(__name__)
 
 
-class ModelHubMixin(object):
+class ModelHubMixin:
     def __init__(self, *args, **kwargs):
         """
         Mix this class with your torch-model class for ease process of saving & loading from huggingface-hub
diff --git a/src/huggingface_hub/repository.py b/src/huggingface_hub/repository.py
index e1607d36..5b1f4b22 100644
--- a/src/huggingface_hub/repository.py
+++ b/src/huggingface_hub/repository.py
@@ -206,9 +206,8 @@ class Repository:
                 )
 
                 if "main" not in output.stdout.split():
-                    # TODO(check if we really want the --force flag)
                     subprocess.run(
-                        "git checkout origin/main -ft".split(),
+                        "git checkout origin/main -t".split(),
                         stderr=subprocess.PIPE,
                         stdout=subprocess.PIPE,
                         encoding="utf-8",
@@ -415,7 +414,7 @@ class Repository:
 
     def push_to_hub(self, commit_message="commit files to HF hub") -> str:
         """
-        Helper to add, commit, and pushe file to remote repository on the HuggingFace Hub.
+        Helper to add, commit, and push files to remote repository on the HuggingFace Hub.
         Args:
             commit_message: commit message.
         """


Test patch:
diff --git a/tests/test_hf_api.py b/tests/test_hf_api.py
index 46e0fc49..0aa81d1b 100644
--- a/tests/test_hf_api.py
+++ b/tests/test_hf_api.py
@@ -21,7 +21,7 @@ import time
 import unittest
 from io import BytesIO
 
-from huggingface_hub.constants import REPO_TYPE_DATASET
+from huggingface_hub.constants import REPO_TYPE_DATASET, REPO_TYPE_SPACE
 from huggingface_hub.file_download import cached_download
 from huggingface_hub.hf_api import HfApi, HfFolder, ModelInfo, RepoObj
 from requests.exceptions import HTTPError
@@ -38,6 +38,7 @@ from .testing_utils import (
 REPO_NAME = "my-model-{}".format(int(time.time() * 10e3))
 REPO_NAME_LARGE_FILE = "my-model-largefiles-{}".format(int(time.time() * 10e3))
 DATASET_REPO_NAME = "my-dataset-{}".format(int(time.time() * 10e3))
+SPACE_REPO_NAME = "my-space-{}".format(int(time.time() * 10e3))
 WORKING_REPO_DIR = os.path.join(
     os.path.dirname(os.path.abspath(__file__)), "fixtures/working_repo"
 )
@@ -59,7 +60,7 @@ class HfApiLoginTest(HfApiCommonTest):
         self.assertIsInstance(token, str)
 
 
-class HfApiEndpointsTest(HfApiCommonTest):
+class HfApiCommonTestWithLogin(HfApiCommonTest):
     @classmethod
     def setUpClass(cls):
         """
@@ -67,6 +68,8 @@ class HfApiEndpointsTest(HfApiCommonTest):
         """
         cls._token = cls._api.login(username=USER, password=PASS)
 
+
+class HfApiEndpointsTest(HfApiCommonTestWithLogin):
     def test_whoami(self):
         user, orgs = self._api.whoami(token=self._token)
         self.assertEqual(user, USER)
@@ -93,25 +96,51 @@ class HfApiEndpointsTest(HfApiCommonTest):
 
     def test_create_update_and_delete_dataset_repo(self):
         self._api.create_repo(
-            token=self._token, name=REPO_NAME, repo_type=REPO_TYPE_DATASET
+            token=self._token, name=DATASET_REPO_NAME, repo_type=REPO_TYPE_DATASET
         )
         res = self._api.update_repo_visibility(
-            token=self._token, name=REPO_NAME, private=True, repo_type=REPO_TYPE_DATASET
+            token=self._token,
+            name=DATASET_REPO_NAME,
+            private=True,
+            repo_type=REPO_TYPE_DATASET,
         )
         self.assertTrue(res["private"])
         res = self._api.update_repo_visibility(
             token=self._token,
-            name=REPO_NAME,
+            name=DATASET_REPO_NAME,
             private=False,
             repo_type=REPO_TYPE_DATASET,
         )
         self.assertFalse(res["private"])
         self._api.delete_repo(
-            token=self._token, name=REPO_NAME, repo_type=REPO_TYPE_DATASET
+            token=self._token, name=DATASET_REPO_NAME, repo_type=REPO_TYPE_DATASET
+        )
+
+    @unittest.skip("skipped while spaces in beta")
+    def test_create_update_and_delete_space_repo(self):
+        self._api.create_repo(
+            token=self._token, name=SPACE_REPO_NAME, repo_type=REPO_TYPE_SPACE
+        )
+        res = self._api.update_repo_visibility(
+            token=self._token,
+            name=SPACE_REPO_NAME,
+            private=True,
+            repo_type=REPO_TYPE_SPACE,
+        )
+        self.assertTrue(res["private"])
+        res = self._api.update_repo_visibility(
+            token=self._token,
+            name=SPACE_REPO_NAME,
+            private=False,
+            repo_type=REPO_TYPE_SPACE,
+        )
+        self.assertFalse(res["private"])
+        self._api.delete_repo(
+            token=self._token, name=SPACE_REPO_NAME, repo_type=REPO_TYPE_SPACE
         )
 
 
-class HfApiUploadFileTest(HfApiEndpointsTest):
+class HfApiUploadFileTest(HfApiCommonTestWithLogin):
     def setUp(self) -> None:
         super().setUp()
         self.tmp_dir = tempfile.mkdtemp()