Repository: huggingface/huggingface_hub
Pull request #39 (2021-05-06 12:44:30)

Alignment on modelcard metadata specification
Hi all, opening this PR so that we can all align on the metadata spec for model cards. This metadata is important as it closes the bridge between tasks, datasets, and metrics for a given checkpoint. This will eventually allow programmatic analysis and handling of model cards' metadata on the hub.

The metadata was drafted during the collaboration with papers-with-code in order to have a ranked leaderboard for the XLSR sprint. The following format was adopted:

```
model-index:  
- name: {model_id}
  results:
  - task: 
      name: {task_name}
      type: {task_type}
    dataset:
      name: {dataset_name}
      type: {dataset_type}
      args: {arg_0}
    metrics:
      - name: {metric_name} 
        type: {metric_type}
        value: {metric_value}
        args: {arg_0}
```

This format should allow for multiple tasks and multiple metrics within each task.

Some existing examples of modelcards with this format are the modelcards uploaded during the XLSR sprint, like the following [ydshieh/wav2vec2-large-xlsr-53-chinese-zh-cn-gpt](https://huggingface.co/ydshieh/wav2vec2-large-xlsr-53-chinese-zh-cn-gpt/blob/main/README.md)

Looking forward to your feedback.

Transformers: @sgugger @patrickvonplaten 
Datasets: @lhoestq 
AutoNLP: @abhi1thakur @SBrandeis 
Evaluation: @lewtun 

@Pierrci @julien-c @thomwolf 

lhoestq: Cool thanks !

One note regarding metrics: they might need `args` as well.
For example model-based metrics such a BLEURT, BERTScore or COMET may require a `model_name` argument.
Another example is BLEU that have a `max_order` parameter (max n-gram order to use when computing BLEU score), as well smoothing parameters, or the tokenizer to use.
julien-c: Very excited about this, and let's also ping the Paperswithcode team here for validation?
LysandreJik: Absolutely! Pinged them offline for a review.
jspisak: validation and integration? :) 
rstojnic: Thanks for looping us in! Yeah the format looks good. Since we already have the integration for wav2vec2 running, it should be pretty easy from our side to extend it to any result on any benchmark. 

A couple of things to consider:
1) To make sure we are using the same task/data names, I think it would be great to ask users to check the task/dataset name on https://paperswithcode.com/sota and https://paperswithcode.com/datasets . This would ensure all the results are put in the correct leaderboard on Papers with Code, and that it remains interoperable with anyone else wanting to use this data. 
2) It would be great if there was an automated badge or something like that would link to the Papers with Code leaderboard on the model page. We could create a URL you could easily construct from the metadata, e.g. for the example above it could be `https://paperswithcode.com/sota/?task=Speech Recognition&dataset=Common Voice zh-CN` that would redirect you to the correct leaderboard Or, there could also be a small embeddable graph or just a badge with ranking. 
3) For us to be able to efficiently track this metadata it would be useful to have an API endpoint where we can access all the latest modelcard changes. I.e. something similar to what we are using in the current integration (https://huggingface.co/api/models), but just with the ability to order by last changed.  


Patch:
diff --git a/.github/conda/meta.yaml b/.github/conda/meta.yaml
index 47ae207e..3fd26340 100644
--- a/.github/conda/meta.yaml
+++ b/.github/conda/meta.yaml
@@ -17,12 +17,14 @@ requirements:
     - filelock
     - requests
     - tqdm
+    - typing-extensions
   run:
     - python
     - pip
     - filelock
     - requests
     - tqdm
+    - typing-extensions
 
 test:
   imports:
diff --git a/README.md b/README.md
index 244806cd..4259c21b 100644
--- a/README.md
+++ b/README.md
@@ -93,7 +93,7 @@ Users add tags into their README.md model cards (e.g. your `library_name`, a dom
 
 ### API utilities in `hf_api.py`
 
-You don't need them for the standard publishing workflow, however, if you need a programmatic way of creating a repo, deleting it (`⚠️ caution`), or listing models from the hub, you'll find helpers in `hf_api.py`.
+You don't need them for the standard publishing workflow, however, if you need a programmatic way of creating a repo, deleting it (`⚠️ caution`), pushing a single file to a repo or listing models from the hub, you'll find helpers in `hf_api.py`.
 
 We also have an API to query models by specific tags (e.g. if you want to list models compatible to your library)
 
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 00000000..5a79f42d
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,128 @@
+---
+title: Model Hub docs
+---
+
+<h1 class="no-top-margin">Model Hub documentation</h1>
+
+
+## What's the Hugging Face model hub?
+
+We are helping the community work together towards the goal of advancing NLP 🔥.
+
+Not one company, even the Tech Titans, will be able to “solve NLP” by themselves – the only way we'll achieve this is by sharing knowledge and resources. On this model hub we are building the largest collection of models, datasets and metrics to democratize and advance AI and NLP for everyone 🚀.
+
+## When sharing a model, what should I add to my model card?
+
+In your README.md model card you should:
+- describe your model,
+- its intended uses & potential limitations, including bias and ethical considerations as detailed in [[Mitchell, 2018]](https://arxiv.org/abs/1810.03993)
+- your training params and experimental info – you can embed or link to an experiment tracking platform for reference
+- which datasets did you train on, and your eval results.
+
+If needed you can find a template [here](https://github.com/huggingface/model_card).
+
+
+## What metadata can I add to my model card?
+
+In addition to textual (markdown) content, to unlock helpful features you can add any or all of the following items to a [YAML](https://en.wikipedia.org/wiki/YAML) metadata block at the top of your model card:
+
+```yaml
+---
+language: "ISO 639-1 code for your language, or `multilingual`"
+thumbnail: "url to a thumbnail used in social sharing"
+tags:
+- array
+- of
+- tags
+license: "any valid license identifier"
+datasets:
+- array of dataset identifiers
+metrics:
+- array of metric identifiers
+---
+```
+
+License identifiers are those standardized by GitHub in the right column (keywords) [here](https://help.github.com/en/github/creating-cloning-and-archiving-repositories/licensing-a-repository#searching-github-by-license-type).
+
+Dataset and metric identifiers are those listed on the [datasets](https://huggingface.co/datasets) and [metrics](https://huggingface.co/metrics) pages and in the [`datasets`](https://github.com/huggingface/datasets) repository.
+
+All the tags can then be used to filter the list of models on https://huggingface.co/models.
+
+
+## How are model tags determined?
+
+On top of each model page (see e.g. [`distilbert-base-uncased`](/distilbert-base-uncased)) you'll see the model's tags – they help for discovery and condition which features are enabled on which model page.
+
+- The weight files that compose the models condition the framework(s) like `pytorch`, `tf`, etc.
+- The `"architectures"` field of the model's config.json file – which should be automatically filled if you save your model using `.save_pretrained()` – condition the type of pipeline used in the inference API, and the type of widget present on the model page
+	- A simplified snapshot of the mapping code can be found in [this gist](https://gist.github.com/julien-c/857ba86a6c6a895ecd90e7f7cab48046).
+- If your config.json file contains a `task_specific_params` subfield, its sub-keys will be added as `pipeline:` tags. All parameters defined under this sub-key will overwrite the default parameters in config.json when running the corresponding pipeline. See [`t5-base`](https://huggingface.co/t5-base) for example.
+- Most other metadata from the metadata block are also added as extra tags, at the end of the list.
+
+
+## How is a model's type of inference API and widget determined?
+
+To determine which pipeline and widget to display (text-classification, token-classification, translation, etc.), we use a simple mapping from model tags to one particular `pipeline_tag` (we currently only expose *one* pipeline and widget on each model page, even for models that would support several).
+
+We try to use the most specific pipeline for each model, see pseudo-code in [this gist](https://gist.github.com/julien-c/857ba86a6c6a895ecd90e7f7cab48046).
+
+You can always manually override your pipeline type with `pipeline_tag: xxx` in your model card metadata.
+
+## What are all the possible pipeline/widget types?
+
+Here they are, with links to examples:
+- `text-classification`, for instance [`roberta-large-mnli`](https://huggingface.co/roberta-large-mnli)
+- `token-classification`, for instance [`dbmdz/bert-large-cased-finetuned-conll03-english`](https://huggingface.co/dbmdz/bert-large-cased-finetuned-conll03-english)
+- `question-answering`, for instance [`distilbert-base-uncased-distilled-squad`](https://huggingface.co/distilbert-base-uncased-distilled-squad)
+- `translation`, for instance [`t5-base`](https://huggingface.co/t5-base)
+- `summarization`, for instance [`facebook/bart-large-cnn`](https://huggingface.co/facebook/bart-large-cnn)
+- `conversational`, for instance [`facebook/blenderbot-400M-distill`](https://huggingface.co/facebook/blenderbot-400M-distill)
+- `text-generation`, for instance [`gpt2`](https://huggingface.co/gpt2)
+- `fill-mask`, for instance [`distilroberta-base`](https://huggingface.co/distilroberta-base)
+- `zero-shot-classification` (implemented on top of a nli `text-classification` model), for instance [`facebook/bart-large-mnli`](https://huggingface.co/facebook/bart-large-mnli)
+- `table-question-answering`, for instance [`google/tapas-base-finetuned-wtq`](https://huggingface.co/google/tapas-base-finetuned-wtq)
+
+## How can I control my model's widget's example inputs?
+
+Example inputs are the random inputs that pre-populate your widget on page launch (unless you specify an input by URL parameters).
+
+We try to provide example inputs for some languages and widget types, but it's better if you provide your own examples. You can add them to your model card: see [this commit](https://github.com/huggingface/transformers/commit/6a495cae0090307198131c07cd4f3f1e9b38b4e6) for the format you need to use.
+
+If we don't provide default inputs for your model's language, please open a PR against [this DefaultWidget.ts file](https://github.com/huggingface/widgets-server/blob/master/DefaultWidget.ts) to add them. Thanks!
+
+
+## How can I turn off the inference API for my model?
+
+Specify `inference: false` in your model card's metadata.
+
+
+## Can I send large volumes of requests? Can I get accelerated APIs?
+
+If you are interested in accelerated inference and/or higher volumes of requests and/or a SLA, please contact us at `api-enterprise at huggingface.co`.
+
+
+## What technology do you use to power the inference API?
+
+The API is built on top of our [Pipelines](https://huggingface.co/transformers/main_classes/pipelines.html) feature.
+
+On top of Pipelines and depending on the model type, we build a number of production optimizations like:
+- compiling models to optimized intermediary representations (e.g. [ONNX](https://medium.com/microsoftazure/accelerate-your-nlp-pipelines-using-hugging-face-transformers-and-onnx-runtime-2443578f4333)),
+- maintaining a Least Recently Used cache ensuring that the most popular models are always loaded,
+- scaling the underlying compute infrastructure on the fly depending on the load constraints.
+
+
+## Can I write \\( \LaTeX \\) in my model card?
+
+Yes, we use the [KaTeX](https://katex.org/) math typesetting library to render math formulas server-side,
+before parsing the markdown.
+You have to use the following delimiters:
+- `$$ ... $$` for display mode
+- `\\` `(` `...` `\\` `)` for inline mode (no space between the slashes and the parenthesis).
+
+Then you'll be able to write:
+
+$$
+mse = (\frac{1}{n})\sum_{i=1}^{n}(y_{i} - x_{i})^{2}
+$$
+
+$$ e=mc^2 $$
diff --git a/docs/datasets-tags.md b/docs/datasets-tags.md
new file mode 100644
index 00000000..29cd15ac
--- /dev/null
+++ b/docs/datasets-tags.md
@@ -0,0 +1,61 @@
+---
+title: Datasets Tags documentation
+---
+
+<h1 class="no-top-margin">Datasets Tags documentation</h1>
+
+
+## Tasks
+
+- `conditional-text-generation`: data-to-text and text transduction tasks such as translation or summarization
+	- [`machine-translation`](/datasets?filter=task_ids:machine-translation)
+	- [`sentence-splitting-fusion`](/datasets?filter=task_ids:sentence-splitting-fusion)
+	- [`summarization`](/datasets?filter=task_ids:summarization)
+	- [`table-to-text`](/datasets?filter=task_ids:table-to-text)
+	- [`text-simplification`](/datasets?filter=task_ids:text-simplification)
+	- [`explanation-generation`](/datasets?filter=task_ids:explanation-generation)
+	- [`other-stuctured-to-text`](/datasets?filter=task_ids:other-stuctured-to-text)
+	- [`other`](/datasets?filter=task_ids:other)
+- `question-answering`: question answering tasks
+	- [`open-domain-qa`](/datasets?filter=task_ids:open-domain)
+	- [`closed-domain-qa`](/datasets?filter=task_ids:closed-domain)
+	- [`multiple-choice-qa`](/datasets?filter=task_ids:multiple-choice)
+	- [`extractive-qa`](/datasets?filter=task_ids:extractive-qa)
+	- [`abstractive-qa`](/datasets?filter=task_ids:abstractive-qa)
+	- [`other`](/datasets?filter=task_ids:other)
+- `sequence-modeling`: such as language modeling or dialogue
+	- [`dialogue-modeling`](/datasets?filter=task_ids:dialogue-modeling)
+	- [`language-modeling`](/datasets?filter=task_ids:language-modeling)
+	- [`other-multi-turn`](/datasets?filter=task_ids:other-multi-turn)
+	- [`slot-filling`](/datasets?filter=task_ids:slot-filling)
+	- [`other`](/datasets?filter=task_ids:other)
+- `structure-prediction`: predicting structural properties of the text, such as syntax
+	- [`coreference-resolution`](/datasets?filter=task_ids:coreference-resolution)
+	- [`named-entity-recognition`](/datasets?filter=task_ids:named-entity-recognition)
+	- [`part-of-speech-tagging`](/datasets?filter=task_ids:part-of-speech-tagging)
+	- [`parsing`](/datasets?filter=task_ids:parsing)
+	- [`other`](/datasets?filter=task_ids:other)
+- `text-classification`: predicting a class index or boolean value
+	- [`acceptability-classification`](/datasets?filter=task_ids:acceptability-classification)
+	- [`entity-linking-classification`](/datasets?filter=task_ids:entity-linking-classification)
+	- [`fact-checking`](/datasets?filter=task_ids:fact-checking)
+	- [`intent-classification`](/datasets?filter=task_ids:intent-classification)
+	- [`multi-class-classification`](/datasets?filter=task_ids:multi-class-classification)
+	- [`multi-label-classification`](/datasets?filter=task_ids:multi-label-classification)
+	- [`natural-language-inference`](/datasets?filter=task_ids:natural-language-inference)
+	- [`semantic-similarity-classification`](/datasets?filter=task_ids:semantic-similarity-classification)
+	- [`sentiment-classification`](/datasets?filter=task_ids:sentiment-classification)
+	- [`topic-classification`](/datasets?filter=task_ids:topic-classification)
+	- [`other`](/datasets?filter=task_ids:other)
+- `text-retrieval`: information or text retrieval tasks
+	- [`document-retrieval`](/datasets?filter=task_ids:document-retrieval)
+	- [`utterance-retrieval`](/datasets?filter=task_ids:utterance-retrieval)
+	- [`entity-linking-retrieval`](/datasets?filter=task_ids:entity-linking-retrieval)
+	- [`fact-checking-retrieval`](/datasets?filter=task_ids:fact-checking-retrieval)
+	- [`other`](/datasets?filter=task_ids:other)
+- `text-scoring`: text scoring tasks, predicting a real valued score for some text
+	- [`semantic-similarity-scoring`](/datasets?filter=task_ids:semantic-similarity-scoring)
+	- [`sentiment-scoring`](/datasets?filter=task_ids:sentiment-scoring)
+	- [`other`](/datasets?filter=task_ids:other)
+- `other`: other task family not mentioned here
+	- [`other`](/datasets?filter=task_ids:other)
diff --git a/modelcard.md b/modelcard.md
new file mode 100644
index 00000000..b0d8ab7f
--- /dev/null
+++ b/modelcard.md
@@ -0,0 +1,33 @@
+---
+language:
+- {lang_0}  # Example: fr
+- {lang_1}  # Example: en
+license: {license}  # Example: apache-2.0
+tags:
+- {tag_0}  # Example: audio
+- {tag_1}  # Example: automatic-speech-recognition
+- {tag_2}  # Example: speech
+- {tag_3}  # Example to specify a library: allennlp
+datasets:
+- {dataset_0}  # Example: common_voice
+metrics:
+- {metric_0}  # Example: wer
+
+model-index:  
+- name: {model_id}
+  results:
+  - task: 
+      name: {task_name}  # Example: Speech Recognition
+      type: {task_type}  # Example: automatic-speech-recognition
+    dataset:
+      name: {dataset_name}  # Example: Common Voice zh-CN
+      type: {dataset_type}  # Example: common_voice
+      args: {arg_0}  # Example: zh-CN
+    metrics:
+      - name: {metric_name}  # Example: Test WER
+        type: {metric_type}  # Example: wer
+        value: {metric_value}  # Example: 20.90
+        args: {arg_0}  # Example for BLEU: max_order
+---
+
+This markdown file contains the spec for the modelcard metadata regarding evaluation parameters. 
\ No newline at end of file
diff --git a/setup.py b/setup.py
index e91e00fc..e7dad993 100644
--- a/setup.py
+++ b/setup.py
@@ -15,6 +15,7 @@ install_requires = [
     "filelock",
     "requests",
     "tqdm",
+    "typing-extensions",
     "importlib_metadata;python_version<'3.8'",
 ]
 
diff --git a/src/huggingface_hub/__init__.py b/src/huggingface_hub/__init__.py
index 26a26df8..5b318046 100644
--- a/src/huggingface_hub/__init__.py
+++ b/src/huggingface_hub/__init__.py
@@ -16,7 +16,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "0.0.8"
+__version__ = "0.0.9"
 
 from .constants import (
     CONFIG_NAME,
diff --git a/src/huggingface_hub/file_download.py b/src/huggingface_hub/file_download.py
index 81bfb75e..d7b4994b 100644
--- a/src/huggingface_hub/file_download.py
+++ b/src/huggingface_hub/file_download.py
@@ -409,6 +409,15 @@ def cached_download(
 
     # Prevent parallel downloads of the same file with a lock.
     lock_path = cache_path + ".lock"
+
+    # Some Windows versions do not allow for paths longer than 255 characters.
+    # In this case, we must specify it is an extended path by using the "\\?\" prefix.
+    if os.name == "nt" and len(os.path.abspath(lock_path)) > 255:
+        lock_path = "\\\\?\\" + os.path.abspath(lock_path)
+
+    if os.name == "nt" and len(os.path.abspath(cache_path)) > 255:
+        cache_path = "\\\\?\\" + os.path.abspath(cache_path)
+
     with FileLock(lock_path):
 
         # If the download just completed while the lock was activated.
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index c4b2a5d2..9c9650b8 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -15,16 +15,29 @@
 
 
 import os
+import re
+import sys
 import warnings
+from io import BufferedIOBase, RawIOBase
 from os.path import expanduser
-from typing import Dict, List, Optional, Tuple
+from typing import BinaryIO, Dict, Iterable, List, Optional, Tuple, Union
+
+
+if sys.version_info >= (3, 8):
+    from typing import Literal
+else:
+    from typing_extensions import Literal
 
 import requests
+from requests.exceptions import HTTPError
 
-from .constants import REPO_TYPES
+from .constants import REPO_TYPE_DATASET, REPO_TYPE_DATASET_URL_PREFIX, REPO_TYPES
 
 
 ENDPOINT = "https://huggingface.co"
+REMOTE_FILEPATH_REGEX = re.compile(r"^\w[\w\/]*(\.\w+)?$")
+# ^^ No trailing slash, no backslash, no spaces, no relative parts ("." or "..")
+#    Only word characters and an optional extension
 
 
 class RepoObj:
@@ -57,6 +70,7 @@ class ModelInfo:
         self,
         modelId: Optional[str] = None,  # id of model
         sha: Optional[str] = None,  # commit sha at the specified revision
+        lastModified: Optional[str] = None,  # date of last commit to repo
         tags: List[str] = [],
         pipeline_tag: Optional[str] = None,
         siblings: Optional[
@@ -66,6 +80,7 @@ class ModelInfo:
     ):
         self.modelId = modelId
         self.sha = sha
+        self.lastModified = lastModified
         self.tags = tags
         self.pipeline_tag = pipeline_tag
         self.siblings = (
@@ -123,12 +138,68 @@ class HfApi:
         r = requests.post(path, headers={"authorization": "Bearer {}".format(token)})
         r.raise_for_status()
 
-    def list_models(self, filter: Optional[str] = None) -> List[ModelInfo]:
+    def list_models(
+        self,
+        filter: Union[str, Iterable[str], None] = None,
+        sort: Union[Literal["lastModified"], str, None] = None,
+        direction: Optional[Literal[-1]] = None,
+        limit: Optional[int] = None,
+        full: Optional[bool] = None,
+    ) -> List[ModelInfo]:
         """
         Get the public list of all the models on huggingface.co
+
+        Args:
+            filter (:obj:`str` or :class:`Iterable`, `optional`):
+                A string which can be used to identify models on the hub by their tags.
+                Example usage:
+
+                    >>> from huggingface_hub import HfApi
+                    >>> api = HfApi()
+
+                    >>> # List all models
+                    >>> api.list_models()
+
+                    >>> # List only the text classification models
+                    >>> api.list_models(filter="text-classification")
+
+                    >>> # List only the russian models compatible with pytorch
+                    >>> api.list_models(filter=("ru", "pytorch"))
+
+                    >>> # List only the models trained on the "common_voice" dataset
+                    >>> api.list_models(filter="dataset:common_voice")
+
+                    >>> # List only the models from the AllenNLP library
+                    >>> api.list_models(filter="allennlp")
+            sort (:obj:`Literal["lastModified"]` or :obj:`str`, `optional`):
+                The key with which to sort the resulting models. Possible values are the properties of the `ModelInfo`
+                class.
+            direction (:obj:`Literal[-1]` or :obj:`int`, `optional`):
+                Direction in which to sort. The value `-1` sorts by descending order while all other values
+                sort by ascending order.
+            limit (:obj:`int`, `optional`):
+                The limit on the number of models fetched. Leaving this option to `None` fetches all models.
+            full (:obj:`bool`, `optional`):
+                Whether to fetch all model data, including the `lastModified`, the `sha`, the files and the `tags`.
+                This is set to `True` by default when using a filter.
+
         """
         path = "{}/api/models".format(self.endpoint)
-        params = {"filter": filter, "full": True} if filter is not None else None
+        params = {}
+        if filter is not None:
+            params.update({"filter": filter})
+            params.update({"full": True})
+        if sort is not None:
+            params.update({"sort": sort})
+        if direction is not None:
+            params.update({"direction": direction})
+        if limit is not None:
+            params.update({"limit": limit})
+        if full is not None:
+            if full:
+                params.update({"full": True})
+            elif "full" in params:
+                del params["full"]
         r = requests.get(path, params=params)
         r.raise_for_status()
         d = r.json()
@@ -230,7 +301,15 @@ class HfApi:
         if exist_ok and r.status_code == 409:
             d = r.json()
             return d["url"]
-        r.raise_for_status()
+
+        try:
+            r.raise_for_status()
+        except HTTPError as e:
+            if r.json():
+                if "error" in r.json():
+                    raise HTTPError("{} - {}".format(e, r.json()["error"]))
+            raise e
+
         d = r.json()
         return d["url"]
 
@@ -264,6 +343,121 @@ class HfApi:
         )
         r.raise_for_status()
 
+    def upload_file(
+        self,
+        token: str,
+        path_or_fileobj: Union[str, BinaryIO],
+        path_in_repo: str,
+        repo_id: str,
+        repo_type: Optional[str] = None,
+        revision: Optional[str] = None,
+    ) -> str:
+        """
+        Upload a local file (up to 5GB) to the given repo, tracking it with LFS if it's larger than 10MB
+
+        Params:
+            token (``str``):
+                Authentication token, obtained with :function:`HfApi.login` method
+
+            path_or_fileobj (``str`` or ``BinaryIO``):
+                Path to a file on the local machine or binary data stream / fileobj.
+
+            path_in_repo (``str``):
+                Relative filepath in the repo, for example: :obj:`"checkpoints/1fec34a/weights.bin"`
+
+            repo_id (``str``):
+                The repository to which the file will be uploaded, for example: :obj:`"username/custom_transformers"`
+
+            repo_type (``str``, Optional):
+                Set to :obj:`"dataset"` if uploading to a dataset, :obj:`None` if uploading to a model. Default is :obj:`None`.
+
+            revision (``str``, Optional):
+                The git revision to commit from. Defaults to the :obj:`"main"` branch.
+
+        Returns:
+            ``str``: The URL to visualize the uploaded file on the hub
+
+        Raises:
+            :class:`ValueError`: if some parameter value is invalid
+
+            :class:`requests.HTTPError`: if the HuggingFace API returned an error
+
+        Examples:
+            >>> with open("./local/filepath", "rb") as fobj:
+            ...     upload_file(
+            ...         path_or_fileobj=fileobj,
+            ...         path_in_repo="remote/file/path.h5",
+            ...         repo_id="username/my-dataset",
+            ...         repo_type="datasets",
+            ...         token="my_token",
+            ...    )
+            "https://huggingface.co/datasets/username/my-dataset/blob/main/remote/file/path.h5"
+
+            >>> upload_file(
+            ...     path_or_fileobj=".\\\\local\\\\file\\\\path",
+            ...     path_in_repo="remote/file/path.h5",
+            ...     repo_id="username/my-model",
+            ...     token="my_token",
+            ... )
+            "https://huggingface.co/username/my-model/blob/main/remote/file/path.h5"
+
+
+        """
+        if repo_type not in REPO_TYPES:
+            raise ValueError("Invalid repo type, must be one of {}".format(REPO_TYPES))
+
+        # Validate path_or_fileobj
+        if isinstance(path_or_fileobj, str):
+            path_or_fileobj = os.path.normpath(os.path.expanduser(path_or_fileobj))
+            if not os.path.isfile(path_or_fileobj):
+                raise ValueError(
+                    "Provided path: '{}' is not a file".format(path_or_fileobj)
+                )
+        elif not isinstance(path_or_fileobj, (RawIOBase, BufferedIOBase)):
+            # ^^ Test from: https://stackoverflow.com/questions/44584829/how-to-determine-if-file-is-opened-in-binary-or-text-mode
+            raise ValueError(
+                "path_or_fileobj must be either an instance of str or BinaryIO. "
+                "If you passed a fileobj, make sure you've opened the file in binary mode."
+            )
+
+        # Normalize path separators and strip leading slashes
+        if not REMOTE_FILEPATH_REGEX.match(path_in_repo):
+            raise ValueError(
+                "Invalid path_in_repo '{}', path_in_repo must match regex {}".format(
+                    path_in_repo, REMOTE_FILEPATH_REGEX.pattern
+                )
+            )
+
+        repo_id = "{prefix}{repo_id}".format(
+            prefix=REPO_TYPE_DATASET_URL_PREFIX
+            if (repo_type == REPO_TYPE_DATASET)
+            else "",
+            repo_id=repo_id,
+        )
+
+        revision = revision if revision is not None else "main"
+
+        path = "{}/api/{repo_id}/upload/{revision}/{path_in_repo}".format(
+            self.endpoint,
+            repo_id=repo_id,
+            revision=revision,
+            path_in_repo=path_in_repo,
+        )
+
+        headers = (
+            {"authorization": "Bearer {}".format(token)} if token is not None else None
+        )
+
+        if isinstance(path_or_fileobj, str):
+            with open(path_or_fileobj, "rb") as bytestream:
+                r = requests.post(path, headers=headers, data=bytestream)
+        else:
+            r = requests.post(path, headers=headers, data=path_or_fileobj)
+
+        r.raise_for_status()
+        d = r.json()
+        return d["url"]
+
 
 class HfFolder:
     path_token = expanduser("~/.huggingface/token")


Test patch:
diff --git a/tests/test_hf_api.py b/tests/test_hf_api.py
index 70985671..f6fa3d6d 100644
--- a/tests/test_hf_api.py
+++ b/tests/test_hf_api.py
@@ -16,10 +16,13 @@
 import os
 import shutil
 import subprocess
+import tempfile
 import time
 import unittest
+from io import BytesIO
 
 from huggingface_hub.constants import REPO_TYPE_DATASET
+from huggingface_hub.file_download import cached_download
 from huggingface_hub.hf_api import HfApi, HfFolder, ModelInfo, RepoObj
 from requests.exceptions import HTTPError
 
@@ -88,6 +91,134 @@ class HfApiEndpointsTest(HfApiCommonTest):
         )
 
 
+class HfApiUploadFileTest(HfApiEndpointsTest):
+    def setUp(self) -> None:
+        super().setUp()
+        self.tmp_dir = tempfile.mkdtemp()
+        self.tmp_file = os.path.join(self.tmp_dir, "temp")
+        self.tmp_file_content = "Content of the file"
+        with open(self.tmp_file, "w+") as f:
+            f.write(self.tmp_file_content)
+        self.addCleanup(lambda: shutil.rmtree(self.tmp_dir))
+
+    def test_upload_file_validation(self):
+        with self.assertRaises(ValueError, msg="Wrong repo type"):
+            self._api.upload_file(
+                path_or_fileobj=self.tmp_file,
+                path_in_repo="README.md",
+                repo_id=f"{USER}/{REPO_NAME}",
+                repo_type="this type does not exist",
+                token=self._token,
+            )
+
+        with self.assertRaises(ValueError, msg="File opened in text mode"):
+            with open(self.tmp_file, "rt") as ftext:
+                self._api.upload_file(
+                    path_or_fileobj=ftext,
+                    path_in_repo="README.md",
+                    repo_id=f"{USER}/{REPO_NAME}",
+                    token=self._token,
+                )
+
+        with self.assertRaises(
+            ValueError, msg="path_or_fileobj is str but does not point to a file"
+        ):
+            self._api.upload_file(
+                path_or_fileobj=os.path.join(self.tmp_dir, "nofile.pth"),
+                path_in_repo="README.md",
+                repo_id=f"{USER}/{REPO_NAME}",
+                token=self._token,
+            )
+
+        for (invalid_path, msg) in [
+            ("Remote\\README.md", "Has a backslash"),
+            ("/Remote/README.md", "Starts with a slash"),
+            ("Remote/../subtree/./README.md", "Has relative parts"),
+        ]:
+            with self.subTest(msg=msg):
+                with self.assertRaises(ValueError, msg="path_in_repo is invalid"):
+                    self._api.upload_file(
+                        path_or_fileobj=self.tmp_file,
+                        path_in_repo=invalid_path,
+                        repo_id=f"{USER}/{REPO_NAME}",
+                        token=self._token,
+                    )
+
+    def test_upload_file_path(self):
+        self._api.create_repo(token=self._token, name=REPO_NAME)
+        try:
+            self._api.upload_file(
+                path_or_fileobj=self.tmp_file,
+                path_in_repo="temp/new_file.md",
+                repo_id=f"{USER}/{REPO_NAME}",
+                token=self._token,
+            )
+            url = "{}/{user}/{repo}/resolve/main/temp/new_file.md".format(
+                ENDPOINT_STAGING,
+                user=USER,
+                repo=REPO_NAME,
+            )
+            filepath = cached_download(url, force_download=True)
+            with open(filepath) as downloaded_file:
+                content = downloaded_file.read()
+            self.assertEqual(content, self.tmp_file_content)
+
+        except Exception as err:
+            self.fail(err)
+        finally:
+            self._api.delete_repo(token=self._token, name=REPO_NAME)
+
+    def test_upload_file_fileobj(self):
+        self._api.create_repo(token=self._token, name=REPO_NAME)
+        try:
+            with open(self.tmp_file, "rb") as filestream:
+                self._api.upload_file(
+                    path_or_fileobj=filestream,
+                    path_in_repo="temp/new_file.md",
+                    repo_id=f"{USER}/{REPO_NAME}",
+                    token=self._token,
+                )
+            url = "{}/{user}/{repo}/resolve/main/temp/new_file.md".format(
+                ENDPOINT_STAGING,
+                user=USER,
+                repo=REPO_NAME,
+            )
+            filepath = cached_download(url, force_download=True)
+            with open(filepath) as downloaded_file:
+                content = downloaded_file.read()
+            self.assertEqual(content, self.tmp_file_content)
+
+        except Exception as err:
+            self.fail(err)
+        finally:
+            self._api.delete_repo(token=self._token, name=REPO_NAME)
+
+    def test_upload_file_bytesio(self):
+        self._api.create_repo(token=self._token, name=REPO_NAME)
+        try:
+            filecontent = BytesIO(b"File content, but in bytes IO")
+            self._api.upload_file(
+                path_or_fileobj=filecontent,
+                path_in_repo="temp/new_file.md",
+                repo_id=f"{USER}/{REPO_NAME}",
+                token=self._token,
+            )
+            url = "{}/{user}/{repo}/resolve/main/temp/new_file.md".format(
+                ENDPOINT_STAGING,
+                user=USER,
+                repo=REPO_NAME,
+            )
+            filepath = cached_download(url, force_download=True)
+            with open(filepath) as downloaded_file:
+                content = downloaded_file.read()
+            self.assertEqual(content, filecontent.getvalue().decode())
+
+        except Exception as err:
+            self.fail(err)
+        finally:
+            self._api.delete_repo(token=self._token, name=REPO_NAME)
+
+
 class HfApiPublicTest(unittest.TestCase):
     def test_staging_list_models(self):
         _api = HfApi(endpoint=ENDPOINT_STAGING)
@@ -99,6 +230,21 @@ class HfApiPublicTest(unittest.TestCase):
         self.assertGreater(len(models), 100)
         self.assertIsInstance(models[0], ModelInfo)
 
+    def test_list_models_complex_query(self):
+        # Let's list the 10 most recent models
+        # with tags "bert" and "jax",
+        # ordered by last modified date.
+        _api = HfApi()
+        models = _api.list_models(
+            filter=("bert", "jax"), sort="lastModified", direction=-1, limit=10
+        )
+        # we have at least 1 models
+        self.assertGreater(len(models), 1)
+        self.assertLessEqual(len(models), 10)
+        model = models[0]
+        self.assertIsInstance(model, ModelInfo)
+        self.assertTrue(all(tag in model.tags for tag in ["bert", "jax"]))
+
     def test_model_info(self):
         _api = HfApi()
         model = _api.model_info(repo_id=DUMMY_MODEL_ID)