internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: huggingface/optimum

Issue:
Unpin onnx version to allow using v1.14
### Feature request

Remove the version constraint on onnx package to allow using onnx==1.14.

### Motivation

- The latest version of onnxruntime (v1.15 at the time of writing) supports onnx==1.14.
- onnx==1.14 introduces support for protobuf v4, which is also useful.

### Your contribution

Seems removing the pin from setup.py would be all that is needed as long as the existing tests provide enough validation.


Discussion:
Sounds good to me @cody-moveworks ! Would you like to open a PR with this change?
@regisss I'll try to put up a PR with this change.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -41,21 +41,21 @@
 
 EXTRAS_REQUIRE = {
     "onnxruntime": [
-        "onnx<1.14.0",
+        "onnx",
         "onnxruntime>=1.9.0",
         "datasets>=1.2.1",
         "evaluate",
         "protobuf>=3.20.1",
     ],
     "onnxruntime-gpu": [
-        "onnx<1.14.0",
+        "onnx",
         "onnxruntime-gpu>=1.9.0",
         "datasets>=1.2.1",
         "evaluate",
         "protobuf>=3.20.1",
     ],
-    "exporters": ["onnx<1.14.0", "onnxruntime", "timm"],
-    "exporters-gpu": ["onnx<1.14.0", "onnxruntime-gpu", "timm"],
+    "exporters": ["onnx", "onnxruntime", "timm"],
+    "exporters-gpu": ["onnx", "onnxruntime-gpu", "timm"],
     "exporters-tf": ["tensorflow>=2.4,<2.11", "tf2onnx", "onnx", "onnxruntime", "timm", "h5py", "numpy<1.24.0"],
     "intel": "optimum-intel",
     "openvino": "optimum-intel[openvino]",


Test patch:
Repository: huggingface/optimum

Issue:
Running ORT stable diffusion downloads openvino_model.xml
### System Info

```shell
optimum main
transformers main
diffusers 0.20
huggingface-hub 0.16.4
```


### Who can help?

/

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction (minimal, reproducible, runnable)

Run

```
from optimum.onnxruntime import ORTStableDiffusionXLPipeline

model_id = "stabilityai/stable-diffusion-xl-base-1.0"
base = ORTStableDiffusionXLPipeline.from_pretrained(model_id)
```

We see:

```
Downloading openvino_model.xml: 100%|███████████████████████████████████████████████████████████████████████████████████| 22.6M/22.6M [00:00<00:00, 96.6MB/s]
```

which should not happen

### Expected behavior

/


Discussion:

Resolving patch:
diff --git a/optimum/onnxruntime/modeling_diffusion.py b/optimum/onnxruntime/modeling_diffusion.py
--- a/optimum/onnxruntime/modeling_diffusion.py
+++ b/optimum/onnxruntime/modeling_diffusion.py
@@ -315,7 +315,7 @@ def _from_pretrained(
                 use_auth_token=use_auth_token,
                 revision=revision,
                 allow_patterns=allow_patterns,
-                ignore_patterns=["*.msgpack", "*.safetensors", "*.bin"],
+                ignore_patterns=["*.msgpack", "*.safetensors", "*.bin", "*.xml"],
             )
         new_model_save_dir = Path(model_id)
 


Test patch:
Repository: huggingface/huggingface_hub

Issue:
Python versions with `rc...` suffix are not recognized
Error Message

```
../lib/python3.8/site-packages/huggingface_hub/file_download.py", line 37, in <genexpr>
    if tuple(int(i) for i in _PY_VERSION.split(".")) < (3, 8, 0):
ValueError: invalid literal for int() with base 10: '3rc1'
```


My Python version: `3.8.3rc1`


Discussion:
I think this might be fixed by a variation of #27 

Anyone wants to tackle this?

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -17,6 +17,7 @@ def get_version() -> str:
     "tqdm",
     "typing-extensions",
     "importlib_metadata;python_version<'3.8'",
+    "packaging>=20.9"
 ]
 
 extras = {}
diff --git a/src/huggingface_hub/file_download.py b/src/huggingface_hub/file_download.py
--- a/src/huggingface_hub/file_download.py
+++ b/src/huggingface_hub/file_download.py
@@ -13,6 +13,7 @@
 from pathlib import Path
 from typing import BinaryIO, Dict, Optional, Tuple, Union
 
+import packaging.version
 from tqdm.auto import tqdm
 
 import requests
@@ -32,9 +33,9 @@
 
 logger = logging.getLogger(__name__)
 
-_PY_VERSION: str = sys.version.split()[0]
+_PY_VERSION: str = sys.version.split()[0].rstrip("+")
 
-if tuple(int(i) for i in _PY_VERSION.split(".")) < (3, 8, 0):
+if packaging.version.Version(_PY_VERSION) < packaging.version.Version("3.8.0"):
     import importlib_metadata
 else:
     import importlib.metadata as importlib_metadata


Test patch:
Repository: huggingface/huggingface_hub

Issue:
Error when creating a repository
This error happens on huggingface_hub version 0.0.14 (current version) when creating a new dataset repository

```bash
(py38) bash-3.2$ huggingface-cli repo create --type dataset codeparrot-train
git version 2.29.2
git-lfs/2.13.3 (GitHub; darwin amd64; go 1.16.2; git a5e65851)

Traceback (most recent call last):
  File "/Users/thomwolf/miniconda2/envs/py38/bin/huggingface-cli", line 8, in <module>
    sys.exit(main())
  File "/Users/thomwolf/miniconda2/envs/py38/lib/python3.8/site-packages/huggingface_hub/commands/huggingface_cli.py", line 41, in main
    service.run()
  File "/Users/thomwolf/miniconda2/envs/py38/lib/python3.8/site-packages/huggingface_hub/commands/user.py", line 228, in run
    user, _ = self._api.whoami(token)
ValueError: too many values to unpack (expected 2)
```


Discussion:

Resolving patch:
diff --git a/src/huggingface_hub/commands/user.py b/src/huggingface_hub/commands/user.py
--- a/src/huggingface_hub/commands/user.py
+++ b/src/huggingface_hub/commands/user.py
@@ -224,7 +224,7 @@ def run(self):
             )
         print("")
 
-        user, _ = self._api.whoami(token)
+        user = self._api.whoami(token)["name"]
         namespace = (
             self.args.organization if self.args.organization is not None else user
         )


Test patch:
Repository: huggingface/huggingface_hub

Issue:
Add timeout parameter to HfApi.dataset_info to work on JZ
In certain environments such as the Jean Zay cluster, there is no internet connection and internet requests hang indefinitely.

To be able to properly raise an error in such cases instead of making the program hang, I think it would be nice to add a `timeout` parameter to `HfApi.dataset_info` that can be passed to `requests.get`.

The `datasets` CI has a test where it simulates such an environment. So when I try to use `HfApi.dataset_info` in `datasets`, it makes the CI fail.


Discussion:

Resolving patch:
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -492,7 +492,11 @@ def list_datasets(
         return [DatasetInfo(**x) for x in d]
 
     def model_info(
-        self, repo_id: str, revision: Optional[str] = None, token: Optional[str] = None
+        self,
+        repo_id: str,
+        revision: Optional[str] = None,
+        token: Optional[str] = None,
+        timeout: Optional[float] = None,
     ) -> ModelInfo:
         """
         Get info on one specific model on huggingface.co
@@ -509,7 +513,7 @@ def model_info(
         headers = (
             {"authorization": "Bearer {}".format(token)} if token is not None else None
         )
-        r = requests.get(path, headers=headers)
+        r = requests.get(path, headers=headers, timeout=timeout)
         r.raise_for_status()
         d = r.json()
         return ModelInfo(**d)
@@ -532,7 +536,11 @@ def list_repos_objs(
         return [RepoObj(**x) for x in d]
 
     def dataset_info(
-        self, repo_id: str, revision: Optional[str] = None, token: Optional[str] = None
+        self,
+        repo_id: str,
+        revision: Optional[str] = None,
+        token: Optional[str] = None,
+        timeout: Optional[float] = None,
     ) -> DatasetInfo:
         """
         Get info on one specific dataset on huggingface.co
@@ -550,7 +558,7 @@ def dataset_info(
             {"authorization": "Bearer {}".format(token)} if token is not None else None
         )
         params = {"full": "true"}
-        r = requests.get(path, headers=headers, params=params)
+        r = requests.get(path, headers=headers, params=params, timeout=timeout)
         r.raise_for_status()
         d = r.json()
         return DatasetInfo(**d)


Test patch:
Repository: huggingface/huggingface_hub

Issue:
chore: Updated the pillow version specifier
Hello there :wave: 

Following up on #755, I believe the core build requirements were meant to be updated as well. This aligns the version specifier of Pillow between the core build and the "tests" extra build!

Any feedback is welcome!

cc @osanseviero @Narsil 


Discussion:

Resolving patch:
diff --git a/api-inference-community/setup.py b/api-inference-community/setup.py
--- a/api-inference-community/setup.py
+++ b/api-inference-community/setup.py
@@ -3,7 +3,7 @@
 
 setup(
     name="api_inference_community",
-    version="0.0.21",
+    version="0.0.23",
     description="A package with helper tools to build an API Inference docker app for Hugging Face API inference using huggingface_hub",
     url="http://github.com/huggingface/api-inference-community",
     author="Nicolas Patry",


Test patch:
Repository: huggingface/huggingface_hub

Issue:
Relax version constraint on `requests`
#815 introduces a requirement on the minimal version of `requests` (`>=2.27`)

This requirement should be relaxed because 2.27 is way too recent

> Let's merge this with this pinned version, I'll open a PR today to fix it and remove the constraint before we release a new version. The pinned version is way too recent. cc @LysandreJik

_Originally posted by @adrinjalali in https://github.com/huggingface/huggingface_hub/pull/815#discussion_r841533043_


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -13,7 +13,7 @@ def get_version() -> str:
 
 install_requires = [
     "filelock",
-    "requests>=2.27",
+    "requests",
     "tqdm",
     "pyyaml",
     "typing-extensions>=3.7.4.3",  # to be able to import TypeAlias
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -22,7 +22,7 @@
 from typing import IO, Dict, Iterable, List, Optional, Tuple, Union
 
 import requests
-from requests.exceptions import HTTPError, JSONDecodeError
+from requests.exceptions import HTTPError
 
 from .constants import (
     ENDPOINT,
@@ -33,6 +33,7 @@
 )
 from .utils import logging
 from .utils._deprecation import _deprecate_positional_args
+from .utils._fixes import JSONDecodeError
 from .utils.endpoint_helpers import (
     AttributeDictionary,
     DatasetFilter,
diff --git a/src/huggingface_hub/utils/_fixes.py b/src/huggingface_hub/utils/_fixes.py
new file mode 100644
--- /dev/null
+++ b/src/huggingface_hub/utils/_fixes.py
@@ -0,0 +1,10 @@
+# JSONDecodeError was introduced in requests=2.27 released in 2022.
+# This allows us to support older requests for users
+# More information: https://github.com/psf/requests/pull/5856
+try:
+    from requests import JSONDecodeError  # noqa
+except ImportError:
+    try:
+        from simplejson import JSONDecodeError  # noqa
+    except ImportError:
+        from json import JSONDecodeError  # noqa


Test patch:
Repository: huggingface/huggingface_hub

Issue:
Make it easier to modify the model card content
Modifying `card.text` does not modify `card.content`, which is what is pushed to the Hub.


```
card = ModelCard.load('akdeniz27/roberta-large-cuad')
card.text = """
This is the new model card
"""
card.content = str(card)
```

It would work, but it has the last unintuitive step of modifying `content. Ideally, when `text` is modified, the `content` should also be modified.

cc @nateraw 



Discussion:
@nateraw should I assign this issue to you? 
[Related thread](https://huggingface.slack.com/archives/C025H22A3ND/p1667260413472069) (internal slack discussion).

A good way mentioned to solve this is to use a `@property` for the `content` field.
in skops, we had an intermediate object to handle this and we rendered during saving separately, i.e. you can edit all you want, it will only effect the README.md during save, and we have methods to add sections & more programmatically. (but @BenjaminBossan is refactoring again 😂)

Resolving patch:
diff --git a/src/huggingface_hub/repocard.py b/src/huggingface_hub/repocard.py
--- a/src/huggingface_hub/repocard.py
+++ b/src/huggingface_hub/repocard.py
@@ -79,7 +79,22 @@ def __init__(self, content: str):
 
         </Tip>
         """
+
+        # Set the content of the RepoCard, as well as underlying .data and .text attributes.
+        # See the `content` property setter for more details.
         self.content = content
+
+    @property
+    def content(self):
+        """The content of the RepoCard, including the YAML block and the Markdown body."""
+        line_break = _detect_line_ending(self._content) or "\n"
+        return f"---{line_break}{self.data.to_yaml(line_break=line_break)}{line_break}---{line_break}{self.text}"
+
+    @content.setter
+    def content(self, content: str):
+        """Set the content of the RepoCard."""
+        self._content = content
+
         match = REGEX_YAML_BLOCK.search(content)
         if match:
             # Metadata found in the YAML block
@@ -101,8 +116,7 @@ def __init__(self, content: str):
         self.data = self.card_data_class(**data_dict)
 
     def __str__(self):
-        line_break = _detect_line_ending(self.content) or "\n"
-        return f"---{line_break}{self.data.to_yaml(line_break=line_break)}{line_break}---{line_break}{self.text}"
+        return self.content
 
     def save(self, filepath: Union[Path, str]):
         r"""Save a RepoCard to a file.


Test patch:
diff --git a/tests/test_repocard.py b/tests/test_repocard.py
--- a/tests/test_repocard.py
+++ b/tests/test_repocard.py
@@ -656,6 +656,14 @@ def test_preserve_windows_linebreaks(self):
         card = RepoCard.load(card_path)
         self.assertIn("\r\n", str(card))
 
+    def test_updating_text_updates_content(self):
+        sample_path = SAMPLE_CARDS_DIR / "sample_simple.md"
+        card = RepoCard.load(sample_path)
+        card.text = "Hello, world!"
+        self.assertEqual(
+            card.content, f"---\n{card.data.to_yaml()}\n---\nHello, world!"
+        )
+
 
 class ModelCardTest(TestCaseWithCapLog):
     def test_model_card_with_invalid_model_index(self):
Repository: huggingface/huggingface_hub

Issue:
RepoUrl should be able to parse canonical datasets
**Example:**
- `"datasets/squad"` => `"repo_id": "squad"`, `"repo_type": "dataset"`
- `"hf://datasets/squad"` => `"repo_id": "squad", "repo_type": "dataset"`

At the moment, those ids are considered as models with namespace="datasets".
Related to [this comment](https://github.com/huggingface/hffs/pull/12#discussion_r1102937474) in hffs.


Discussion:

Resolving patch:
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -138,6 +138,10 @@ def repo_type_and_id_from_hf_id(hf_id: str, hub_url: Optional[str] = None) -> Tu
             namespace = None
         if len(url_segments) > 2 and hub_url not in url_segments[-3]:
             repo_type = url_segments[-3]
+        elif namespace in REPO_TYPES_MAPPING:
+            # Mean canonical dataset or model
+            repo_type = REPO_TYPES_MAPPING[namespace]
+            namespace = None
         else:
             repo_type = None
     elif is_hf_id:
@@ -145,9 +149,15 @@ def repo_type_and_id_from_hf_id(hf_id: str, hub_url: Optional[str] = None) -> Tu
             # Passed <repo_type>/<user>/<model_id> or <repo_type>/<org>/<model_id>
             repo_type, namespace, repo_id = url_segments[-3:]
         elif len(url_segments) == 2:
-            # Passed <user>/<model_id> or <org>/<model_id>
-            namespace, repo_id = hf_id.split("/")[-2:]
-            repo_type = None
+            if url_segments[0] in REPO_TYPES_MAPPING:
+                # Passed '<model_id>' or 'datasets/<dataset_id>' for a canonical model or dataset
+                repo_type = REPO_TYPES_MAPPING[url_segments[0]]
+                namespace = None
+                repo_id = hf_id.split("/")[-1]
+            else:
+                # Passed <user>/<model_id> or <org>/<model_id>
+                namespace, repo_id = hf_id.split("/")[-2:]
+                repo_type = None
         else:
             # Passed <model_id>
             repo_id = url_segments[0]


Test patch:
diff --git a/tests/test_hf_api.py b/tests/test_hf_api.py
--- a/tests/test_hf_api.py
+++ b/tests/test_hf_api.py
@@ -2447,3 +2447,17 @@ def test_repo_url_url_property(self):
         self.assertEqual(url.url, "https://huggingface.co/gpt2")
         self.assertIsInstance(url, RepoUrl)
         self.assertNotIsInstance(url.url, RepoUrl)
+
+    def test_repo_url_canonical_model(self):
+        for _id in ("gpt2", "hf://gpt2", "https://huggingface.co/gpt2"):
+            with self.subTest(_id):
+                url = RepoUrl(_id)
+                self.assertEqual(url.repo_id, "gpt2")
+                self.assertEqual(url.repo_type, "model")
+
+    def test_repo_url_canonical_dataset(self):
+        for _id in ("datasets/squad", "hf://datasets/squad", "https://huggingface.co/datasets/squad"):
+            with self.subTest(_id):
+                url = RepoUrl(_id)
+                self.assertEqual(url.repo_id, "squad")
+                self.assertEqual(url.repo_type, "dataset")
Repository: huggingface/huggingface_hub

Issue:
Log at warning level when waiting for commands to finish
Related to https://github.com/huggingface/diffusers/issues/2851 (cc @yiyixuxu)

The idea is to update [repository.py](https://github.com/huggingface/huggingface_hub/blob/ce2789c6238eed9ae329366d8409246feb220af0/src/huggingface_hub/repository.py#L1455) to log a warning instead of an error when exiting a training script if the background push commands are not completed yet (_"Waiting for the following commands to finish before shutting down:."_). 

Error level is a bit misleading as it suggests that the push failed.

(for reference, it has been introduced in Sept 2021 in https://github.com/huggingface/huggingface_hub/pull/315)


Discussion:

Resolving patch:
diff --git a/src/huggingface_hub/repository.py b/src/huggingface_hub/repository.py
--- a/src/huggingface_hub/repository.py
+++ b/src/huggingface_hub/repository.py
@@ -1452,7 +1452,7 @@ def wait_for_commands(self):
 
         while self.commands_in_progress:
             if index % 10 == 0:
-                logger.error(
+                logger.warning(
                     f"Waiting for the following commands to finish before shutting down: {self.commands_in_progress}."
                 )
 


Test patch:
Repository: huggingface/huggingface_hub

Issue:
AttributeError: type object 'tqdm' has no attribute '_lock'
### Describe the bug

Getting a tqdm issue when writing a Dask dataframe to the hub.

Similar to huggingface/datasets#6066. Using latest Datasets version doesn't seem to resolve it

### Steps to reproduce the bug

This is a minimal reproducer:
```
import dask.dataframe as dd
import pandas as pd
import random

import huggingface_hub

data = {"number": [random.randint(0,10) for _ in range(1000)]}
df = pd.DataFrame.from_dict(data)
dataframe = dd.from_pandas(df, npartitions=1)
dataframe = dataframe.repartition(npartitions=2)

repo_id = "nielsr/test-dask"
repo_path = f"hf://datasets/{repo_id}"
huggingface_hub.create_repo(repo_id=repo_id, repo_type="dataset", exist_ok=True)
dd.to_parquet(dataframe, path=f"{repo_path}/data")
```
Note: I'm intentionally repartioning the Dask dataframe to 2 partitions, as it does work when only having one partition. 

### Expected behavior

Would expect to write to the hub without any problem.

### Environment info

Datasets version 2.14.4


Discussion:
This snippet doesn't contain `datasets` code, so I think it can be transferred to `huggingface_hub`, no?
Oh yes, feel free to transfer
@NielsRogge What's your full stacktrace? And which version of `tqdm` are you using ?

```
pip freeze | grep tqdm
```
Ah I see, so the issue has already being identified and fixed in https://github.com/huggingface/datasets/pull/6068 ?
We can make the exact same fix for `huggingface_hub` as well. But very weird that I can't reproduce myself and that no-one complained so far :confused: 

**EDIT:** I'll not have time to fix this today and I'll be off next week. @NielsRogge  If you need it right now, feel free to edit the [tqdm](https://github.com/huggingface/huggingface_hub/blob/b9901bf71839850c10ea42162c69aecabc96a46b/src/huggingface_hub/utils/tqdm.py#L122) class here. I don't want to rush the fix now as I'm not able to reproduce the bug yet.

Resolving patch:
diff --git a/src/huggingface_hub/utils/tqdm.py b/src/huggingface_hub/utils/tqdm.py
--- a/src/huggingface_hub/utils/tqdm.py
+++ b/src/huggingface_hub/utils/tqdm.py
@@ -131,6 +131,14 @@ def __init__(self, *args, **kwargs):
             kwargs["disable"] = True
         super().__init__(*args, **kwargs)
 
+    def __delattr__(self, attr: str) -> None:
+        """Fix for https://github.com/huggingface/huggingface_hub/issues/1603"""
+        try:
+            super().__delattr__(attr)
+        except AttributeError:
+            if attr != "_lock":
+                raise
+
 
 @contextmanager
 def tqdm_stream_file(path: Union[Path, str]) -> Iterator[io.BufferedReader]:


Test patch: