Repository: yt-dlp/yt-dlp
Pull request #8348 (2023-10-14 20:18:46)

[rh:requests] Handle both `bytes` and `int` for `IncompleteRead.partial`
**IMPORTANT**: PRs without the template will be CLOSED

### Description of your *pull request* and other information

<!--

Explanation of your *pull request* in arbitrary form goes here. Please **make sure the description explains the purpose and effect** of your *pull request* and is worded well enough to be understood. Provide as much **context and examples** as possible

-->
This PR handles both `bytes` and `int` for `IncompleteRead.partial`. The API of `http.client.IncompleteRead` and `urllib3.exceptions.IncompleteRead` are incompatible here.

<details open><summary>Template</summary> <!-- OPEN is intentional -->

<!--

# PLEASE FOLLOW THE GUIDE BELOW

- You will be asked some questions, please read them **carefully** and answer honestly
- Put an `x` into all the boxes `[ ]` relevant to your *pull request* (like [x])
- Use *Preview* tab to see how your *pull request* will actually look like

-->

### Before submitting a *pull request* make sure you have:
- [x] At least skimmed through [contributing guidelines](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions) including [yt-dlp coding conventions](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions)
- [x] [Searched](https://github.com/yt-dlp/yt-dlp/search?q=is%3Apr&type=Issues) the bugtracker for similar pull requests
- [x] Checked the code with [flake8](https://pypi.python.org/pypi/flake8) and [ran relevant tests](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#developer-instructions)

### In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under [Unlicense](http://unlicense.org/). Check all of the following options that apply:
- [x] I am the original author of this code and I am willing to release it under [Unlicense](http://unlicense.org/)
- [ ] I am not the original author of this code but it is in public domain or released under [Unlicense](http://unlicense.org/) (provide reliable evidence)

### What is the purpose of your *pull request*?
- [ ] Fix or improvement to an extractor (Make sure to add/update tests)
- [ ] New extractor ([Piracy websites will not be accepted](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#is-the-website-primarily-used-for-piracy))
- [x] Core bug fix/improvement
- [ ] New feature (It is strongly [recommended to open an issue first](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#adding-new-feature-or-making-overarching-changes))


<!-- Do NOT edit/remove anything below this! -->
</details><details><summary>Copilot Summary</summary>  

<!--
copilot:all
-->
### <samp>🤖 Generated by Copilot at ad29447</samp>

### Summary
🐛📚🚀

<!--
1.  🐛 - This emoji represents a bug fix, which is the main purpose of the first change.
2.  📚 - This emoji represents documentation, which is the category of the second change, as it updates the type annotation of the code.
3.  🚀 - This emoji represents a performance improvement, which is a possible outcome of the first change, as it avoids raising an unnecessary exception and handles the connection closing more gracefully.
-->
Fix a bug in reading responses from servers that close the connection early and update the type annotation syntax for union types in `yt_dlp/networking/exceptions.py`.

> _Fix `AttributeError`_
> _when server closes early_
> _autumn leaves incomplete_

### Walkthrough
* Fix a bug that causes an `AttributeError` when reading a response from a server that closes the connection prematurely ([link](https://github.com/yt-dlp/yt-dlp/pull/8348/files?diff=unified&w=0#diff-ef5ef94777b4b7a8fd1579b2053264fb5068b9e7baf33e95c23d3541a9eab1e4L156-R159))
* Update the type annotation of the `expected` parameter of the `IncompleteRead` exception to use the `|` operator instead of the `,` operator ([link](https://github.com/yt-dlp/yt-dlp/pull/8348/files?diff=unified&w=0#diff-e9d38cb769fe93d3c21fff5e55791529fcee61111522d27b2cbeda281ac94cc6L78-R78))



</details>



Patch:
diff --git a/yt_dlp/networking/_requests.py b/yt_dlp/networking/_requests.py
index 27974357a..fe3f60b0b 100644
--- a/yt_dlp/networking/_requests.py
+++ b/yt_dlp/networking/_requests.py
@@ -142,18 +142,17 @@ def read(self, amt: int = None):
         except urllib3.exceptions.SSLError as e:
             raise SSLError(cause=e) from e
 
-        except urllib3.exceptions.IncompleteRead as e:
-            # urllib3 IncompleteRead.partial is always an integer
-            raise IncompleteRead(partial=e.partial, expected=e.expected) from e
-
         except urllib3.exceptions.ProtocolError as e:
-            # http.client.IncompleteRead may be contained within ProtocolError
+            # IncompleteRead is always contained within ProtocolError
             # See urllib3.response.HTTPResponse._error_catcher()
             ir_err = next(
                 (err for err in (e.__context__, e.__cause__, *variadic(e.args))
                  if isinstance(err, http.client.IncompleteRead)), None)
             if ir_err is not None:
-                raise IncompleteRead(partial=len(ir_err.partial), expected=ir_err.expected) from e
+                # `urllib3.exceptions.IncompleteRead` is subclass of `http.client.IncompleteRead`
+                # but uses an `int` for its `partial` property.
+                partial = ir_err.partial if isinstance(ir_err.partial, int) else len(ir_err.partial)
+                raise IncompleteRead(partial=partial, expected=ir_err.expected) from e
             raise TransportError(cause=e) from e
 
         except urllib3.exceptions.HTTPError as e:
diff --git a/yt_dlp/networking/exceptions.py b/yt_dlp/networking/exceptions.py
index f58dc246e..12441901c 100644
--- a/yt_dlp/networking/exceptions.py
+++ b/yt_dlp/networking/exceptions.py
@@ -75,7 +75,7 @@ def __repr__(self):
 
 
 class IncompleteRead(TransportError):
-    def __init__(self, partial: int, expected: int = None, **kwargs):
+    def __init__(self, partial: int, expected: int | None = None, **kwargs):
         self.partial = partial
         self.expected = expected
         msg = f'{partial} bytes read'


Test patch:
diff --git a/test/test_networking.py b/test/test_networking.py
index 2b45deac7..689161fb2 100644
--- a/test/test_networking.py
+++ b/test/test_networking.py
@@ -865,7 +865,7 @@ def request(self, *args, **kwargs):
             '3 bytes read, 4 more expected'
         ),
         (
-            lambda: urllib3.exceptions.IncompleteRead(partial=3, expected=5),
+            lambda: urllib3.exceptions.ProtocolError('error', urllib3.exceptions.IncompleteRead(partial=3, expected=5)),
             IncompleteRead,
             '3 bytes read, 5 more expected'
         ),