Repository: yt-dlp/yt-dlp
Issue category: Bug Report

[archive.org] ERROR: can only concatenate str (not "NoneType") to str - sporadic, only on certain URLs
### DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

- [X] I understand that I will be **blocked** if I *intentionally* remove or skip any mandatory\* field

### Checklist

- [X] I'm reporting that yt-dlp is broken on a **supported** site
- [X] I've verified that I have **updated yt-dlp to nightly or master** ([update instructions](https://github.com/yt-dlp/yt-dlp#update-channels))
- [X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
- [X] I've checked that all URLs and arguments with special characters are [properly quoted or escaped](https://github.com/yt-dlp/yt-dlp/wiki/FAQ#video-url-contains-an-ampersand--and-im-getting-some-strange-output-1-2839-or-v-is-not-recognized-as-an-internal-or-external-command)
- [X] I've searched [known issues](https://github.com/yt-dlp/yt-dlp/issues/3766) and the [bugtracker](https://github.com/yt-dlp/yt-dlp/issues?q=) for similar issues **including closed ones**. DO NOT post duplicates
- [X] I've read the [guidelines for opening an issue](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#opening-an-issue)
- [X] I've read about [sharing account credentials](https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed) and I'm willing to share it if required

### Region

United States

### Provide a description that is worded well enough to be understood

I'm getting the error in the title, and in the verbose output, when attempting to download certain archive.org URLs, but not others.

Downloading https://archive.org/details/gd95-04-02.sbd.11622.sbeok.shnf fails with the provided log output; however, downloading, say, https://archive.org/details/gd1995-04-02.nak300.holtz.91056.flac16 is successful.

However, I was able to "fix" the bug by editing my local installation of `yt-dlp`. Apparently, in some cases, the "reviewbody" attribute might be missing from the review, which causes a `TypeError` when an attempt at string concatenation is made. Forcing the body to empty-string in these cases was enough to bypass the crash and allow the download to proceed.
```diff
diff --git a/yt_dlp/extractor/archiveorg.py b/yt_dlp/extractor/archiveorg.py
index f5a55efc4..2869e5233 100644
--- a/yt_dlp/extractor/archiveorg.py
+++ b/yt_dlp/extractor/archiveorg.py
@@ -335,7 +335,7 @@ def _real_extract(self, url):
                 info['comments'].append({
                     'id': review.get('review_id'),
                     'author': review.get('reviewer'),
-                    'text': str_or_none(review.get('reviewtitle'), '') + '\n\n' + review.get('reviewbody'),
+                    'text': str_or_none(review.get('reviewtitle'), '') + '\n\n' + str_or_none(review.get('reviewbody'), ''),
                     'timestamp': unified_timestamp(review.get('createdate')),
                     'parent': 'root'})
```
I put "fix" in quotes, though, because I'm not familiar enough with the yt-dlp codebase as a whole to know whether this "fix" doesn't cause its own problems; I'm decent at Python and I figured I might as well take a stab at patching over the "obvious" problem, and it did work in my case.

However, it might well be the case that this hack breaks other components of the archive.org extractor - for example, some sort of other advanced functionality that I'm not familiar with (that my simple download request didn't invoke), which depends on a correctly-parsed review body in order to do its job. 

That said, I can certainly file that PR if a maintainer indicates that the change wouldn't have unintended consequences.

### Provide verbose output that clearly demonstrates the problem

- [X] Run **your** yt-dlp command with **-vU** flag added (`yt-dlp -vU <your command line>`)
- [ ] If using API, add `'verbose': True` to `YoutubeDL` params instead
- [X] Copy the WHOLE output (starting with `[debug] Command-line config`) and insert it below

### Complete Verbose Output

```shell
[debug] Command-line config: ['-vU', '--playlist-items', '4:5', 'https://archive.org/details/gd95-04-02.sbd.11622.sbeok.shnf']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.11.04 from yt-dlp/yt-dlp [197d0b03b]
[debug] Python 3.12.7 (CPython x86_64 64bit) - Linux-6.11.6-arch1-1-x86_64-with-glibc2.40 (OpenSSL 3.4.0 22 Oct 2024, glibc 2.40)
[debug] exe versions: ffmpeg 7.1 (setts), ffprobe 7.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.08.30, mutagen-1.47.0, requests-2.32.3, sqlite3-3.46.1, urllib3-1.26.20, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests
[debug] Loaded 1838 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.11.04 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.11.04 from yt-dlp/yt-dlp)
[archive.org] Extracting URL: https://archive.org/details/gd95-04-02.sbd.11622.sbeok.shnf
[archive.org] gd95-04-02.sbd.11622.sbeok.shnf: Downloading webpage
[archive.org] gd95-04-02.sbd.11622.sbeok.shnf: Downloading JSON metadata
ERROR: can only concatenate str (not "NoneType") to str
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 1625, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/yt_dlp/YoutubeDL.py", line 1760, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/yt_dlp/extractor/common.py", line 742, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/yt_dlp/extractor/archiveorg.py", line 338, in _real_extract
    'text': str_or_none(review.get('reviewtitle'), '') + '\n\n' + review.get('reviewbody'),
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: can only concatenate str (not "NoneType") to str
```



Functions to edit:
yt_dlp/extractor/archiveorg.py:ArchiveOrgIE
yt_dlp/extractor/archiveorg.py:ArchiveOrgIE._real_extract
Functions added: yt_dlp/extractor/archiveorg.py:ArchiveOrgIE

Patch:
diff --git a/yt_dlp/extractor/archiveorg.py b/yt_dlp/extractor/archiveorg.py
index f5a55efc4ff1..2849d9fd5b0d 100644
--- a/yt_dlp/extractor/archiveorg.py
+++ b/yt_dlp/extractor/archiveorg.py
@@ -205,6 +205,26 @@ class ArchiveOrgIE(InfoExtractor):
                 },
             },
         ],
+    }, {
+        # The reviewbody is None for one of the reviews; just need to extract data without crashing
+        'url': 'https://archive.org/details/gd95-04-02.sbd.11622.sbeok.shnf/gd95-04-02d1t04.shn',
+        'info_dict': {
+            'id': 'gd95-04-02.sbd.11622.sbeok.shnf/gd95-04-02d1t04.shn',
+            'ext': 'mp3',
+            'title': 'Stuck Inside of Mobile with the Memphis Blues Again',
+            'creators': ['Grateful Dead'],
+            'duration': 338.31,
+            'track': 'Stuck Inside of Mobile with the Memphis Blues Again',
+            'description': 'md5:764348a470b986f1217ffd38d6ac7b72',
+            'display_id': 'gd95-04-02d1t04.shn',
+            'location': 'Pyramid Arena',
+            'uploader': 'jon@archive.org',
+            'album': '1995-04-02 - Pyramid Arena',
+            'upload_date': '20040519',
+            'track_number': 4,
+            'release_date': '19950402',
+            'timestamp': 1084927901,
+        },
     }]
 
     @staticmethod
@@ -335,7 +355,7 @@ def _real_extract(self, url):
                 info['comments'].append({
                     'id': review.get('review_id'),
                     'author': review.get('reviewer'),
-                    'text': str_or_none(review.get('reviewtitle'), '') + '\n\n' + review.get('reviewbody'),
+                    'text': join_nonempty('reviewtitle', 'reviewbody', from_dict=review, delim='\n\n'),
                     'timestamp': unified_timestamp(review.get('createdate')),
                     'parent': 'root'})