Repository: Serene-Arc/bulk-downloader-for-reddit

Issue #385: [BUG] Filenames exceed the MAX_PATH limit of Windows
Created: 2021-05-16T21:41:18Z
Description
Filenames exceeds the MAX_PATH limit of Windows
Command
python -m bdfr archive ~/Desktop/hello --all-comments --user me

Environment (please complete the following information):

OS: Windows 10
Python version: 3.9.4

Logs
[2021-05-16 15:24:27,202 - bdfr.downloader - DEBUG] - Setting maximum download wait time to 120 seconds
[2021-05-16 15:24:27,202 - bdfr.downloader - DEBUG] - Setting datetime format string to ISO
[2021-05-16 15:24:27,203 - bdfr.downloader - Level 9] - Created download filter
[2021-05-16 15:24:27,203 - bdfr.downloader - Level 9] - Created time filter
[2021-05-16 15:24:27,203 - bdfr.downloader - Level 9] - Created sort filter
[2021-05-16 15:24:27,203 - bdfr.downloader - Level 9] - Create file name formatter
[2021-05-16 15:24:27,203 - bdfr.downloader - DEBUG] - Using authenticated Reddit instance
[2021-05-16 15:24:27,203 - bdfr.downloader - Level 9] - Commencing OAuth2 authentication
[2021-05-16 15:24:27,607 - bdfr.oauth2 - WARNING] - Authentication action required before the program can proceed
[2021-05-16 15:24:27,608 - bdfr.oauth2 - WARNING] - Authenticate at https://www.reddit.com/api/v1/authorize?client_id=U-6gk4ZCh3IeNQ&duration=permanent&redirect_uri=http%3A%2F%2Flocalhost%3A7634&response_type=code&scope=identity+read+history+save&state=47870
[2021-05-16 15:24:27,610 - bdfr.oauth2 - Level 9] - Server listening on 0.0.0.0:7634
[2021-05-16 15:24:41,642 - bdfr.oauth2 - Level 9] - Server closed
[2021-05-16 15:24:42,046 - bdfr.oauth2 - Level 9] - Loaded OAuth2 token for authoriser
[2021-05-16 15:24:42,493 - bdfr.oauth2 - Level 9] - Written OAuth2 token from authoriser to C:\Users\aliparlakci\AppData\Local\BDFR\bdfr\default_config.cfg
[2021-05-16 15:24:42,946 - bdfr.downloader - Level 9] - Resolved user to aliparlakci
[2021-05-16 15:24:42,946 - bdfr.downloader - Level 9] - Created site authenticator
[2021-05-16 15:24:42,946 - bdfr.downloader - Level 9] - Retrieved subreddits
[2021-05-16 15:24:42,946 - bdfr.downloader - Level 9] - Retrieved multireddits
[2021-05-16 15:24:42,946 - bdfr.archiver - DEBUG] - Retrieving comments of user aliparlakci
[2021-05-16 15:24:42,947 - bdfr.downloader - Level 9] - Retrieved user data
[2021-05-16 15:24:42,947 - bdfr.downloader - Level 9] - Retrieved submissions for given links
[2021-05-16 15:24:43,991 - bdfr.archiver - DEBUG] - Attempting to archive submission gyba24d
[2021-05-16 15:24:45,314 - bdfr.archiver - DEBUG] - Writing entry gyba24d to file in JSON format at C:\Users\aliparlakci\Desktop\hello\DataHoarder\aliparlakci_Bulk Downloader for Reddit, tool for archiving reddit, has a major release!_gyba24d.json
[2021-05-16 15:24:45,314 - bdfr.archiver - INFO] - Record for entry item gyba24d written to disk
[2021-05-16 15:26:21,087 - bdfr.archiver - DEBUG] - Attempting to archive submission er558z3
[2021-05-16 15:26:25,017 - root - ERROR] - Downloader exited unexpectedly
Traceback (most recent call last):
  File "C:\Users\aliparlakci\source\bulk-downloader-for-reddit\bdfr\__main__.py", line 86, in cli_archive
    reddit_archiver.download()
  File "C:\Users\aliparlakci\source\bulk-downloader-for-reddit\bdfr\archiver.py", line 32, in download
    self._write_entry(submission)
  File "C:\Users\aliparlakci\source\bulk-downloader-for-reddit\bdfr\archiver.py", line 65, in _write_entry
    self._write_entry_json(archive_entry)
  File "C:\Users\aliparlakci\source\bulk-downloader-for-reddit\bdfr\archiver.py", line 77, in _write_entry_json
    self._write_content_to_disk(resource, content)
  File "C:\Users\aliparlakci\source\bulk-downloader-for-reddit\bdfr\archiver.py", line 92, in _write_content_to_disk
    with open(file_path, 'w', encoding="utf-8") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\aliparlakci\\Desktop\\hello\\Showerthoughts\\aliparlakci_If you are cautious enough to carry one of those pointy car window-breakers in case you drive your car into a lake, you are probably cautious enough to not drive your car into a lake in the first place._er558z3.json'

Duplicate issue #426: [BUG]Program will not download certain files
Created: 2021-05-25T04:09:23Z
[Yes ] I am reporting a bug.
[Yes ] I am running the latest version of BDfR
[ Yes] I have read the Opening an issue

Description
Program will not download files
Command
python -m bdfr download ./path/to/output --subreddit dataisbeautiful --sort top --time week --limit 100
Environment (please complete the following information):
Windows 10
Python 3.9,2
Logs
[2021-05-24 20:58:15,990 - bdfr.downloader - DEBUG] - Setting maximum download wait time to 120 seconds
[2021-05-24 20:58:15,990 - bdfr.downloader - DEBUG] - Setting datetime format string to ISO
[2021-05-24 20:58:15,991 - bdfr.downloader - Level 9] - Created download filter
[2021-05-24 20:58:15,991 - bdfr.downloader - Level 9] - Created time filter
[2021-05-24 20:58:15,991 - bdfr.downloader - Level 9] - Created sort filter
[2021-05-24 20:58:15,991 - bdfr.downloader - Level 9] - Create file name formatter
[2021-05-24 20:58:15,991 - bdfr.downloader - DEBUG] - Using unauthenticated Reddit instance
[2021-05-24 20:58:16,168 - bdfr.downloader - Level 9] - Created site authenticator
[2021-05-24 20:58:16,744 - bdfr.downloader - DEBUG] - Added submissions from subreddit dataisbeautiful
[2021-05-24 20:58:16,744 - bdfr.downloader - Level 9] - Retrieved subreddits
[2021-05-24 20:58:16,744 - bdfr.downloader - Level 9] - Retrieved multireddits
[2021-05-24 20:58:16,744 - bdfr.downloader - Level 9] - Retrieved user data
[2021-05-24 20:58:16,744 - bdfr.downloader - Level 9] - Retrieved submissions for given links
[2021-05-24 20:58:18,251 - bdfr.downloader - DEBUG] - Attempting to download submission nhnw0a
[2021-05-24 20:58:19,577 - bdfr.downloader - DEBUG] - Using YoutubeDlFallback with url https://v.redd.it/t119cpsltf071
[2021-05-24 20:58:30,325 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\jcceagle_[OC] The Covid-19 death toll_nhnw0a.m4a
[2021-05-24 20:58:30,325 - bdfr.downloader - DEBUG] - Hash added to master list: 63a563a4e2e7bc6a635580d5a5aa7770
[2021-05-24 20:58:30,325 - bdfr.downloader - INFO] - Downloaded submission nhnw0a from dataisbeautiful
[2021-05-24 20:58:30,326 - bdfr.downloader - DEBUG] - Attempting to download submission nghzv7
[2021-05-24 20:58:30,327 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/n9c6zh7oc5071.png
[2021-05-24 20:58:30,579 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\academiaadvice_[OC] Who Makes More Teachers or Cops_nghzv7.png
[2021-05-24 20:58:30,579 - bdfr.downloader - DEBUG] - Hash added to master list: fb41aceca52b1a666ca7eacb694f8dda
[2021-05-24 20:58:30,579 - bdfr.downloader - INFO] - Downloaded submission nghzv7 from dataisbeautiful
[2021-05-24 20:58:30,579 - bdfr.downloader - DEBUG] - Attempting to download submission ng417y
[2021-05-24 20:58:30,580 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/pivjllgki2071.png
[2021-05-24 20:58:30,800 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\Udzu_Color composition of Red-White-Blue flags [OC]ng417y.png
[2021-05-24 20:58:30,800 - bdfr.downloader - DEBUG] - Hash added to master list: 60c550e533cfa60edd5f9b712981397b
[2021-05-24 20:58:30,800 - bdfr.downloader - INFO] - Downloaded submission ng417y from dataisbeautiful
[2021-05-24 20:58:30,801 - bdfr.downloader - DEBUG] - Attempting to download submission nh3o3g
[2021-05-24 20:58:31,059 - bdfr.downloader - DEBUG] - Using YoutubeDlFallback with url https://v.redd.it/3mac3awlqa071
[2021-05-24 20:58:33,825 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\PieChartPirate[OC] Covid-19 Vaccination Doses Administered per 100 in the G20_nh3o3g.m4a
[2021-05-24 20:58:33,825 - bdfr.downloader - DEBUG] - Hash added to master list: 58b93ab361e9d93e0816d9bc1c415615
[2021-05-24 20:58:33,825 - bdfr.downloader - INFO] - Downloaded submission nh3o3g from dataisbeautiful
[2021-05-24 20:58:33,825 - bdfr.downloader - DEBUG] - Attempting to download submission njgzfp
[2021-05-24 20:58:33,826 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/bhl1g242ox071.jpg
[2021-05-24 20:58:35,956 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\JoeWDavies_[OC] Digital elevation model of Greece_njgzfp.jpg
[2021-05-24 20:58:35,956 - bdfr.downloader - DEBUG] - Hash added to master list: 2c61af19f666e82126918d5485fb7d87
[2021-05-24 20:58:35,956 - bdfr.downloader - INFO] - Downloaded submission njgzfp from dataisbeautiful
[2021-05-24 20:58:35,958 - bdfr.downloader - DEBUG] - Attempting to download submission nig4gv
[2021-05-24 20:58:36,274 - bdfr.downloader - DEBUG] - Using YoutubeDlFallback with url https://v.redd.it/dqktgd4zkn071
[2021-05-24 20:58:38,717 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\PieChartPirate_[OC] Mobile Operating System Market Share 2000 - 2021_nig4gv.m4a
[2021-05-24 20:58:38,717 - bdfr.downloader - DEBUG] - Hash added to master list: 1df5d5291276950f00ab32b28cf62c34
[2021-05-24 20:58:38,717 - bdfr.downloader - INFO] - Downloaded submission nig4gv from dataisbeautiful
[2021-05-24 20:58:38,717 - bdfr.downloader - DEBUG] - Attempting to download submission nfdtdf
[2021-05-24 20:58:38,718 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/mjmv6ytrewz61.jpg
[2021-05-24 20:58:39,057 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\migueltres_[OC] How often was the phrase Oh, my God spoken on the FRIENDS television show_nfdtdf.jpg
[2021-05-24 20:58:39,057 - bdfr.downloader - DEBUG] - Hash added to master list: 1fc3e3e2e0e564a745cb881104fc8100
[2021-05-24 20:58:39,057 - bdfr.downloader - INFO] - Downloaded submission nfdtdf from dataisbeautiful
[2021-05-24 20:58:39,058 - bdfr.downloader - DEBUG] - Attempting to download submission njw7kg
[2021-05-24 20:58:39,058 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/if9brfwt32171.png
[2021-05-24 20:58:39,391 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\lfg10101_[OC] Comparing Emissions Sources - How to Shrink your Carbon Footprint More Effectively_njw7kg.png
[2021-05-24 20:58:39,391 - bdfr.downloader - DEBUG] - Hash added to master list: e2a591568b41e490db7b892c270a7100
[2021-05-24 20:58:39,391 - bdfr.downloader - INFO] - Downloaded submission njw7kg from dataisbeautiful
[2021-05-24 20:58:39,392 - bdfr.downloader - DEBUG] - Attempting to download submission nia8e5
[2021-05-24 20:58:39,392 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/jw0fjnzdil071.jpg
[2021-05-24 20:58:39,724 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\shivamdamani_Most distinguishing words spoken by the cast of FRIENDS [OC]nia8e5.jpg
[2021-05-24 20:58:39,724 - bdfr.downloader - DEBUG] - Hash added to master list: 2cf12045df2c5654bb4684021601e7ae
[2021-05-24 20:58:39,724 - bdfr.downloader - INFO] - Downloaded submission nia8e5 from dataisbeautiful
[2021-05-24 20:58:39,725 - bdfr.downloader - DEBUG] - Attempting to download submission nj5qix
[2021-05-24 20:58:39,725 - bdfr.downloader - DEBUG] - Using Direct with url https://i.redd.it/xlu6jmp0tu071.gif
[2021-05-24 20:58:56,658 - bdfr.downloader - DEBUG] - Written file to C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\desfirsit[OC] Fastest routes on land (and sometimes, boat) between all 990 pairs of European capitals_nj5qix.gif
[2021-05-24 20:58:56,658 - bdfr.downloader - DEBUG] - Hash added to master list: d00d8c5e4ad9999b863f6d6c0738a007
[2021-05-24 20:58:56,659 - bdfr.downloader - INFO] - Downloaded submission nj5qix from dataisbeautiful
[2021-05-24 20:58:56,668 - bdfr.downloader - DEBUG] - Attempting to download submission nit0bb
[2021-05-24 20:58:56,670 - bdfr.downloader - DEBUG] - Using Direct with url https://i.imgur.com/eAfONbE.png
[2021-05-24 20:58:56,934 - root - ERROR] - Downloader exited unexpectedly
Traceback (most recent call last):
File "C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\bdfr_main_.py", line 67, in cli_download
reddit_downloader.download()
File "C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\bdfr\downloader.py", line 397, in download
self._download_submission(submission)
File "C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\bdfr\downloader.py", line 440, in download_submission
with open(destination, 'wb') as file:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\Owen\Downloads\bulk-downloader-for-reddit-2.1.1\bulk-downloader-for-reddit-2.1.1\path\to\output\dataisbeautiful\IconicIsotope[OC] Knight moves - a simple table I made showing the importance of keeping your knights near the middle of the chessboard_nit0bb.png'

Closing comment (2021-05-25T13:09:48Z): Duplicate of #385