Repository: miurahr/aqtinstall

Issue #517: `MetadataFactory` makes inappropriate assumptions about the HTML files it retrieves.
Created: 2022-04-20T01:30:19Z
Describe the bug
MetadataFactory improperly assumes that HTML pages at download.qt.io will always contain a body that contains a table. Infrequently, the server will return something unexpected, and aqt should be able to provide a reasonable error message.
To Reproduce
Run aqt list-qt linux desktop, under the condition that the URL https://download.qt.io/online/qtsdkrepository/linux_x64/desktop/ returns a page without a table.
Expected behavior
The aqt list-qt linux desktop command returns a list of Qt versions.
aqt output
'NoneType' object has no attribute 'find_all'
Traceback (most recent call last):
File "/home/garald/.local/lib/python3.8/site-packages/aqt/installer.py", line 108, in run
args.func(args)
File "/home/garald/.local/lib/python3.8/site-packages/aqt/installer.py", line 524, in run_list_qt
show_list(meta)
File "/home/garald/.local/lib/python3.8/site-packages/aqt/metadata.py", line 775, in show_list
output = meta.getList()
File "/home/garald/.local/lib/python3.8/site-packages/aqt/metadata.py", line 435, in getList
return self._action()
File "/home/garald/.local/lib/python3.8/site-packages/aqt/metadata.py", line 473, in fetch_versions
versions = sorted(filter(None, map(get_version, filter(filter_by, versions_extensions))))
File "/home/garald/.local/lib/python3.8/site-packages/aqt/metadata.py", line 600, in iterate_folders
for row in soup.body.table.find_all("tr"):
AttributeError: 'NoneType' object has no attribute 'find_all'
aqtinstall(aqt) v2.1.0 on Python 3.8.10 [CPython GCC 9.4.0]
Working dir: /home/garald
Arguments: ['/home/garald/.local/bin/aqt', 'list-qt', 'linux', 'desktop'] Host: uname_result(system='Linux', node='garald-H310M-S2-2-0', release='5.4.0-109-generic', version='#123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022', machine='x86_64', processor='x86_64')
===========================PLEASE FILE A BUG REPORT===========================
You have discovered a bug in aqt.
Please file a bug report at https://github.com/miurahr/aqtinstall/issues.
Please remember to include a copy of this program's output in your report.

Desktop (please complete the following information):

OS: Linux
aqtinstall(aqt) v2.1.0 on Python 3.8.10 [CPython GCC 9.4.0]

Additional context
This report comes from an off-topic comment posted in #496, but it is certainly a bug that needs to be addressed. I do not have all the context for this bug, but there's enough information in the program output to understand what happened.

Duplicate issue #519: aqtinstall does not work when https://download.qt.io is not available
Created: 2022-04-26T12:53:14Z
Hello! When https://download.qt.io is not available it is not possible to use mirrors from the settings.ini config to list available tools:
$ aqt list-tool linux desktop
'NoneType' object has no attribute 'find_all'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/aqt/installer.py", line 108, in run
    args.func(args)
  File "/usr/local/lib/python3.8/dist-packages/aqt/installer.py", line 540, in run_list_tool
    show_list(meta)
  File "/usr/local/lib/python3.8/dist-packages/aqt/metadata.py", line 775, in show_list
    output = meta.getList()
  File "/usr/local/lib/python3.8/dist-packages/aqt/metadata.py", line 435, in getList
    return self._action()
  File "/usr/local/lib/python3.8/dist-packages/aqt/metadata.py", line 482, in fetch_tools
    return list(MetadataFactory.iterate_folders(html_doc, "tools"))
  File "/usr/local/lib/python3.8/dist-packages/aqt/metadata.py", line 600, in iterate_folders
    for row in soup.body.table.find_all("tr"):
AttributeError: 'NoneType' object has no attribute 'find_all'
aqtinstall(aqt) v2.1.0 on Python 3.8.10 [CPython GCC 9.4.0]
Working dir: `/home/user`
Arguments: `['/usr/local/bin/aqt', 'list-tool', 'linux', 'desktop']` Host: `uname_result(system='Linux', node='laptop', release='5.13.0-40-generic', version='#45~20.04.1-Ubuntu SMP Mon Apr 4 09:38:31 UTC 2022', machine='x86_64', processor='x86_64')`
===========================PLEASE FILE A BUG REPORT===========================
You have discovered a bug in aqt.
Please file a bug report at https://github.com/miurahr/aqtinstall/issues.
Please remember to include a copy of this program's output in your report.

Is it possible to use aqtinstall with mirrors when https://download.qt.io is not available?

Closing comment (2022-04-26T14:24:59Z): The stacktrace looks like it could be a duplicate of #517, which already has a fix in the master branch. Can you please try again with the master branch? I don't think there's been a release yet that includes the patch.
If setting the urls to something else doesn't work, please post the stacktrace/error messages here; I would like to see them.