Repository: lycheeverse/lychee

Issue #1296: Issues with relative links
Created: 2023-11-13T21:29:46Z
A user was trying out lychee for NLnet and found that version 0.13 misreports many links.
E.g.
lychee https://nlnet.nl/thema/OpenDocumentFormat.html

Issues found in 1 input. Find details below.

[https://nlnet.nl/thema/OpenDocumentFormat.html]:
✗ [404] https://nlnet.nl/BinaryAnalysisFund.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Services+Applications.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/InternetInfrastructure.html | Failed: Network error: Not Found
✗ [ERR] http://www.viewerjs.org/examples | Failed: Network error: The certificate was not trusted.
✗ [404] https://nlnet.nl/GetEduroam.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Deployability.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Decentralisedsolutions.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Conferences.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NetworkApplications.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Applicationprotocols.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/EducationalPrograms.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/InternetHardeningFund.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Reportsandstudies.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NGIZeroCore.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NGIZeroDiscovery.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/InformationRetrieval.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Verticals+Search.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/User-operatedInternetFund.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NGI0Entrust.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Measurement.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Middlewareandidentity.html | Failed: Network error: Not Found
✗ [500] http://translate.org.za/ | Failed: Network error: Internal Server Error
✗ [404] https://nlnet.nl/Networkinfrastructure.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Privacyandsecurity.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/CommunityPrograms.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/OperatingSystems.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NGIAssure.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/DataandAI.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/OpenDocumentFormat.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Softwareengineering.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/Hardware.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/OpenData.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NREN.html | Failed: Network error: Not Found
✗ [404] https://nlnet.nl/NGIZeroPET.html | Failed: Network error: Not Found

🔍 113 Total (in 11s) ✅ 78 OK 🚫 34 Errors 💤 1 Excluded

The errors are all from relative links.
It incorrectly strips away the path.
E.g. instead of https://nlnet.nl/thema/BinaryAnalysisFund.html it detects https://nlnet.nl/BinaryAnalysisFund.html

Duplicate issue #1480: Bug: lychee can not detect error relative url
Created: 2024-08-07T00:17:04Z
for this site: https://awang-01.github.io/testing/, there is an image with src="testing/images/lychee.png" that I was expecting to fail, but
lychee -v https://awang-01.github.io/testing/
✔ [200] https://awang-01.github.io/testing/images/lychee.png

🔍 1 Total (in 0s) ✅ 1 OK 🚫 0 Errors


the https://awang-01.github.iotesting/images/lychee.png should fail, but lychee automatically add a / before the src

Closing comment (2024-08-07T01:01:56Z): No, mre. You got that wrong.
The image on https://awang-01.github.io/testing/ has a relative src of testing/images/lychee.png and should result in an absolute URL of https://awang-01.github.io/testing/testing/images/lychee.png (see duplicated testing) which would fail, because it does not exist (which is expected here).
But lychee uses https://awang-01.github.io/testing/images/lychee.png which is the correct path to the image, but wrong in this context, because the relative image URL was resolved as relative to the hostname instead of the current location, I think.
I think this could be a duplicate of #1296


Edit: Simple reproduction:
(For the other way around. Pages noted as missing, when they're there.)
File tree:
root
└── test
    ├── index.html
    └── next.html
root/test/index.html:
<a href="next.html">next</a>
root/test/next.html:
just needs to exist.
Serve a site from root (e.g. python3 -m http.server -d . 3000)
lychee http://localhost:3000/test/
# or
lychee http://localhost:3000/test/index.html
Results in:
> lychee http://localhost:3000/test/index.html                                                                                                                                                                                                               
  1/1 ━━━━━━━━━━━━━━━━━━━━ Finished extracting links                                                                                                                                                                                                                                       Issues found in 1 input. Find details below.

[http://localhost:3000/test/index.html]:
✗ [404] http://localhost:3000/next.html | Failed: Network error: Not Found

🔍 1 Total (in 0s) ✅ 0 OK 🚫 1 Error
As you can see, the relative link is not resolved correctly by lychee.
You can open the entry page in a browser of your choice and see that you can access the next page.

Just as a note:
> lychee --version
lychee 0.15.1

NoteAlso, running the above example like lychee .  where . == root, means, testing on file system instead of http(s), works correctly.