Repository: sherlock-project/sherlock

Issue:
Yandex Music has a captcha
<!--

######################################################################
  WARNING!
  IGNORING THE FOLLOWING TEMPLATE WILL RESULT IN ISSUE CLOSED AS INCOMPLETE
######################################################################

-->

## Checklist
<!--
Put x into all boxes (like this [x]) once you have completed what they say.
Make sure complete everything in the checklist.
-->
- [x] I'm reporting a website that is returning **false positive** results
- [x] I've checked for similar site support requests including closed ones
- [x] I've checked for pull requests attempting to fix this false positive
- [x] I'm only reporting **one** site (create a separate issue for each site)

## Description
<!--
Provide the username that is causing Sherlock to return a false positive, along with any other information that might help us fix this false positive.
-->
Here's a random username that can't possibly exist: [ecfhlmiuewfimcuhem](https://music.yandex/users/ecfhlmiuewfimcuhem/playlists).

Here's the username from [data.json](https://github.com/sherlock-project/sherlock/blob/master/sherlock/resources/data.json#L2214): [ya.playlist](https://music.yandex/users/ya.playlist/playlists)

When I visit either, I get a captcha (note: JS is disabled in my browser):
![image](https://github.com/sherlock-project/sherlock/assets/65649991/3c93012d-991a-46a2-b0af-60edf5772eec)

Unless Sherlock uses Selenium/Pyppeteer, which i highly doubt (it's not in requirements.txt), this captcha isn't really avoidable (I think). Maybe it even shows up with JS enabled, which I didn't check.

I'm not opening a PR removing YandexMusic because it could be an issue that only happens for me, or maybe it's possible to bypass this captcha.
Use lite version of archive.org, change error message.
Archive.org detects the use of old browsers, and redirects them to a "lite" version of archive.org. For some reason, the requests module is also redirected. This PR fixes this issue.


Discussion:

Resolving patch:
diff --git a/sherlock/sherlock.py b/sherlock/sherlock.py
--- a/sherlock/sherlock.py
+++ b/sherlock/sherlock.py
@@ -232,7 +232,7 @@ def sherlock(
         # A user agent is needed because some sites don't return the correct
         # information since they think that we are bots (Which we actually are...)
         headers = {
-            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0",
+            "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0",
         }
 
         if "headers" in net_info:
@@ -351,7 +351,6 @@ def sherlock(
 
         # Get the expected error type
         error_type = net_info["errorType"]
-        error_code = net_info.get("errorCode")
 
         # Retrieve future and ensure it has finished
         future = net_info["request_future"]
@@ -407,13 +406,16 @@ def sherlock(
             else:
                 query_status = QueryStatus.AVAILABLE
         elif error_type == "status_code":
-            # Checks if the Status Code is equal to the optional "errorCode" given in 'data.json'
-            if error_code == r.status_code:
+            error_codes = net_info.get("errorCode")
+            query_status = QueryStatus.CLAIMED
+
+            # Type consistency, allowing for both singlets and lists in manifest
+            if isinstance(error_codes, int):
+                error_codes = [error_codes]
+            
+            if error_codes is not None and r.status_code in error_codes:
                 query_status = QueryStatus.AVAILABLE
-            # Checks if the status code of the response is 2XX
-            elif not r.status_code >= 300 or r.status_code < 200:
-                query_status = QueryStatus.CLAIMED
-            else:
+            elif r.status_code >= 300 or r.status_code < 200:
                 query_status = QueryStatus.AVAILABLE
         elif error_type == "response_url":
             # For this detection method, we have turned off the redirect.


Test patch: