Repository: lipoja/URLExtract

Issue:
Does Not extract the URL that is leading special character
Hi,

I am trying to extract the URL from the following text,
But it is not grabbing the link due to the hyphen.

EncodedText="* test link -https://www.google.com";

extractor = URLExtract()
urls = extractor.find_urls(EncodedText)

print(URLs)

---output
[]

Do you have any idea to fix this?


 

Discussion:

Patch:
diff --git a/urlextract/urlextract_core.py b/urlextract/urlextract_core.py
index 0f5bc0a..8f7d5c8 100644
--- a/urlextract/urlextract_core.py
+++ b/urlextract/urlextract_core.py
@@ -488,6 +488,9 @@ class URLExtract(CacheFile):
         # URL should not start with two backslashes
         if complete_url.startswith("//"):
             complete_url = complete_url[2:]
+        # URL should not start with unreserved characters
+        if complete_url.startswith(("-", ".", "~", "_")):
+            complete_url = complete_url[1:]
         if not self._is_domain_valid(
             complete_url, tld, check_dns=check_dns, with_schema_only=with_schema_only
         ):


Test patch:
diff --git a/tests/unit/test_find_urls.py b/tests/unit/test_find_urls.py
index 03b6c17..d776bbd 100644
--- a/tests/unit/test_find_urls.py
+++ b/tests/unit/test_find_urls.py
@@ -37,8 +37,8 @@ import pytest
             ["https://example.com/what.com"],
         ),
         (
-            "https://i2.wp.com/siliconfilter.com/2011/06/example.jpg",
-            ["https://i2.wp.com/siliconfilter.com/2011/06/example.jpg"],
+            "* test link -https://www.example.com",
+            ["https://www.example.com"],
         ),
         (
             "https://www.test.org/paper/apostrophe'in-url",