Repository: pymedusa/Medusa

Issue:
Jackett use?
Is anyone else using Jackett? I'm seeing log hits for Jackett searches but nothing else showing number of items returned. Manual search in jackett returns results but I'm not seeing any when manual searching in Medusa. I've also checked the categories which seem to be set ok


Discussion:
Debug mode is not more verbose unfortunately either 
I use it... are you sure that you have it set up correctly? It's returning results for manual and daily searches for me. Here is how it should look like:
![img](https://snapr.pw/i/7956d2e037.png)
Yep that's how I have it configured as well. Using TPB as an example of a
public site w/ lots of results, nothing comes up in manual search (despite
also having it checked as a manual results option under Provider options.
I've never used it before but in the Jackett UI it pulls results fine.
Medusa logs show the search completes normally but even in debug doesn't
return anything more helpful unfortunately (even # of results or URL used
would be helpful).

On Wed, Nov 1, 2017 at 11:09 PM Dario <notifications@github.com> wrote:

> I use it... are you sure that you have it set up correctly? It's returning
> results for manual and daily searches for me. Here is how it should look
> like:
> [image: img]
> <https://camo.githubusercontent.com/0548aeafd58948181a2e07fc5ad725b02230acd7/68747470733a2f2f736e6170722e70772f692f373935366432653033372e706e67>
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/pymedusa/Medusa/issues/3213#issuecomment-341270220>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AASBwUiQPtpSCsZfdh1e-d5Al1OtWJOFks5syPoygaJpZM4QO6Iq>
> .
>

Try with all 3 options enabled, that's how I have it configured:
![img](https://snapr.pw/i/288e84fe25.png)
Hm. So I wasn't seeing the full debug logs earlier (my stupidity).
Looking at a manual search using TPB as an example, the logs show:

```
2017-11-01 23:31:26 INFO     SEARCHQUEUE-MANUAL-311954 :: [JACKETT RUTRACKER] :: [b16e0e5] Performing episode search for Riverdale
2017-11-01 23:31:26 INFO     SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Performing season pack search for Riverdale
2017-11-01 23:31:26 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Fallback season pack search initiate
2017-11-01 23:31:25 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:25 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&maxage=2700&cat=5030%2C5040&q=Riverdale+S02E02&limit=100&t=search&offset=0 [Status: 200]
2017-11-01 23:31:22 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search show using search string: Riverdale S02E02
2017-11-01 23:31:22 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:22 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&maxage=2700&cat=5030%2C5040&q=Riverdale+US+S02E02&limit=100&t=search&offset=0 [Status: 200]
2017-11-01 23:31:19 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search show using search string: Riverdale US S02E02
2017-11-01 23:31:19 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] No results returned from provider. Check chosen Newznab search categories in provider settings and/or usenet retention
2017-11-01 23:31:19 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:19 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&maxage=2700&cat=5030%2C5040&q=Riverdale+%28USA%29+S02E02&limit=100&t=search&offset=0 [Status: 200]
2017-11-01 23:31:16 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search show using search string: Riverdale (USA) S02E02
2017-11-01 23:31:16 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search mode: Episode
2017-11-01 23:31:15 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:15 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&maxage=2700&cat=5030%2C5040&q=Riverdale+S02E02&limit=100&t=search&offset=0 [Status: 200]
2017-11-01 23:31:12 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search show using search string: Riverdale S02E02
2017-11-01 23:31:12 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:12 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&maxage=2700&cat=5030%2C5040&q=Riverdale+US+S02E02&limit=100&t=search&offset=0 [Status: 200]
2017-11-01 23:31:09 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search show using search string: Riverdale US S02E02
2017-11-01 23:31:09 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] No results returned from provider. Check chosen Newznab search categories in provider settings and/or usenet retention
2017-11-01 23:31:09 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:09 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&maxage=2700&cat=5030%2C5040&q=Riverdale+%28USA%29+S02E02&limit=100&t=search&offset=0 [Status: 200]
2017-11-01 23:31:06 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search show using search string: Riverdale (USA) S02E02
2017-11-01 23:31:06 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Search mode: Episode
2017-11-01 23:31:06 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] User-Agent: Medusa/0.0.1 (Linux; 4.9.49-moby; 27a3d95e-bf5c-11e7-bdcb-0242ac110004)
2017-11-01 23:31:06 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] GET URL: **********api?apikey=**********&t=caps [Status: 200]
2017-11-01 23:31:06 INFO     SEARCHQUEUE-MANUAL-311954 :: [JACKETT TPB] :: [b16e0e5] Performing episode search for Riverdale
2017-11-01 23:31:06 INFO     SEARCHQUEUE-MANUAL-311954 :: [USENET-CRAWLER] :: [b16e0e5] Performing season pack search for Riverdale
2017-11-01 23:31:06 DEBUG    SEARCHQUEUE-MANUAL-311954 :: [USENET-CRAWLER] :: [b16e0e5] Fallback season pack search initiate
```
Now manually building the URL myself as suggested by Jackett
```
http://myserver.com:9117/api/v2.0/indexers/thepiratebay/results/torznab/?apikey=xxxxxxxxx&maxage=2700&cat=5030%2C5040&q=Riverdale+S02E02&limit=100&t=search&offset=0
```
Returns valid results. 

So it seems that from the logs, Medusa is requesting a bad URL? The full URL should end in torznab/ but the logs seem to end at ...api?
Also it doesn't seem the newznab category numbers are sticking anywhere. I can choose them, but they aren't showing up anywhere saved in the config.ini (although this is not the cause as in the above case the default works fine)
Hm, trying the v1 API format still shows results from browser but no results returned in Medusa
```
http://myserver.com:9117/torznab/thepiratebay/api?apikey=xx&maxage=2700&cat=5030%2C5040&q=Empire+%282015%29+S04E01&limit=100&t=search&offset=0
```
Logs don't say "no results returned" but no results appear on manual search returned results.

Resolving patch:
diff --git a/medusa/providers/generic_provider.py b/medusa/providers/generic_provider.py
--- a/medusa/providers/generic_provider.py
+++ b/medusa/providers/generic_provider.py
@@ -74,17 +74,11 @@ def __init__(self, name):
 
         self.anime_only = False
         self.bt_cache_urls = [
-            'https://torrentproject.se/torrent/{info_hash}.torrent',
             'http://reflektor.karmorra.info/torrent/{info_hash}.torrent',
-            'http://thetorrent.org/torrent/{info_hash}.torrent',
-            'http://piratepublic.com/download.php?id={info_hash}',
-            'http://www.legittorrents.info/download.php?id={info_hash}',
             'https://torrent.cd/torrents/download/{info_hash}/.torrent',
             'https://asnet.pw/download/{info_hash}/',
-            'https://skytorrents.in/file/{info_hash}/.torrent',
             'http://p2pdl.com/download/{info_hash}',
             'http://itorrents.org/torrent/{info_hash}.torrent',
-            'https://torcache.pro/{info_hash}.torrent',
         ]
         self.cache = tv.Cache(self)
         self.enable_backlog = False
@@ -725,7 +719,8 @@ def _make_url(self, result):
         result_name = sanitize_filename(result.name)
 
         # Some NZB providers (e.g. Jackett) can also download torrents
-        if result.url.endswith(GenericProvider.TORRENT) and self.provider_type == GenericProvider.NZB:
+        if (result.url.endswith(GenericProvider.TORRENT) or
+                result.url.startswith('magnet')) and self.provider_type == GenericProvider.NZB:
             filename = join(app.TORRENT_DIR, result_name + '.torrent')
         else:
             filename = join(self._get_storage_dir(), result_name + '.' + self.provider_type)
diff --git a/medusa/providers/nzb/newznab.py b/medusa/providers/nzb/newznab.py
--- a/medusa/providers/nzb/newznab.py
+++ b/medusa/providers/nzb/newznab.py
@@ -203,15 +203,20 @@ def parse(self, data, mode):
                 try:
                     title = item.title.get_text(strip=True)
                     download_url = None
+
                     if item.link:
-                        if validators.url(item.link.get_text(strip=True)):
-                            download_url = item.link.get_text(strip=True)
-                        elif validators.url(item.link.next.strip()):
-                            download_url = item.link.next.strip()
+                        url = item.link.get_text(strip=True)
+                        if validators.url(url) or url.startswith('magnet'):
+                            download_url = url
+
+                        if not download_url:
+                            url = item.link.next.strip()
+                            if validators.url(url) or url.startswith('magnet'):
+                                download_url = url
 
                     if not download_url and item.enclosure:
                         url = item.enclosure.get('url', '').strip()
-                        if validators.url(url):
+                        if validators.url(url) or url.startswith('magnet'):
                             download_url = url
 
                     if not (title and download_url):
@@ -222,8 +227,7 @@ def parse(self, data, mode):
                         size_regex = re.search(r'\d*.?\d* [KMGT]B', str(item.description))
                         item_size = size_regex.group() if size_regex else -1
                     else:
-                        item_size = item.size.get_text(
-                            strip=True) if item.size else -1
+                        item_size = item.size.get_text(strip=True) if item.size else -1
                         # Use regex to find name-spaced tags
                         # see BeautifulSoup4 bug 1720605
                         # https://bugs.launchpad.net/beautifulsoup/+bug/1720605


Test patch: