Repository: pymedusa/Medusa

Issue:
Problem Searching for Norwegian Shows
### Before submitting your issue:

Enable debug logging in Medusa settings, reproduce the error (be sure to disable after the bug is fixed)

**Branch/Commit:master/d6eb72d4e1019e5d1286ae6ac1d7e4b318229ebe
**OS:CentOS 7
**What you did: Searching for a Show (manual and auto) on Provider Norbits (Norwegian Tracker) for a Norwegian show
**What happened: No results
**What you expected: There are available files so expected results
**Logs:**
```
2018-03-10 09:35:54 INFO     SEARCHQUEUE-MANUAL-342737 :: [d6eb72d] Unable to find manual results for: Stories From Norway - S01E01 - 001 - Stupetårnet i Hamar
2018-03-10 09:35:53 INFO     SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Performing season pack search for Stories From Norway
2018-03-10 09:35:53 INFO     SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Performing episode search for Stories From Norway

```

I was wondering if there is some settings that ignores Norwegian?
I can't find any exceptions.
I know this worked in Sickrage with no problems. (coming from Sickrage)

This goes for all Norwegian shows I have


Discussion:
Need debug logs of a search. 
Sorry thought I added. but forgot to change the filter in log window :)
Here is the debug log

```
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] No unknown qualities in results
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Found qualities: []
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Found 0 unique items
2018-03-10 12:38:37 INFO     SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Performing season pack search for Stories From Norway
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Fallback season pack search initiate
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] No unknown qualities in results
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Found qualities: []
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] Found 0 unique items
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] With post data: username=**********&category=2&search=Stories+From+Norway+S01E01&passkey=**********
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] User-Agent: Medusa/0.1.23 (Linux; 3.10.0-693.17.1.el7.x86_64; 9c7a96be-2435-11e8-abc1-618de2a07cc8)
2018-03-10 12:38:37 DEBUG    SEARCHQUEUE-MANUAL-342737 :: [Norbits] :: [d6eb72d] POST URL: https://norbits.net/api2.php?action=torrents [Status: 200]
```

![image](https://user-images.githubusercontent.com/10724791/37241863-6ade5c8c-2460-11e8-9101-e920b7af4120.png)

How do you search for this show on Norbits? Which category is it? The API doesn't return any results, so there is probably an issue with the category.
hmm category is called TV on Norbits, if thats what you mean?
After you search on the site, it should be written in the address bar (sorry for the late response).
If you post a json object to the API with same search-parameters as in the debug info from issue-creator, like so:

`{
"username":"user",
"category":"2",
"search":"Stories From Norway S01E01",
"passkey":"validPassKey"
}`

You do get a json string back containing a result. I have the same issue no matter what show it is, where the result is: 

> [Norbits] :: [d6eb72d] Found 0 unique items

Could there be some issue with posting the correct json object to the API, for example that is does some weird things in the search string or something?
Can you try to replace this line: https://github.com/pymedusa/Medusa/blob/master/medusa/providers/torrent/json/norbits.py#L81
With this one:
`response = self.session.post(self.urls['search'], data=json.dumps(post_data))`

Let me know if it works, please.
Yes, I can, this is the debug result:

`2018-04-12 21:10:16 DEBUG    SEARCHQUEUE-MANUAL-121361 :: [Norbits] :: [d6eb72d] Traceback (most recent call last):
  File "/opt/medusa/medusa/search/queue.py", line 351, in run
    self.manual_search, self.manual_search_type)
  File "/opt/medusa/medusa/search/core.py", line 609, in search_providers
    down_cur_quality, manual_search, manual_search_type)
  File "/opt/medusa/medusa/providers/generic_provider.py", line 235, in find_search_results
    search_string, ep_obj=episode, manual_search=manual_search
  File "/opt/medusa/medusa/providers/torrent/json/norbits.py", line 81, in search
    response = self.session.post(self.urls['search'], data=json.dumps(post_data))
NameError: global name 'json' is not defined`
Oh sorry, you also need to add: `import json` at the top of the file.
It does this now

[Norbits] :: [d6eb72d] No unknown qualities in results
[Norbits] :: [d6eb72d] Found qualities: []
[Norbits] :: [d6eb72d] Found 0 unique items
[Norbits] :: [d6eb72d] With post data: {"username": "\*\*\*\*\*\*\*\*\*\*", "category": "2", "search": "Stories from Norway S01E01", "passkey": "**********"}'

So it does seem to post it correctly now. But still with the 0 results. Could there be some error in how it reads the returned json object as well?
That's really strange. Can you compare a result where items are found with this one that returns 0 results? That'd be of great help.
@medariox you don't need to import json, instead do `self.session.post(self.urls['search'], json=post_data)` and let the Requests session do the json conversion for you.
@labrys Yes, just trying to find out the issue for now.
@medariox the problem is in `NorbitsProvider.search`.  It returns results before parsing. https://github.com/pymedusa/Medusa/blob/develop/medusa/providers/torrent/json/norbits.py#L93

https://github.com/pymedusa/Medusa/blob/develop/medusa/providers/torrent/json/norbits.py#L92 should be `if not self._check_auth_from_data(jdata):`
Additionally https://github.com/pymedusa/Medusa/blob/develop/medusa/providers/torrent/json/norbits.py#L95 should be indented one more block.
@massifman 
If you change the following code starting at Line 92 making the `if` statement `if not` and indenting the `results += ` it should work.
```python
                if self._check_auth_from_data(jdata):
                    return results

            results += self.parse(jdata, mode)
```

After edits it should look like the following:
```python
                if not self._check_auth_from_data(jdata):
                    return results

                results += self.parse(jdata, mode)
```
@labrys I implemented what you said so far, but it still gives the found 0 unique items, I also made it print the response.content to the debug log to verify that it get's a proper response, and it seems like the kind of json object that the parse function is built for. I was also able to verify that it does enter the parse function by setting a little log.debug entry in the top of it, so there might be some indent problem in the parse function as well(?). I don't have more time for today unfortunately, but if you find anything, I'll test it when I get the time and get back to you.
If you get a chance to post the response.content then I'll take a look at the parser.
@labrys @medariox Ok, I couldn't stop before I looked at it a bit more, and I figured it out. Line 109 is to remove the first part of the json object I presume, https://github.com/pymedusa/Medusa/blob/develop/medusa/providers/torrent/json/norbits.py#L109
but the returned object doesn't get a name, so the torrent_rows doesn't get populated on line 110, because it is still operating on the nested object, if you change line 109 and 110 to 
```
json_data = data.get('data', '')
torrent_rows = json_data.get('torrents', [])
```
in addition to the changes allready mentioned, it works. I just tested it.
@massifman I figured that was the case but couldn't say for sure without the response.  Thanks for looking into it.  

A couple other changes that should be made L109 should instead be `json_data = data.get('data', {})` to make sure L110 doesn't crash if there's no data.  And `_check_auth_from_data` is missing `return False` at L169 for when authentication fails.

Resolving patch:
diff --git a/medusa/providers/torrent/json/norbits.py b/medusa/providers/torrent/json/norbits.py
--- a/medusa/providers/torrent/json/norbits.py
+++ b/medusa/providers/torrent/json/norbits.py
@@ -5,7 +5,6 @@
 from __future__ import unicode_literals
 
 import logging
-import traceback
 
 from medusa import tv
 from medusa.helper.common import (
@@ -49,7 +48,7 @@ def __init__(self):
         self.minleech = None
 
         # Cache
-        self.cache = tv.Cache(self, min_time=20)  # only poll Norbits every 15 minutes max
+        self.cache = tv.Cache(self, min_time=20)
 
     def search(self, search_strings, age=0, ep_obj=None, **kwargs):
         """
@@ -78,7 +77,7 @@ def search(self, search_strings, age=0, ep_obj=None, **kwargs):
                     'search': search_string,
                 }
 
-                response = self.session.post(self.urls['search'], data=post_data)
+                response = self.session.post(self.urls['search'], json=post_data)
                 if not response or not response.content:
                     log.debug('No data returned from provider')
                     continue
@@ -89,10 +88,10 @@ def search(self, search_strings, age=0, ep_obj=None, **kwargs):
                     log.debug('No data returned from provider')
                     continue
 
-                if self._check_auth_from_data(jdata):
+                if not self._check_auth_from_data(jdata):
                     return results
 
-            results += self.parse(jdata, mode)
+                results += self.parse(jdata, mode)
 
         return results
 
@@ -106,10 +105,9 @@ def parse(self, data, mode):
         :return: A list of items found
         """
         items = []
-        data.get('data', '')
-        torrent_rows = data.get('torrents', [])
+        json_data = data.get('data', {})
+        torrent_rows = json_data.get('torrents', [])
 
-        # Skip column headers
         for row in torrent_rows:
             try:
                 title = row.pop('name', '')
@@ -147,16 +145,14 @@ def parse(self, data, mode):
 
                 items.append(item)
             except (AttributeError, TypeError, KeyError, ValueError, IndexError):
-                log.error('Failed parsing provider. Traceback: {0!r}',
-                          traceback.format_exc())
+                log.exception('Failed parsing provider.')
 
         return items
 
     def _check_auth(self):
-
         if not self.username or not self.passkey:
-            raise AuthException(('Your authentication credentials for %s are '
-                                 'missing, check your config.') % self.name)
+            raise AuthException('Your authentication credentials for {0} are missing,'
+                                ' check your config.'.format(self.name))
 
         return True
 
@@ -166,6 +162,7 @@ def _check_auth_from_data(self, parsed_json):
             if parsed_json.get('status') == 3:
                 log.warning('Invalid username or password.'
                             ' Check your settings')
+                return False
 
         return True
 


Test patch: