Repository: pymedusa/Medusa

Issue:
Existing TV Show folder names with accented characters or apostrophes cause exception
When adding existing shows, if the tv show folder name has accented characters or apostrophes ('), Medusa does not add the show and an exception is shown in the log.

Branch: master 
Commit: 4614efc77151ded92ef458a09dec39f8bd5acfc6 
Version: v0.2.10 
Database: 44.12
--
Python Version: | 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)]
OS: | Windows-10-10.0.18252
Locale: | en_US.cp1252

Fresh installation of Medusa under Windows 10 1809.  I have lots of shows with accented characters in the name (I have a lot of Spanish and Latin American shows), as well as shows with ' in the folder name.

When trying to add "Dirk Gently's Holistic Detective Agency", the following appears in the log and the show is not added:
```
2018-10-16 10:58:19 INFO     SHOWQUEUE-ADD :: [4614efc] Starting to add show by ShowDir: U:\Dirk Gentlyâ€™s Holistic Detective Agency
2018-10-16 10:58:19 INFO     SHOWQUEUE-ADD :: [4614efc] TVDBv2: {u'session': <medusa.session.core.MedusaSession object at 0x00000000086AFF98>, u'cache': u'C:\\pyMedusa\\cache\\indexers\\TVDBv2', u'language': u'en', u'use_zip': True}
2018-10-16 10:58:22 INFO     SHOWQUEUE-ADD :: [4614efc] 312505: Unable to find the show in the database
Traceback (most recent call last):
  File "C:\pyMedusa\medusa\show_queue.py", line 521, in run
    self.show.location = self.showDir
  File "C:\pyMedusa\medusa\tv\base.py", line 57, in __setattr__
    super(TV, self).__setattr__(key, value)
  File "C:\pyMedusa\medusa\tv\series.py", line 398, in location
    raise ShowDirectoryNotFoundException(u'Invalid show folder!')
ShowDirectoryNotFoundException: Invalid show folder!
2018-10-16 10:58:24 ERROR    SHOWQUEUE-ADD :: [4614efc] Error trying to add show: Invalid show folder!
2018-10-16 10:58:25 INFO     SHOWQUEUE-REMOVE-SHOW :: [4614efc] 312505: Removing Dirk Gently's Holistic Detective Agency
```

The same happens when trying to add tv shows with accented characters in the name, such as "Diablo Guardián".

EDIT: Medusa added correctly the show "James Cameron's Story of Science Fiction", so I am not sure what is going on with '  in the folder name.


Discussion:
@medariox im pretty sure this has worked in the past. Maybe issue with locale? 
@p0psicles How would I go about checking/fixing that?
First see if someone can reproduce on win10
I can test in a few days. Maybe ill need a reminder
@p0psicles, I can remind you next week.  In the meantime, I will leave out the shows with accented characters.  Does it matter for folder names what language you search with in the tvdb? I ask since I will remove the nfos for all my Spanish shows and add them again using Spanish as language. 
Oow im not sure. @duramato knows
Is this only happening if you are adding with NFOs or also without?
@medariox So far only with NFOs. Since I am in the middle of adding all of my shows to a fresh install, I have not tried without NFOs yet.  I will be doing that for all my Spanish shows after the others finish (i have over 500 shows), and I can try that as well for the ones that did not get added.
@medariox @p0psicles I deleted the NFOs of my Spanish shows and added them to Medusa, selecting Spanish as the indexer language.  All of my shows were added fine, except that Medusa did not create an NFO for any of the shows.  Is this by design or a bug? Should I open another issue? This creates a problem for me, with my media managers (Kodi, for instance, I don't do online scraping and rely always on NFOs).  Please advise.  Thanks.

This behavior of not creating NFOs happens with all existing shows for which I deleted or renamed my old NFO and re-added them to Medusa, including English ones.
You need to enable it in Gear > Post Processing > Metadata > Show Metadata
@medariox Thanks, I thought I had selected all of the relevant metadata for KODI 12+, I guess I did not save them correctly.

In terms of the initial issue, I found that even though for some shows with accented characters Medusa was creating a folder with garbled characters in the name, I could edit the show and select the correct folder, so in the end I was able to add all of my shows correctly.  From that perspective, this issue can be closed.  I have not tried adding a new show with accented characters, so I don't know if in that scenario Medusa performs correctly.  
Do you still have one of the nfo with accent.
> Does it matter for folder names what language you search with in the tvdb?

The folder name is the show name in the selected language. (If it's avaiable in that language)
@p0psicles, unfortunately I don't have any of the NFOs with accents anymore.  I had Emby (which I also use) recreate the NFOs for the shows for which I deleted them.
@medariox I guess I did save the metadata settings correctly, but Medusa reverts them after a restart.  I saw that this is fixed in the develop branch, but I am not sure if this is enough for me to switch to it.  Do you recommend switching at this time?
If it is not urgent I advise to wait a few days until the next release.
@medariox I will wait, thanks.
I'm experiencing a similar issue adding existing shows. If the folder name contains an accent or an apostrophe, I get an error like:

```
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 37: ordinal not in range(128)
2018-11-04 11:09:35 ERROR    Thread_1 :: [0c0a735] Exception generated: 'ascii' codec can't decode byte 0xe2 in position 37: ordinal not in range(128)
Traceback (most recent call last):
  File "/share/CACHEDEV1_DATA/.qpkg/Medusa/Medusa/medusa/server/web/core/base.py", line 258, in async_call
    result = function(**kwargs)
  File "/share/CACHEDEV1_DATA/.qpkg/Medusa/Medusa/medusa/server/web/home/add_shows.py", line 543, in addExistingShows
    if '|' not in cur_dir:
```

I've been able to manually add the shows though. The resulting folders have unicode character codes in them though:

For instance, `Tom Clancy's Jack Ryan` shows as `Tom\ Clancy$'\342'$'\200'$'\231's\ Jack\ Ryan` and `Le Bureau des Légendes` shows as `Le\ Bureau\ des\ L$'\303'$'\251'gendes/`.

I'm using QNAP.

In the UI, it looks fine though (when browsing for the folder).

Resolving patch:
diff --git a/medusa/metadata/kodi_12plus.py b/medusa/metadata/kodi_12plus.py
--- a/medusa/metadata/kodi_12plus.py
+++ b/medusa/metadata/kodi_12plus.py
@@ -5,7 +5,7 @@
 import datetime
 import logging
 import re
-from builtins import str
+
 
 from babelfish import Country
 
@@ -19,7 +19,7 @@
 from medusa.logger.adapters.style import BraceAdapter
 from medusa.metadata import generic
 
-from six import string_types
+from six import string_types, text_type
 
 try:
     import xml.etree.cElementTree as etree
@@ -117,11 +117,11 @@ def _show_data(self, series_obj):
 
         if getattr(my_show, 'rating', None):
             rating = etree.SubElement(tv_node, 'rating')
-            rating.text = str(my_show['rating'])
+            rating.text = text_type(my_show['rating'])
 
         if getattr(my_show, 'firstaired', None):
             try:
-                year_text = str(datetime.datetime.strptime(my_show['firstaired'], dateFormat).year)
+                year_text = text_type(datetime.datetime.strptime(my_show['firstaired'], dateFormat).year)
                 if year_text:
                     year = etree.SubElement(tv_node, 'year')
                     year.text = year_text
@@ -148,7 +148,7 @@ def _show_data(self, series_obj):
 
         if getattr(my_show, 'id', None):
             indexer_id = etree.SubElement(tv_node, 'id')
-            indexer_id.text = str(my_show['id'])
+            indexer_id.text = text_type(my_show['id'])
 
         if getattr(my_show, 'genre', None) and isinstance(my_show['genre'], string_types):
             for genre in self._split_info(my_show['genre']):
@@ -242,7 +242,7 @@ def _ep_data(self, ep_obj):
                 return None
 
             if not getattr(my_ep, 'firstaired', None):
-                my_ep['firstaired'] = str(datetime.date.fromordinal(1))
+                my_ep['firstaired'] = text_type(datetime.date.fromordinal(1))
 
             if not getattr(my_ep, 'episodename', None):
                 log.debug(u'Not generating nfo because the ep has no title')
@@ -265,17 +265,17 @@ def _ep_data(self, ep_obj):
                 showtitle.text = series_obj['seriesname']
 
             season = etree.SubElement(episode, 'season')
-            season.text = str(ep_to_write.season)
+            season.text = text_type(ep_to_write.season)
 
             episodenum = etree.SubElement(episode, 'episode')
-            episodenum.text = str(ep_to_write.episode)
+            episodenum.text = text_type(ep_to_write.episode)
 
             uniqueid = etree.SubElement(episode, 'uniqueid')
-            uniqueid.text = str(ep_to_write.indexerid)
+            uniqueid.text = text_type(ep_to_write.indexerid)
 
             if ep_to_write.airdate != datetime.date.fromordinal(1):
                 aired = etree.SubElement(episode, 'aired')
-                aired.text = str(ep_to_write.airdate)
+                aired.text = text_type(ep_to_write.airdate)
 
             if getattr(my_ep, 'overview', None):
                 plot = etree.SubElement(episode, 'plot')
@@ -283,7 +283,7 @@ def _ep_data(self, ep_obj):
 
             if ep_to_write.season and getattr(series_obj, 'runtime', None):
                 runtime = etree.SubElement(episode, 'runtime')
-                runtime.text = str(series_obj['runtime'])
+                runtime.text = text_type(series_obj['runtime'])
 
             if getattr(my_ep, 'airsbefore_season', None):
                 displayseason = etree.SubElement(episode, 'displayseason')
@@ -302,7 +302,7 @@ def _ep_data(self, ep_obj):
 
             if getattr(my_ep, 'rating', None):
                 rating = etree.SubElement(episode, 'rating')
-                rating.text = str(my_ep['rating'])
+                rating.text = text_type(my_ep['rating'])
 
             if getattr(my_ep, 'writer', None) and isinstance(my_ep['writer'], string_types):
                 for writer in self._split_info(my_ep['writer']):
diff --git a/medusa/server/web/home/add_shows.py b/medusa/server/web/home/add_shows.py
--- a/medusa/server/web/home/add_shows.py
+++ b/medusa/server/web/home/add_shows.py
@@ -24,7 +24,6 @@
 from medusa.show.show import Show
 
 from requests import RequestException
-from requests.compat import unquote_plus
 
 from simpleanidb import REQUEST_HOT
 
@@ -532,7 +531,7 @@ def addExistingShows(self, shows_to_add=None, promptForSettings=None):
         elif not isinstance(shows_to_add, list):
             shows_to_add = [shows_to_add]
 
-        shows_to_add = [unquote_plus(x) for x in shows_to_add]
+        shows_to_add = [text_type(x, 'utf-8') if not isinstance(x, text_type) else x for x in shows_to_add]
 
         prompt_for_settings = config.checkbox_to_value(prompt_for_settings)
 


Test patch: