Repository: pymedusa/Medusa

Issue:
Medusa not starting using Python 3.9 due to old feedparser
I installed Python 3.9 and I get errors on startup

Traceback (most recent call last):
  File "C:\Medusa\start.py", line 6, in <module>
    from medusa.__main__ import main
  File "C:\Medusa\medusa\__main__.py", line 67, in <module>
    from medusa import (
  File "C:\Medusa\medusa\naming.py", line 27, in <module>
    from medusa.tv import Episode, Series
  File "C:\Medusa\medusa\tv\__init__.py", line 6, in <module>
    from medusa.tv.cache import Cache
  File "C:\Medusa\medusa\tv\cache.py", line 27, in <module>
    from medusa.rss_feeds import getFeed
  File "C:\Medusa\medusa\rss_feeds.py", line 7, in <module>
    from feedparser.api import parse
  File "C:\Medusa\ext\feedparser\__init__.py", line 39, in <module>
    from . import api
  File "C:\Medusa\ext\feedparser\api.py", line 54, in <module>
    from . import http
  File "C:\Medusa\ext\feedparser\http.py", line 44, in <module>
    _base64decode = getattr(base64, 'decodebytes', base64.decodestring)
AttributeError: module 'base64' has no attribute 'decodestring'

I replaced the feedparser directory in Medusa with the latest version obtained from pip install feedparser and it starts without a problem.



Discussion:
Tnx for the help. We'll update it
Sorry, one update. My process above worked on a new bare instance. On the one I use, I also had to edit \medusa\helpers\__init__.py and replace encodestring and decodestring with encodebytes and decodebytes
Maybe we'll make it py3.9 compatible after dropping py2.7 support
Cool thanks. You guys do an awesome job. I really appreciate all the work!
how to temporarily fix this manually on Linux? 
@medariox what do you think? Is this a lot of work? 
What's the problem with Python 3.8?
No problem with 3.8. I just wanted to see if I could make it work with 3.9.
Just think w should motivate users to use latest version. New major versions excluded of course. Just was interested in your opinion on this
IMO it's not worth the time considering we should have already dropped Python 2.7 compatibility. Once we update feedparser it will be fixed.
Fedora 33 was released on Oct 27, 2020 and has python 3.9 so Medusa isn't working.  I just downloaded the feedparser-0.6.2 source and replaced the Medusa/ext/feedparser with feedparser-0.6.3/feedparser directory.  Medusa is running but I'm not sure how to exercise feedparser.
Indeed after upgrade to Fedora 33, Medusa stops working. I replaced feedparser with the version from git but it's not enough.
Now I get:

```
systemd[1]: Started Medusa Daemon.
python[22148]: detected unhandled Python exception in '/opt/medusa/start.py'
python[22148]: Traceback (most recent call last):
python[22148]:   File "/opt/medusa/medusa/config.py", line 729, in check_setting_str
python[22148]:     my_val = helpers.decrypt(config[cfg_name][item_name], encryption_version)
python[22148]:   File "/opt/medusa/medusa/helpers/__init__.py", line 965, in decrypt
python[22148]:     return encrypt(data, encryption_version, _decrypt=True)
python[22148]:   File "/opt/medusa/medusa/helpers/__init__.py", line 957, in encrypt
python[22148]:     data = ensure_text(base64.decodestring(ensure_binary(data)))
python[22148]: AttributeError: module 'base64' has no attribute 'decodestring'
python[22148]: During handling of the above exception, another exception occurred:
python[22148]: Traceback (most recent call last):
python[22148]:   File "/opt/medusa/medusa/config.py", line 735, in check_setting_str
python[22148]:     config[cfg_name][item_name] = helpers.encrypt(my_val, encryption_version)
python[22148]:   File "/opt/medusa/medusa/helpers/__init__.py", line 961, in encrypt
python[22148]:     return ensure_text(base64.encodestring(ensure_binary(data))).strip()
python[22148]: AttributeError: module 'base64' has no attribute 'encodestring'
python[22148]: During handling of the above exception, another exception occurred:
python[22148]: Traceback (most recent call last):
python[22148]:   File "/opt/medusa/start.py", line 9, in <module>
python[22148]:     main()
python[22148]:   File "/opt/medusa/medusa/__main__.py", line 2212, in main
python[22148]:     application.start(sys.argv[1:])
python[22148]:   File "/opt/medusa/medusa/__main__.py", line 328, in start
python[22148]:     self.initialize(console_logging=self.console_logging)
python[22148]:   File "/opt/medusa/medusa/__main__.py", line 439, in initialize
python[22148]:     app.GIT_PASSWORD = check_setting_str(app.CFG, 'General', 'git_password', '', censor_log='low')
python[22148]:   File "/opt/medusa/medusa/config.py", line 738, in check_setting_str
python[22148]:     config[cfg_name][item_name] = helpers.encrypt(my_val, encryption_version)
python[22148]:   File "/opt/medusa/medusa/helpers/__init__.py", line 961, in encrypt
python[22148]:     return ensure_text(base64.encodestring(ensure_binary(data))).strip()
python[22148]: AttributeError: module 'base64' has no attribute 'encodestring'
systemd[1]: medusa.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: medusa.service: Failed with result 'exit-code'.
systemd[1]: medusa.service: Consumed 1.754s CPU time.
```

Then you can always temp revert back to 2.7. Well always make sure medusa will work with 2.7 or 3.9. then you can switch when we move to py 3 only
> Indeed after upgrade to Fedora 33, Medusa stops working. I replaced feedparser with the version from git but it's not enough.

Medusa is running fine for me after updating feedparser.  I noticed that it seems to be failing for you in trying to decode git_password from the config file.  I don't have git authentication enabled so that is likely why its not failing for me.  I don't see any need why you can't download Medusa updates anonymously.


The problem is just Python 3.9 - Don't use it until we say it's officially supported.
> 
> 
> > Indeed after upgrade to Fedora 33, Medusa stops working. I replaced feedparser with the version from git but it's not enough.
> 
> Medusa is running fine for me after updating feedparser. I noticed that it seems to be failing for you in trying to decode git_password from the config file. I don't have git authentication enabled so that is likely why its not failing for me. I don't see any need why you can't download Medusa updates anonymously.

I don't actually have git_password set. It's at the default of `git_password = ''"`.  Also `git_auth_type = 0`.

It's the deprecation of `base64.encodestring()` and `base64.decodestring()`.

So basically all this needs to be fixed is replace `base64.encodestring()` and `base64.decodestring()` calls with `base64.encodebytes()` and `base64.decodebytes()` respectively in `medusa/medusa/helpers/__init__.py` and `medusa\ext\feedparser\http.py` (or replacement of the feedparser module with the latest version which is not affected).
I can't downgrade python on termux. Made the proposed replacement above but still have an error

> python /data/data/com.termux/files/home/medusa/start.py -q --daemon --nolaunch #--pidfile=/usr/var/run/medusa.pid
> 
> 
> Traceback (most recent call last):
>   File "/data/data/com.termux/files/home/medusa/start.py", line 6, in <module>
>     from medusa.__main__ import main
>   File "/data/data/com.termux/files/home/medusa/medusa/__main__.py", line 67, in <module>
>     from medusa import (
>   File "/data/data/com.termux/files/home/medusa/medusa/helpers/__init__.py", line 964
>     def decrypt(data, encryption_version=0):
>     ^
> SyntaxError: invalid syntax
Can you run docker? 
I believe not possible on termux android S10+ 
Docker is technically possible on android. You run a Fedora image inside termux and run docker from there (or more likely podman). The overhead though would be rather unforgiving for an android device.

That traceback though leads me to believe that you just made a typo when you replaced the function call as line 964 is directly below the call you replaced. Make sure that part of the file looks like this or copy paste:
```
def encrypt(data, encryption_version=0, _decrypt=False):
    # Version 0: Plain text
    if encryption_version == 0:
        return data
    else:
        # Simple XOR encryption, Base64 encoded
        # Version 1: unique_key1; Version 2: app.ENCRYPTION_SECRET
        key = unique_key1 if encryption_version == 1 else app.ENCRYPTION_SECRET
        if _decrypt:
            data = ensure_text(base64.decodebytes(ensure_binary(data)))
            return ''.join(chr(ord(x) ^ ord(y)) for (x, y) in zip(data, cycle(key)))
        else:
            data = ''.join(chr(ord(x) ^ ord(y)) for (x, y) in zip(data, cycle(key)))
            return ensure_text(base64.encodebytes(ensure_binary(data))).strip()


def decrypt(data, encryption_version=0):
    return encrypt(data, encryption_version, _decrypt=True)
```
Also do notice that these manual changes will break medusa updating. So you will have to handle that manually until this is addressed upstream. This means you'd have to manually do:
```
git stash save --keep-index
git stash drop
```
Then update medusa and apply the changes again if current head doesn't include them.
If someone can do a PR that works with py2 and py3.9 its also resolved 
I could but 2.7 doesn't have `base64.encodebytes()` and `base64.decodebytes()` so that has to be handled with a `sys.version_info.major` check for major version and use the appropriate call. Which begs the question how that should be done for the feedparser module? I'm not familiar with the project's internals and how you handle the dependencies. The latest feedparser that works requires `Python >=3.6` according to pypi so can I just patch the bundled one the same way?
You need differint libs for /ext2 and /ext3, that's we we wanted to wait for dropping py2
Yeah figured it wouldn't be that simple although the hackish solution would work, heh. I need to find time to examine the code and see exactly how that's handled. So no promises although I'm inclined to contribute as it will save some hassle on updates untill you drop py2. :)
So here's an even better workaround for now for Fedora 33 that doesn't require downgrading system python, or editing medusa files/breaking updates and comes with a bonus of increasing medusa performance by a LOT!

Install pypy3 with `dnf install pypy3` and use `/usr/bin/pypy3` instead of `/usr/bin/python` in your systemd unit.

You will also need a custom selinux policy module so if you don't know how to write one yourself, here you go:

```
module my-pypy3 1.0;

require {
        type default_t;
        type init_t;
        type xserver_port_t;
        type usr_t;
        type lib_t;
        class process execmem;
        class file { append create rename unlink write };
        class dir { add_name remove_name rmdir };
        class tcp_socket name_connect;
}

#============= init_t ==============
allow init_t default_t:dir rmdir;
allow init_t lib_t:dir { add_name remove_name };
allow init_t lib_t:file { create rename write };
allow init_t self:process execmem;
allow init_t usr_t:dir rmdir;
allow init_t usr_t:file { append create rename unlink write };
allow init_t xserver_port_t:tcp_socket name_connect;
```

Save the above to `my-pypy3.te`
Then run:
`checkmodule -M -m -o my-pypy3.mod my-pypy3.te`
`semodule_package -o my-pypy3.pp -m my-pypy3.mod`
`semodule -i my-pypy3.pp`

And you're good to go.

EDIT: Edited selinux policy to allow also snatching via torrent client and removing a show from library with file deletion option checked
Can you try latest commit develop branch? 
Thanks. You still need to change encodestring and decodestring. 

C:\Medusa\medusa\helpers\__init__.py

        if _decrypt:
            data = ensure_text(base64.decodebytes(ensure_binary(data)))
            return ''.join(chr(ord(x) ^ ord(y)) for (x, y) in zip(data, cycle(key)))
        else:
            data = ''.join(chr(ord(x) ^ ord(y)) for (x, y) in zip(data, cycle(key)))
            return ensure_text(base64.encodebytes(ensure_binary(data))).strip()

Making this change it starts up for me without a problem
Can you create  a pull request? We'll make sure it gets merged ASAP. 
> Can you create a pull request? We'll make sure it gets merged ASAP.

I'm sorry but that's not something I know how to do. I'm a sysadmin but I have 0 development skills. I can google with the best of them, that's how I figured out the right change to make :) But I don't want to mess around here. It's literally just changing

decodestring to decodebytes
encodestring to encodebytes

In that one file.

Resolving patch:
diff --git a/lib/rtorrent/lib/xmlrpc/basic_auth.py b/lib/rtorrent/lib/xmlrpc/basic_auth.py
--- a/lib/rtorrent/lib/xmlrpc/basic_auth.py
+++ b/lib/rtorrent/lib/xmlrpc/basic_auth.py
@@ -20,7 +20,7 @@
 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
-from base64 import encodestring
+from base64 import encodebytes
 import string
 
 try:
@@ -39,7 +39,7 @@ def __init__(self, username=None, password=None):
     def send_auth(self, h):
         if self.username is not None and self.password is not None:
             h.putheader('AUTHORIZATION', "Basic %s" % string.replace(
-                encodestring("%s:%s" % (self.username, self.password)),
+                encodebytes("%s:%s" % (self.username, self.password)),
                 "\012", ""
             ))
 
diff --git a/medusa/helpers/__init__.py b/medusa/helpers/__init__.py
--- a/medusa/helpers/__init__.py
+++ b/medusa/helpers/__init__.py
@@ -923,11 +923,11 @@ def encrypt(data, encryption_version=0, _decrypt=False):
         # Version 1: unique_key1; Version 2: app.ENCRYPTION_SECRET
         key = unique_key1 if encryption_version == 1 else app.ENCRYPTION_SECRET
         if _decrypt:
-            data = ensure_text(base64.decodestring(ensure_binary(data)))
+            data = ensure_text(base64.decodebytes(ensure_binary(data)))
             return ''.join(chr(ord(x) ^ ord(y)) for (x, y) in zip(data, cycle(key)))
         else:
             data = ''.join(chr(ord(x) ^ ord(y)) for (x, y) in zip(data, cycle(key)))
-            return ensure_text(base64.encodestring(ensure_binary(data))).strip()
+            return ensure_text(base64.encodebytes(ensure_binary(data))).strip()
 
 
 def decrypt(data, encryption_version=0):


Test patch: