Repository: scrapy/scrapy (Python)

Issue:
Wrong charset handling in MailSender
Looks like passing `charset='utf-8'` makes a plain text message with `Content-Transfer-Encoding: base64` which then can't be read.

At https://github.com/scrapy/scrapy/blob/5a75b14a5fbbbd37c14aa7317761655ac7706b70/scrapy/mail.py#L81 `set_charset` is called but as the payload is not set yet, the underlying class just sets the headers. When later `set_payload` is called, it doesn't do any encoding, but the `Content-Transfer-Encoding` is already set. Looks like the fix should be passing the encoding to `set_payload` too, like was proposed in #3722 (and `set_charset` may be safe to be removed, not sure about this). Note that we have tests but they don't catch this.

Note also, that all of this seems to be compat code according to the Python docs.


Discussion:
Hello, i am new here and i would like to try and fix this bug. Can you please give me some guidlines and tell me how to recreate this issue?
Sure, you can go ahead.
> Can you please give me some guidlines and tell me how to recreate this issue?

The documentation about using the `MailSender` class is at https://docs.scrapy.org/en/latest/topics/email.html . Then it should be a matter of following the instructions on the issue description (“passing charset='utf-8'”), and verify that indeed the resulting email message cannot be read.
Hello, I have made a small change and now when you send an email with utf-8 encoding the body of the email looks fine and does not have a base64 encoding. Do you want me to do a pull request or maybe a draft pull request to see what I have done and give me feedback?
> Do you want me to do a pull request or maybe a draft pull request to see what I have done and give me feedback?

That would be great! Feel free to make it a regular pull request if it’s ready for review and discussion.
I created a pull request and I am looking forward for your feedback. Thank you!! @Gallaecio @wRAR 



Patch:
diff --git a/scrapy/mail.py b/scrapy/mail.py
index c11f3898d0d..237327451ee 100644
--- a/scrapy/mail.py
+++ b/scrapy/mail.py
@@ -96,10 +96,9 @@ def send(
             rcpts.extend(cc)
             msg["Cc"] = COMMASPACE.join(cc)
 
-        if charset:
-            msg.set_charset(charset)
-
         if attachs:
+            if charset:
+                msg.set_charset(charset)
             msg.attach(MIMEText(body, "plain", charset or "us-ascii"))
             for attach_name, mimetype, f in attachs:
                 part = MIMEBase(*mimetype.split("/"))
@@ -110,7 +109,7 @@ def send(
                 )
                 msg.attach(part)
         else:
-            msg.set_payload(body)
+            msg.set_payload(body, charset)
 
         if _callback:
             _callback(to=to, subject=subject, body=body, cc=cc, attach=attachs, msg=msg)


Test patch:
diff --git a/tests/test_mail.py b/tests/test_mail.py
index 504c7848647..2535e58db26 100644
--- a/tests/test_mail.py
+++ b/tests/test_mail.py
@@ -111,7 +111,7 @@ def test_send_utf8(self):
 
         msg = self.catched_msg["msg"]
         self.assertEqual(msg["subject"], subject)
-        self.assertEqual(msg.get_payload(), body)
+        self.assertEqual(msg.get_payload(decode=True).decode("utf-8"), body)
         self.assertEqual(msg.get_charset(), Charset("utf-8"))
         self.assertEqual(msg.get("Content-Type"), 'text/plain; charset="utf-8"')