Repository: django-helpdesk/django-helpdesk

Issue:
Email BODY html attachements weirdly formatted
Here's an example with sensitive information removed:

```
Please s= ee the confirmation email (about my Dec survey) as below.=C2=A0

---------------------

Hello,

This is a courtesy e-mail to let you know that t= icket [bugs-133] ("Missing survey: NHpokemon8") for=C2=SENSITIVE INFORMATION= =C2=A0has been updated.

Ticket ID: [bugs-133]
Queue: Bugs
Title: Missi= ng survey: SENSITIVE INFORMATION
Opened: Saturday Feb. 2nd 2019, at 6:39 p.m= .
Submitter:=C2=A0SENSITIVE INFORMATION
Priority: 2. High
Statu= s: Reopened
Assigned to:=C2=A0
View Online=C2=A0to upd= ate this ticket (login required)

Just for reference, the original ticket description was:

I submitted my Dec 2= 018 survey 11:54pm right before midnight, and I received email confirmation= immediately. But my data wasn=E2=80=99t in for that month. Now I am prompt= ed for =E2=80=9Cimporting undefined surveys=E2=80=9D. However nothin happen= ed after I click that button again and again.
```
Is there any reason this might be happening? 

Also, a few images are being saved as octlet-streams


Discussion:
For some reason the email's encoding is not being properly converted. Is this an email generated by `django-helpdesk`? Which version are you using? (current 0.2.x release or develop branch?)
This happens to all emails that gets pulled in via `get_email`, even plaintext email. I'm on version 0.2.13 😊
It's something to do with UTF encoding, I suppose it is getting encoded to ascii (maybe base64'd?). I think I might know the line that's doing it. I wonder if anyone else is running into this, and why the unit tests aren't catching it.
I get these errors quite regularly as well.
Hmm we should definitely look into this. Can anyone provide a simple email that recreates the issue?
This is caused by the fact that html email bodies are being encoded with the "Content-Transfer-Encoding" "quoted-printable" and helpdesk is not converting the files back to the "original" encoding of "utf-8". We need to be able to detect the transfer encoding and decode email parts.

https://stackoverflow.com/questions/14249288/change-quoted-printable-encoding-to-utf-8#14249341

Resolving patch:
diff --git a/helpdesk/email.py b/helpdesk/email.py
--- a/helpdesk/email.py
+++ b/helpdesk/email.py
@@ -481,8 +481,9 @@ def object_from_message(message, queue, logger):
                     body.encode('utf-8')
                 logger.debug("Discovered plain text MIME part")
             else:
+                payload = encoding.smart_bytes(part.get_payload(decode=True))
                 files.append(
-                    SimpleUploadedFile(_("email_html_body.html"), encoding.smart_bytes(part.get_payload()), 'text/html')
+                    SimpleUploadedFile(_("email_html_body.html"), payload, 'text/html')
                 )
                 logger.debug("Discovered HTML MIME part")
         else:


Test patch:
diff --git a/helpdesk/tests/test_files/quoted_printable.eml b/helpdesk/tests/test_files/quoted_printable.eml
new file mode 100644
--- /dev/null
+++ b/helpdesk/tests/test_files/quoted_printable.eml
@@ -0,0 +1,22 @@
+MIME-Version: 1.0
+Date: Tue, 5 Mar 2019 18:03:06 +0100
+Message-ID: <CACf_UpPA7=DRF4NLFf4WpOVwHsRMtfNEwy1DHLjfh09DjuKZFA@mail.gmail.com>
+Subject: =?UTF-8?B?xIxlc2vDvSB0ZXN0?=
+From: Tim hobbs <tim.thelion@gmail.com>
+To: helpdesk@auto-mat.cz
+Content-Type: multipart/alternative; boundary="00000000000002a7c505835bd998"
+
+--00000000000002a7c505835bd998
+Content-Type: text/plain; charset="UTF-8"
+Content-Transfer-Encoding: quoted-printable
+
+Tohle je test =C4=8Desk=C3=BDch p=C3=ADsmen odeslan=C3=BDch z gmailu.
+
+--00000000000002a7c505835bd998
+Content-Type: text/html; charset="UTF-8"
+Content-Transfer-Encoding: quoted-printable
+
+<div dir=3D"ltr">Tohle je test =C4=8Desk=C3=BDch p=C3=ADsmen odeslan=C3=BDc=
+h z gmailu.</div>
+
+--00000000000002a7c505835bd998--
\ No newline at end of file
diff --git a/helpdesk/tests/test_get_email.py b/helpdesk/tests/test_get_email.py
--- a/helpdesk/tests/test_get_email.py
+++ b/helpdesk/tests/test_get_email.py
@@ -56,6 +56,23 @@ def test_email_with_blank_body_and_attachment(self):
         self.assertEqual(ticket.title, "Attachment without body")
         self.assertEqual(ticket.description, "")
 
+    def test_email_with_blank_body_and_attachment(self):
+        """
+        Tests that emails with quoted-printable bodies work.
+        """
+        with open(os.path.join(THIS_DIR, "test_files/quoted_printable.eml")) as fd:
+            test_email = fd.read()
+        ticket = helpdesk.email.object_from_message(test_email, self.queue_public, self.logger)
+        self.assertEqual(ticket.title, "Český test")
+        self.assertEqual(ticket.description, "Tohle je test českých písmen odeslaných z gmailu.")
+        followups = FollowUp.objects.filter(ticket=ticket)
+        self.assertEqual(len(followups), 1)
+        followup = followups[0]
+        attachments = Attachment.objects.filter(followup=followup)
+        self.assertEqual(len(attachments), 1)
+        attachment = attachments[0]
+        self.assertEqual(attachment.file.read().decode("utf-8"), '<div dir="ltr">Tohle je test českých písmen odeslaných z gmailu.</div>\n')
+
 
 class GetEmailParametricTemplate(object):
     """TestCase that checks basic email functionality across methods and socks configs."""