Repository: twisted/twisted

Issue:
IMAP Server support is broken for iOS Mail.app and shorter messages
**Describe the incorrect behavior you saw**

Using the IMAPServer class with the latest (at least, I've not tested on older versions) Mail.app on iOS appears to be broken. The client can load the folder structure, but will never load the body of the message if the message is less than 393216 bytes.

This appears to be due to iOS using `BODY.PEEK` with a very large partial window. The IMAP RFC (RFC3501) states:

>  Any partial fetch that attempts to read beyond the end of the
         text is truncated as appropriate.  A partial fetch that starts
         at octet 0 is returned as a partial fetch, even if this
         truncation happened.

> Note: This means that BODY[]<0.2048> of a 1500-octet message
    will return BODY[]<0> with a literal of size 1500, not
    BODY[].

Twisted [does not respect this](https://github.com/twisted/twisted/blob/trunk/src/twisted/mail/imap4.py#L5714) and returns the original partial window and the content of the message. This is then never displayed in mail.app and causes the app to go into a message retrieval loop.

iOS appears to always use the same partial range, with UID enabled:
```UID BODY.PEEK[]<0.393216>```
This taken from both debug output in twisted and a wireshark packet trace.

**Describe how to cause this behavior**

Create a test IMAP server using the IMAP4 support in Twisted and populate it with some messages. Attempt to connect to it using the iOS native mail.app.

This https://github.com/mistio/localmail/tree/master is an example of a server that exhibits the problem.

**Describe the correct behavior you'd like to see**

The mail should be viewable in mail.app, and the server should follow the RFC spec (as much as possible with the IMAP spec)

**Testing environment**
 - Operating System and Version; paste the output of these commands:
```
Linux SanctionedPartsOnly 6.11.0-8-generic #8-Ubuntu SMP PREEMPT_DYNAMIC Mon Sep 16 13:41:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.10
DISTRIB_CODENAME=oracular
DISTRIB_DESCRIPTION="Ubuntu 24.10"
```
 - Twisted version [e.g. 22.2.0]
```
attrs==24.2.0
Automat==24.8.1
constantly==23.10.4
hyperlink==21.0.0
idna==3.10
incremental==24.7.2
setuptools==75.2.0
Twisted==24.7.0
typing_extensions==4.12.2
zope.interface==7.1.1
```
 - Reactor [e.g. select, iocp]
 `select`


**Additional context**
Add any other context about the problem here.


Discussion:

Patch:
diff --git a/src/twisted/mail/imap4.py b/src/twisted/mail/imap4.py
index 9a9f140795..f38d094f89 100644
--- a/src/twisted/mail/imap4.py
+++ b/src/twisted/mail/imap4.py
@@ -28,7 +28,7 @@ import uuid
 from base64 import decodebytes, encodebytes
 from io import BytesIO
 from itertools import chain
-from typing import Any, List, cast
+from typing import Any, List, Optional, cast
 
 from zope.interface import implementer
 
@@ -2205,7 +2205,7 @@ class IMAP4Server(basic.LineReceiver, policies.TimeoutMixin):
             hdrs = _formatHeaders(msg.getHeaders(True))
             _w(part.__bytes__() + b" " + _literal(hdrs))
         elif part.empty:
-            _w(part.__bytes__() + b" ")
+            _w(part.getBytes(length=msg.getSize()) + b" ")
             _f()
             if part.part:
                 return FileProducer(msg.getBodyFile()).beginProducing(self.transport)
@@ -5691,7 +5691,14 @@ class _FetchParser:
         def __str__(self) -> str:
             return self.__bytes__().decode("ascii")
 
-        def __bytes__(self) -> bytes:
+        def getBytes(self, length: Optional[int] = None) -> bytes:
+            """
+            Prepare the initial command response for a Fetch BODY request.
+            Interpret the Fetch request from the client and return the
+            appropriate response based on RFC 3501.
+            This is not the body itself of the response, merely the section
+            of the first response line that describes the body part.
+            """
             base = b"BODY"
             part = b""
             separator = b""
@@ -5711,9 +5718,19 @@ class _FetchParser:
             elif self.empty:
                 base += b"[" + part + b"]"
             if self.partialBegin is not None:
-                base += b"<%d.%d>" % (self.partialBegin, self.partialLength)  # type: ignore[unreachable]
+                if length is None or length > self.partialLength:  # type: ignore[unreachable]
+                    base += b"<%d.%d>" % (self.partialBegin, self.partialLength)
+                else:
+                    # IMAP4rev1 says that if the partial length is greater than
+                    # the length of the data, the server should send the entire
+                    # data., with a "0" as the partial length
+                    # https://datatracker.ietf.org/doc/html/rfc3501#section-6.4.5
+                    base += b"<0>"
             return base
 
+        def __bytes__(self) -> bytes:
+            return self.getBytes()
+
     class BodyStructure:
         type = "bodystructure"
         __str__ = lambda self: "bodystructure"
diff --git a/src/twisted/mail/newsfragments/12345.bugfix b/src/twisted/mail/newsfragments/12345.bugfix
new file mode 100644
index 0000000000..bc416f8291
--- /dev/null
+++ b/src/twisted/mail/newsfragments/12345.bugfix
@@ -0,0 +1,1 @@
+twisted.mail.imap.IMAPServer.arg_fetchatt can now detect that a IMAP partial request has been made that is longer than the original message and alter the output of the response in order to match the IMAPRev4 spec.


Test patch:
diff --git a/src/twisted/mail/test/test_imap.py b/src/twisted/mail/test/test_imap.py
index ff1263ca43..d2c63eb690 100644
--- a/src/twisted/mail/test/test_imap.py
+++ b/src/twisted/mail/test/test_imap.py
@@ -1161,6 +1161,24 @@ class IMAP4HelperTests(TestCase):
             b"BODY[1.3.9.11.HEADER.FIELDS.NOT (Message-Id Date)]<103.69>",
         )
 
+    def test_fetchParserParseStringPartial(self):
+        """
+        Partial requests starting at 0 and longer than the message
+        return the entire body.
+        """
+        p = imap4._FetchParser()
+        p.parseString(b"BODY.PEEK[]<0.393216>")
+        self.assertEqual(len(p.result), 1)
+        self.assertTrue(isinstance(p.result[0], p.Body))
+        self.assertEqual(p.result[0].peek, True)
+        self.assertEqual(p.result[0].partialBegin, 0)
+        self.assertEqual(p.result[0].partialLength, 393216)
+        self.assertEqual(p.result[0].empty, True)
+        self.assertEqual(
+            p.result[0].getBytes(length=100),
+            b"BODY[]<0>",
+        )
+
     def test_fetchParserQuotedHeader(self):
         """
         Parsing a C{BODY} whose C{HEADER} values require quoting