Repository: postlund/pyatv

Issue:
Add RSA encryption to support AirPort Express streaming
**What feature would you like?**
The AirPort Express requires encryption when streaming. This is well known and reverse engineered, so it shouldn't be that hard to implement. Just need to do it.

**Describe the solution you'd like**
No particular keys needs to be provided by the user, the private and public keys used for the crypto needs to be bundled. Otherwise this should be transparent (other than the check for supported cryptos needs to be changed).

**Any other information to share?**
Relates to #1059


Comments:

Patch:
diff --git a/docs/documentation/protocols.md b/docs/documentation/protocols.md
index 42da2c6d..08430c35 100644
--- a/docs/documentation/protocols.md
+++ b/docs/documentation/protocols.md
@@ -1183,6 +1183,47 @@ CSeq: 3
 
 This section deals with "video part" of AirPlay. TBD
 
+### Commands
+
+#### /auth-setup
+
+Devices supporting MFi authentication (e.g. has `et=4`) might require an authentication step
+initiated by `/auth-setup`. This is always the case for AirPlay 2. More details
+[here](https://openairplay.github.io/airplay-spec/audio/rtsp_requests/post_auth_setup.html).
+
+*TODO: document more*
+
+The request consists of one byte encryption type (0x01: unencrypted,
+0x02: MFi-SAP-encrypted AES key) and 32 bytes Curve25519 public key. Normally this step is used
+to verify MFi authenticity, but no further action needs to be taken (i.e. just send request
+and ignore response) for devices requiring this step. Implementation in `pyatv` has been stolen
+from owntone [here](https://github.com/owntone/owntone-server/blob/c1db4d914f5cd8e7dbe6c1b6478d68a4c14824af/src/outputs/raop.c#L1568).
+
+**Sender -> Receiver:**
+```raw
+POST /auth-setup RTSP/1.0
+CSeq: 0
+User-Agent: AirPlay/540.31
+DACP-ID: BFAA2A9155BD093C
+Active-Remote: 347218209
+Client-Instance: BFAA2A9155BD093C
+Content-Type: application/octet-stream
+Content-Length: 33
+
+015902ede90d4ef2bd4cb68a6330038207a94dbd50d8aa465b5d8c012a0c7e1d4e27
+```
+
+**Receiver -> Sender:**
+```raw
+RTSP/1.0 200 OK
+Content-Length: 1076
+Content-Type: application/octet-stream
+Server: AirTunes/366.0
+CSeq: 0
+
+97a02c0d0a31486316de944d8404f4e01f93b05dde4543cc022a5727e8a352330000038c3082038806092a864886f70d010702a0820379308203750201013100300b06092a864886f70d010701a082035d3082035930820241a003020102020f1212aa121127aa00aa8023aa238776300d06092a864886f70d0101050500308183310b300906035504061302555331133011060355040a130a4170706c6520496e632e31263024060355040b131d4170706c652043657274696669636174696f6e20417574686f72697479313730350603550403132e4170706c652069506f64204163636573736f726965732043657274696669636174696f6e20417574686f72697479301e170d3132313132373138323135305a170d3230313132373138323135305a3070310b300906035504061302555331133011060355040a0c0a4170706c6520496e632e311f301d060355040b0c164170706c652069506f64204163636573736f72696573312b302906035504030c224950415f31323132414131323131323741413030414138303233414132333837373630819f300d06092a864886f70d010101050003818d003081890281810097e302c45e7b6f387dd390201b0dd902b19dc30d72a93a8b9f1313c6108e90ee93daff24177526736e4f1f58a2c2382cf4b7f7359bb1b1a3a7595850d489f335557a48653d96e9407ccc05eba6c867716e446b31d2bdc9c5122af4c213e7d7f0635b74e323094483a900bd3f93ce8833785b2fd14d88fb2dd4c581e1189b38250203010001a360305e301d0603551d0e04160414d74ea8b90475ee5140d2be7d2f9258931c7543cb300c0603551d130101ff04023000301f0603551d23041830168014ff4b1a439af51996ab18002b61c9ee409d8ec704300e0603551d0f0101ff0404030203b8300d06092a864886f70d0101050500038201010012e8b29b1e1b81e7a14b6435b92a9c58f0a28e6bcb645edd223969b77a70dda3ddc280562f53cb87e3ccd5fea213ccc9c2a4f005c3aa4447b84a895df649f74e9f6612d6cc69eeb7561706fa718f5e1d80b0554affe911c6fa3f99ca06bcf4debf03b64449bde16058392c830be55ae33273d24eecaf0f4aef6f6c46bed87192e2773e5ae092098b32563a532164df5eecd3fc299c8b267cf555b516b02a013920242f4162e6cb5d8d555356d3999c989860ed8c4ea2a0f34af4bcc74b864a07c6d952115dd28b0cc5d8bc780567dcaafc721e678391a048b00cf8664d5c0ad1949b57165a7c98144480ac0510a1887e27821d966b14478c901f6c7548f8563e310000000080121b14309c641bc593196f886c633d19986c11ca9cb4be2fdad1f2ec1427eeb8da23aaeaf7a713f2b8e05a6942db364e3dd408d5a1eeb1525baadc5ccb46614dadef1bfa565c65f46a54f576802209faa39ac442ac7cd43995be833f7794d0517fd93218e86c0228b30b036d3055476114d926de2875bed7cef4970492df58a3
+```
+
 ## References
 
 [RAOP-Player](https://github.com/philippe44/RAOP-Player)
diff --git a/docs/documentation/supported_features.md b/docs/documentation/supported_features.md
index e5e64285..a9203a47 100644
--- a/docs/documentation/supported_features.md
+++ b/docs/documentation/supported_features.md
@@ -192,8 +192,24 @@ AirTunes).
 
 * Metadata and push updates only reflect what pyatv is currently playing as there
   seems to not be possible to get current play state from an AirPlay receiver
-* Devices requiring password, pairing or any kind of encryption are not supported,
-  e.g. Apple TV 4+ or AirPort Express {% include issue no="1077,1078" %}
+* Devices requiring password are not supported
+* Devices requiring HAP pairing, e.g. Apple TV 4+, are not supported {% include issue no="1078" %}
 * Track length is hardcoded to 3s for now
 * Remote control commands does not work, e.g. play or pause {% include issue no="1068" %}
 * Retransmission of lost packets is not supported {% include issue no="1079" %}
+
+### Verified Devices
+
+Audio streaming has been verified to work with these devices:
+
+* Apple TV 3 (v8.4.4)
+* HomePod Mini (v14.5)
+* AirPort Express (v7.8.1)
+* Yamaha RX-V773 (v1.98)
+
+The following 3rd party software receivers have also been verified to work:
+
+* [shairport-sync](https://github.com/mikebrady/shairport-sync) (v3.3.8)
+
+If you have verified another device or receiver, please update the list by pressing
+*Edit this page* below and opening a pull request.
diff --git a/pyatv/raop/parsers.py b/pyatv/raop/parsers.py
index ea47bb70..96c8f00e 100644
--- a/pyatv/raop/parsers.py
+++ b/pyatv/raop/parsers.py
@@ -23,6 +23,15 @@ class EncryptionType(IntFlag):
     FairPlaySAPv25 = 16
 
 
+class MetadataType(IntFlag):
+    """Metadata types supported by receiver."""
+
+    NotSupported = 0
+    Text = 1
+    Artwork = 2
+    Progress = 4
+
+
 # pylint: enable=invalid-name
 
 
@@ -43,9 +52,9 @@ def get_encryption_types(properties: Mapping[str, str]) -> EncryptionType:
 
     Input format from zeroconf is a comma separated list:
 
-        0,1,3
+        et=0,1,3
 
-    Each number represents one encryption type.
+    0=unencrypted, 1=RSA, 3=FairPlay, 4=MFiSAP, 5=FairPlay SAPv2.5
     """
     output = EncryptionType.Unknown
     try:
@@ -63,3 +72,27 @@ def get_encryption_types(properties: Mapping[str, str]) -> EncryptionType:
                 5: EncryptionType.FairPlaySAPv25,
             }.get(enc_type, EncryptionType.Unknown)
     return output
+
+
+def get_metadata_types(properties: Mapping[str, str]) -> MetadataType:
+    """Return metadata types supported by receiver.
+
+    Input format from zeroconf is comma separated list:
+
+        md=0,1,2
+
+    0=text, 1=artwork, 2=progress
+    """
+    output = MetadataType.NotSupported
+    try:
+        md_types = [int(x) for x in properties["md"].split(",")]
+    except (KeyError, ValueError):
+        return output
+    else:
+        for md_type in md_types:
+            output |= {
+                0: MetadataType.Text,
+                1: MetadataType.Artwork,
+                2: MetadataType.Progress,
+            }.get(md_type, MetadataType.NotSupported)
+    return output
diff --git a/pyatv/raop/raop.py b/pyatv/raop/raop.py
index fcc60fdd..5b00d0e2 100644
--- a/pyatv/raop/raop.py
+++ b/pyatv/raop/raop.py
@@ -15,8 +15,10 @@ from pyatv.raop.metadata import EMPTY_METADATA, AudioMetadata
 from pyatv.raop.packets import AudioPacketHeader, SyncPacket, TimingPacket
 from pyatv.raop.parsers import (
     EncryptionType,
+    MetadataType,
     get_audio_properties,
     get_encryption_types,
+    get_metadata_types,
 )
 from pyatv.raop.rtsp import FRAMES_PER_PACKET, RtspContext, RtspSession
 from pyatv.support import log_binary
@@ -33,6 +35,8 @@ MISSING_METADATA = AudioMetadata(
     title="Streaming with pyatv", artist="pyatv", album="RAOP"
 )
 
+SUPPORTED_ENCRYPTIONS = EncryptionType.Unencrypted | EncryptionType.MFiSAP
+
 
 class ControlClient(asyncio.Protocol):
     """Control client responsible for e.g. sync packets."""
@@ -240,6 +244,8 @@ class RaopClient:
         self.context: RtspContext = context
         self.control_client: Optional[ControlClient] = None
         self.timing_client: Optional[TimingClient] = None
+        self._encryption_types: EncryptionType = EncryptionType.Unknown
+        self._metadata_types: MetadataType = MetadataType.NotSupported
         self._metadata: AudioMetadata = EMPTY_METADATA
         self._keep_alive_task: Optional[asyncio.Future] = None
         self._listener: Optional[weakref.ReferenceType[Any]] = None
@@ -291,11 +297,20 @@ class RaopClient:
 
     async def initialize(self, properties: Mapping[str, str]):
         """Initialize the session."""
+        self._encryption_types = get_encryption_types(properties)
+        self._metadata_types = get_metadata_types(properties)
+
+        _LOGGER.debug(
+            "Initializing RTSP with encryption=%s, metadata=%s",
+            self._encryption_types,
+            self._metadata_types,
+        )
+
         # Misplaced check that unencrypted data is supported
-        encryption_types = get_encryption_types(properties)
-        if EncryptionType.Unencrypted not in encryption_types:
+        intersection = self._encryption_types & SUPPORTED_ENCRYPTIONS
+        if not intersection or intersection == EncryptionType.Unknown:
             raise exceptions.NotSupportedError(
-                f"no supported encryption types in {str(encryption_types)}"
+                f"no supported encryption types in {str(self._encryption_types)}"
             )
 
         self._update_output_properties(properties)
@@ -331,6 +346,10 @@ class RaopClient:
         )
 
     async def _setup_session(self):
+        # Do auth-setup if MFiSAP encryption is supported by receiver
+        if EncryptionType.MFiSAP in self._encryption_types:
+            await self.rtsp.auth_setup()
+
         await self.rtsp.announce()
 
         resp = await self.rtsp.setup(self.control_client.port, self.timing_client.port)
@@ -377,14 +396,18 @@ class RaopClient:
             # Start sending sync packets
             self.control_client.start(self.rtsp.remote_ip)
 
+            # Apply text metadata if it is supported
             self._metadata = metadata
-            _LOGGER.debug("Playing with metadata: %s", self.metadata)
-            await self.rtsp.set_metadata(
-                self.context.rtpseq, self.context.rtptime, self.metadata
-            )
+            if MetadataType.Text in self._metadata_types:
+                _LOGGER.debug("Playing with metadata: %s", self.metadata)
+                await self.rtsp.set_metadata(
+                    self.context.rtpseq, self.context.rtptime, self.metadata
+                )
 
             # Start keep-alive task to ensure connection is not closed by remote device
-            self._keep_alive_task = asyncio.ensure_future(self._send_keep_alive())
+            # but only if "text" metadata is supported
+            if MetadataType.Text in self._metadata_types:
+                self._keep_alive_task = asyncio.ensure_future(self._send_keep_alive())
 
             listener = self.listener
             if listener:
diff --git a/pyatv/raop/rtsp.py b/pyatv/raop/rtsp.py
index 1f7d5a69..86c3c406 100644
--- a/pyatv/raop/rtsp.py
+++ b/pyatv/raop/rtsp.py
@@ -4,7 +4,7 @@ import logging
 from random import randrange
 import re
 from socket import socket
-from typing import Dict, Mapping, NamedTuple, Optional, Tuple, Union
+from typing import Dict, Mapping, NamedTuple, Optional, Tuple, Union, cast
 
 from pyatv import exceptions
 from pyatv.dmap import tags
@@ -28,6 +28,20 @@ ANNOUNCE_PAYLOAD = (
     + "{bits_per_channel} 40 10 14 {channels} 255 0 0 {sample_rate}\r\n"
 )
 
+# Used to signal that traffic is to be unencrypted
+AUTH_SETUP_UNENCRYPTED = b"\x01"
+
+# Just a static Curve25519 public key used to satisfy the auth-setup step for devices
+# requiring that (e.g. AirPort Express). We never verify anything. Source:
+# https://github.com/owntone/owntone-server/blob/
+# c1db4d914f5cd8e7dbe6c1b6478d68a4c14824af/src/outputs/raop.c#L276
+CURVE25519_PUB_KEY = (
+    b"\x59\x02\xed\xe9\x0d\x4e\xf2\xbd"
+    b"\x4c\xb6\x8a\x63\x30\x03\x82\x07"
+    b"\xa9\x4d\xbd\x50\xd8\xaa\x46\x5b"
+    b"\x5d\x8c\x01\x2a\x0c\x7e\x1d\x4e"
+)
+
 
 class RtspContext:
     """Data used for one RTSP session.
@@ -79,16 +93,16 @@ class RtspResponse(NamedTuple):
     code: int
     message: str
     headers: Mapping[str, str]
-    body: str
+    body: Union[str, bytes]
 
 
-def parse_response(response: str) -> Tuple[RtspResponse, str]:
+def parse_response(response: bytes) -> Tuple[Optional[RtspResponse], bytes]:
     """Parse RTSP response."""
     try:
-        header_str, body = response.split("\r\n\r\n", maxsplit=1)
+        header_str, body = response.split(b"\r\n\r\n", maxsplit=1)
     except ValueError as ex:
         raise ValueError("missing end lines") from ex
-    headers = header_str.split("\r\n")
+    headers = header_str.decode("utf-8").split("\r\n")
 
     match = re.match(r"RTSP/1.0 (\d+) (.+)", headers[0])
     if not match:
@@ -103,11 +117,18 @@ def parse_response(response: str) -> Tuple[RtspResponse, str]:
             resp_headers[key] = value
 
     content_length = int(resp_headers.get("Content-Length", 0))
-    if body and len(body) < content_length:
-        raise ValueError("too short body")
+    if len(body or []) < content_length:
+        return None, response
+
+    response_body: Union[str, bytes] = body[0:content_length]
+
+    # Assume body is text unless content type is application/octet-stream
+    if resp_headers.get("Content-Type") != "application/octet-stream":
+        # We know it's bytes here
+        response_body = cast(bytes, response_body).decode("utf-8")
 
     return (
-        RtspResponse(int(code), message, resp_headers, body[0:content_length]),
+        RtspResponse(int(code), message, resp_headers, response_body),
         body[content_length:],
     )
 
@@ -121,6 +142,7 @@ class RtspSession(asyncio.Protocol):
         self.transport = None
         self.requests: Dict[int, Tuple[asyncio.Semaphore, RtspResponse]] = {}
         self.cseq = 0
+        self.buffer = b""
 
     @property
     def local_ip(self) -> str:
@@ -157,9 +179,13 @@ class RtspSession(asyncio.Protocol):
     def data_received(self, data: bytes) -> None:
         """Handle incoming RTSP data."""
         _LOGGER.debug("Received: %s", data)
-        rest = data.decode("utf-8")
-        while rest != "":
-            parsed, rest = parse_response(rest)
+        self.buffer += data
+        while self.buffer:
+            parsed, self.buffer = parse_response(self.buffer)
+            if parsed is None:
+                _LOGGER.debug("Not enough data to decode message")
+                break
+
             if "CSeq" in parsed.headers:
                 cseq = int(parsed.headers["CSeq"])
                 if cseq in self.requests:
@@ -176,6 +202,18 @@ class RtspSession(asyncio.Protocol):
         """Handle that connection was lost."""
         _LOGGER.debug("RTSP Connection closed")
 
+    async def auth_setup(self) -> RtspResponse:
+        """Send auth-setup message."""
+        # Payload to say that we want to proceed unencrypted
+        body = AUTH_SETUP_UNENCRYPTED + CURVE25519_PUB_KEY
+
+        return await self.send_and_receive(
+            "POST",
+            uri="/auth-setup",
+            content_type="application/octet-stream",
+            body=body,
+        )
+
     async def announce(self) -> RtspResponse:
         """Send ANNOUNCE message."""
         body = ANNOUNCE_PAYLOAD.format(
@@ -249,6 +287,7 @@ class RtspSession(asyncio.Protocol):
     async def send_and_receive(
         self,
         method: str,
+        uri: Optional[str] = None,
         content_type: Optional[str] = None,
         headers: Mapping[str, object] = None,
         body: Union[str, bytes] = None,
@@ -260,7 +299,7 @@ class RtspSession(asyncio.Protocol):
         if isinstance(body, str):
             body = body.encode("utf-8")
 
-        msg = f"{method} {self.uri} RTSP/1.0"
+        msg = f"{method} {uri or self.uri} RTSP/1.0"
         msg += f"\r\nCSeq: {cseq}"
         msg += f"\r\nUser-Agent: {USER_AGENT}"
         msg += f"\r\nDACP-ID: {self.context.dacp_id}"


Test patch:
diff --git a/tests/raop/test_parsers.py b/tests/raop/test_parsers.py
index c58bd541..ef71eb04 100644
--- a/tests/raop/test_parsers.py
+++ b/tests/raop/test_parsers.py
@@ -5,8 +5,10 @@ import pytest
 from pyatv.exceptions import ProtocolError
 from pyatv.raop.parsers import (
     EncryptionType,
+    MetadataType,
     get_audio_properties,
     get_encryption_types,
+    get_metadata_types,
 )
 
 
@@ -19,7 +21,7 @@ from pyatv.raop.parsers import (
         ({"ss": "32"}, 44100, 2, 4),
     ],
 )
-def test_parse_properties(properties, expected_sr, expected_ch, expected_ss):
+def test_parse_audio_properties(properties, expected_sr, expected_ch, expected_ss):
     sample_rate, channels, sample_size = get_audio_properties(properties)
     assert sample_rate == expected_sr
     assert channels == expected_ch
@@ -27,7 +29,7 @@ def test_parse_properties(properties, expected_sr, expected_ch, expected_ss):
 
 
 @pytest.mark.parametrize("properties", [{"sr": "abc"}, {"ch": "cde"}, {"ss": "fgh"}])
-def test_parse_invalid_property_raises(properties):
+def test_parse_invalid_audio_property_raises(properties):
     with pytest.raises(ProtocolError):
         get_audio_properties(properties)
 
@@ -64,3 +66,19 @@ def test_parse_encryption_include_unknown_type():
         get_encryption_types({"et": "0,1000"})
         == EncryptionType.Unknown | EncryptionType.Unencrypted
     )
+
+
+@pytest.mark.parametrize(
+    "properties,expected",
+    [
+        ({}, MetadataType.NotSupported),
+        ({"md": "0"}, MetadataType.Text),
+        ({"md": "1"}, MetadataType.Artwork),
+        (
+            {"md": "0,1,2"},
+            MetadataType.Text | MetadataType.Artwork | MetadataType.Progress,
+        ),
+    ],
+)
+def test_parse_metadata_types(properties, expected):
+    assert get_metadata_types(properties) == expected
diff --git a/tests/raop/test_rtsp.py b/tests/raop/test_rtsp.py
index cdc3a8d7..dc373c18 100644
--- a/tests/raop/test_rtsp.py
+++ b/tests/raop/test_rtsp.py
@@ -6,51 +6,62 @@ from pyatv.raop.rtsp import parse_response
 
 
 def test_parse_ok_first_line():
-    resp, rest = parse_response("RTSP/1.0 200 OK\r\n\r\n")
+    resp, rest = parse_response(b"RTSP/1.0 200 OK\r\n\r\n")
     assert resp.code == 200
     assert resp.message == "OK"
-    assert rest == ""
+    assert rest == b""
 
 
 def test_parse_missing_ending():
     with pytest.raises(ValueError) as exc:
-        parse_response("RTSP/1.0 200 OK\r\n")
+        parse_response(b"RTSP/1.0 200 OK\r\n")
 
     assert "missing end lines" in str(exc)
 
 
 def test_parse_headers():
-    resp, rest = parse_response("RTSP/1.0 200 OK\r\nA: B\r\nC: D\r\n\r\n")
+    resp, rest = parse_response(b"RTSP/1.0 200 OK\r\nA: B\r\nC: D\r\n\r\n")
     assert len(resp.headers) == 2
     assert resp.headers["A"] == "B"
     assert resp.headers["C"] == "D"
-    assert rest == ""
+    assert rest == b""
 
 
 def test_parse_body():
-    resp, rest = parse_response("RTSP/1.0 200 OK\r\nContent-Length: 4\r\n\r\nbody")
+    resp, rest = parse_response(b"RTSP/1.0 200 OK\r\nContent-Length: 4\r\n\r\nbody")
     assert resp.body == "body"
-    assert rest == ""
+    assert rest == b""
+
+
+def test_parse_too_no_body():
+    content = b"RTSP/1.0 200 OK\r\nContent-Length: 5\r\n\r\n"
+    resp, rest = parse_response(content)
+
+    assert resp is None
+    assert rest == content
 
 
 def test_parse_too_short_body():
-    with pytest.raises(ValueError) as exc:
-        parse_response("RTSP/1.0 200 OK\r\nContent-Length: 5\r\n\r\nbody")
+    content = b"RTSP/1.0 200 OK\r\nContent-Length: 5\r\n\r\nbody"
+    resp, rest = parse_response(content)
 
-    assert "too short body" in str(exc)
+    assert resp is None
+    assert rest == content
 
 
 def test_parse_body_excessive_data():
-    resp, rest = parse_response("RTSP/1.0 200 OK\r\nContent-Length: 4\r\n\r\nbodyextra")
+    resp, rest = parse_response(
+        b"RTSP/1.0 200 OK\r\nContent-Length: 4\r\n\r\nbodyextra"
+    )
     assert resp.body == "body"
-    assert rest == "extra"
+    assert rest == b"extra"
 
 
 def test_parse_sequent_messages():
     resp, rest = parse_response(
-        "RTSP/1.0 200 OK\r\nA: B\r\n\r\n"
-        "RTSP/1.0 200 OK\r\nContent-Length: 2\r\n\r\nAB"
-        "RTSP/1.0 200 OK\r\nContent-Length: 0\r\n\r\n"
+        b"RTSP/1.0 200 OK\r\nA: B\r\n\r\n"
+        b"RTSP/1.0 200 OK\r\nContent-Length: 2\r\n\r\nAB"
+        b"RTSP/1.0 200 OK\r\nContent-Length: 0\r\n\r\n"
     )
     assert resp.headers["A"] == "B"
     assert resp.body == ""
@@ -61,4 +72,13 @@ def test_parse_sequent_messages():
     resp, rest = parse_response(rest)
     assert resp.headers["Content-Length"] == "0"
     assert resp.body == ""
-    assert rest == ""
+    assert rest == b""
+
+
+def test_parse_raw_body():
+    resp, rest = parse_response(
+        b"RTSP/1.0 200 OK\r\nContent-Length: 4\r\n"
+        + b"Content-Type: application/octet-stream\r\n\r\nbodyextra"
+    )
+    assert resp.body == b"body"
+    assert rest == b"extra"