Streaming control for real-time transport protocol

Dynamic control of transport protocols utilized in the streaming of media content based in part on the type of content, latency requirements, network conditions and/or device capabilities is described. The techniques provide a source device to dynamically switch between User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) to stream media content to a sink device. For example, during a Wi-Fi peer-to-peer remote display session associated with real-time media content (e.g., live streaming sporting event, or gaming applications), the source device may utilize Real-time Transport Protocol (RTP) over UDP to transmit the media stream to the sink device. Conversely, when the media content is not latency critical, such as playback of stored media (e.g., movie), the source device may dynamically switch to RTP over TCP in order to provide reliable data transmission.

BACKGROUND

The following relates generally to wireless communication, and more specifically to Wi-Fi peer-to-peer remote display. Wireless communications systems are widely deployed to provide various types of communication content such as voice, video, packet data, messaging, broadcast, and so on. These systems may be wireless local area network (WLAN), also known as Wi-Fi systems which utilize carrier sense multiple access with collision avoidance (CSMA/CA) mechanisms to access a wireless medium. These systems may also be multiple-access systems capable of supporting communication with multiple users by sharing the available system resources (e.g., time, frequency, and power). Examples of such multiple-access systems include code-division multiple access (CDMA) systems, time-division multiple access (TDMA) systems, frequency-division multiple access (FDMA) systems, and orthogonal frequency-division multiple access (OFDMA) systems.

As mobile devices are used to capture or generate content such as audio, video, or multimedia, users may desire to share content between mobile devices and other devices such as TVs, computers, audio systems, and the like. One approach is for one device (i.e., a sink device) to mirror what is displayed on another device (i.e., a source device). Examples of a source device may include a smartphone, tablet, and the like. Examples of a sink device may include a TV, computer screen, etc. In some applications, the source device may transmit the media stream over a wireless link. For example, a Wi-Fi peer-to-peer network allows wireless devices to directly communicate with each other. Devices within range of each other may discover and communicate directly without involving central access points.

Wi-Fi peer-to-peer connections allow source devices to wirelessly transmit media content such as video and/or audio to a compatible sink display. Various approaches have been considered to effectively transmit the media content from the source device to the sink device. For instance, on approach allows a source device to encapsulate media content using an MPEG2 Transport Stream (MPEG-TS) for transmission over a medium utilizing Real-time Transport Protocol (RTP) over User Datagram Protocol (UDP). The RTP, designed for end-to-end, real-time transfer of streaming content, defines a standardized packet format for delivering media content over Internet Protocol (IP) networks. Conventional methods for Wi-Fi peer-to-peer remote display implement RTP over UDP because UDP is suitable for real-time transfer of latency critical multimedia content. UDP protocol, however, relies on Wi-Fi MAC layer retransmissions, which may result in quality artifacts if the network congestion grows above a certain threshold.

SUMMARY

The described features generally relate to one or more improved systems, methods, and/or apparatuses for dynamic control of transport protocols utilized in transmission of the streaming media content based in part on the type of content, latency requirements, network conditions and/or device capabilities. In some examples, the source device may dynamically switch between User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) to stream media content to the sink device. For example, during a Wi-Fi peer-to-peer remote display session associated with real-time media content (e.g., live streaming, gaming, or display of interactive user inputs), the source device may utilize Real-time Transport Protocol (RTP) over UDP to transmit the media stream to the sink device. Conversely, when the media content is not latency critical, such as playback of stored media (e.g., movie), the source device may dynamically switch to RTP over TCP in order to provide reliable data transmission. As a result, the source device and the sink device may select an appropriate transport mechanism based on the type of content to stream, network conditions and/or device capabilities during a Wi-Fi peer-to-peer remote display session.

In another example, the source device may control playback functions, such as volume of the sink device processing the media stream. As a result, the volume change may be reflected immediately even when there is buffered data yet to be played at the sink device. Further, the playback volume control functionalities of the present disclosure may be used to control the volume at the sink device irrespective of the volume level of the audio in the media stream transmitted from the source device.

In a first set of illustrative examples, a method for wireless communication is provided. The method may comprise initiating, by a source device, a media stream to a sink device over a first protocol during a Wi-Fi peer-to-peer remote display session. The source device may further identify a second protocol supported by the sink device and dynamically switching to the second protocol to transmit the media stream during the Wi-Fi peer-to-peer remote display session.

In one example, the first protocol may comprise a user datagram protocol (UDP) and the second protocol may comprise transmission control protocol (TCP). In another example, the first protocol may comprise TCP and the second protocol may comprise UDP. In some examples, identifying the second protocol supported by the sink device may comprise transmitting a query to the sink device to determine whether the sink device supports TCP and one or more ports to be used for a TCP connection. In response, the source device may receive a message affirming that the sink device supports TCP and the ports information.

The source device may further transmit, in response to receiving the message, a setup request to the sink device, wherein the setup request may comprise profile and port information. The source device may further query the sink device to determine decoder latency and a buffer size of the sink device. In some examples, dynamically switching from the first protocol to the second protocol is based at least in part on a type of media content, network conditions, or available buffer space at the sink device. The source device may further select, based at least in part on the type of media content, network conditions, or available buffer space at the sink device, a buffer size to be used by the sink device for buffering the media stream prior to initiating playback. The source device may further transmit the selected buffer size to the sink device.

In one example, the source device may further query the sink device for presentation timestamp (PTS) value of the media stream being played and a pending buffer size of the media stream waiting to be played. The source device may receive a message, in response to the query, indicating the PTS value of the media stream currently being played and the buffer size of the media stream waiting to be played. In some examples, the source device may issue a flush control command to flush the buffer of the sink device, wherein the flush control command instructs the sink device to discard data in the sink buffer until a data packet with the PTS value. In another example, the source device may issue a volume control command to the sink device, wherein the volume control command controls the playback volume of the media stream at the sink device.

In a second set of illustrative examples, an apparatus for wireless communication is provided. The apparatus may comprise a processor and a memory in electronic communications with the processor. The memory may embody instructions being executable by the processor to initiate, by a source device, a media stream to a sink device over a first protocol during a Wi-Fi peer-to-peer remote display session. The instructions may further be executable by the processor to identify a second protocol supported by the sink device and dynamically switch to the second protocol to transmit the media stream during the Wi-Fi peer-to-peer remote display session. In certain examples, the apparatus may implement one or more aspects of the method for wireless communications described above with respect to the first set of illustrative examples.

In a third set of illustrative examples, an apparatus for wireless communication is provided. The apparatus may comprise a multi-protocol communication component to initiate, by a source device, a media stream to a sink device over a first protocol during a Wi-Fi peer-to-peer remote display session. The apparatus may further comprise a capability analysis component to identify a second protocol supported by the sink device and a dynamic switching component to dynamically switch to the second protocol to transmit the media stream during the Wi-Fi peer-to-peer remote display session. In certain examples, the apparatus may implement one or more aspects of the method for wireless communications described above with respect to the first set of illustrative examples.

DETAILED DESCRIPTION

Described embodiments are directed to systems and methods for the source device to dynamically switch between User Datagram Protocol (UDP) and Transmission Control Protocol (TCP) to stream media content, such as audio and/or video data, to the sink device. For instance, during a Wi-Fi peer-to-peer remote display session associated with real-time media content (e.g., streaming sporting event), the source device may utilize Real-time Transport Protocol (RTP) over UDP to stream the media content to the sink device. Conversely, when the media content is not latency critical, such as playback of stored media (e.g., movie), the source device may dynamically switch to RTP over TCP in order to provide reliable data transmission. As a result, the source device and the sink device may dynamically select the transport mechanism based on the type of content to stream, network conditions and/or device capabilities during a Wi-Fi peer-to-peer remote display session.

In another example, the source device may control playback functions, such as volume of the sink device that processes the media stream. As a result, the volume change may be reflected immediately even when there is buffered data yet to be played at the sink device. Further, the playback volume control functionalities of the present disclosure may be used to control the volume at the sink device irrespective of the volume level of the audio in the media stream transmitted from the source device.

Referring now toFIG. 1, a system100includes a source device115and a sink device135and may include one or more access points105. Examples of the source device115may include, but are not limited to, smartphones, cell phones, wireless headphones, wearable computing devices, tablets, personal digital assistants (PDAs), laptops, or any other device capable of communicating with a sink device135via a connection (e.g., wired, cellular wireless, Wi-Fi, etc.). Examples of the sink device135may include, but are not limited to, in-vehicle infotainment devices, TVs, computers, laptops, projectors, cameras, smartphones, wearable computing devices, or any other device capable of communicating with a source device115and displaying content received from the source device115. The sink device135may be a combination of devices. For example, the sink device135may include a display device and a separate device for receiving, buffering, and decoding content for display on the display device.

Source device115may be connected to sink device135via link125. Link125is illustrated inFIG. 1as a wireless link but may be a wired or wireless link in some embodiments. Communications between a source device115and a sink device135connected via a wireless peer-to-peer connection may be performed to remotely render content of the source device115at the sink device135. Wi-Fi remote display includes, but is not limited to the Wi-Fi Display specification, also known as Miracast® from Wi-Fi Alliance, Discovery and Launch (DIAL), Digital Living Network Alliance® (DLNA), Airplay, WirelessHD, Wireless Home Digital Interface (WHDI), Intel's Wireless Display (Wi-Di) technology, and Ultra-wideband (UWB) connections. While the following techniques are described using the wireless networking architecture illustrated inFIG. 1, the described techniques are applicable to any suitable wired or wireless communication technology.

In one embodiment, the source device115is connected to the sink device135via a Wi-Fi Display connection. Wi-Fi Display protocol, which may be known as Miracast, allows a portable device or computer to transmit media content (e.g., video, audio, images, etc.) to a compatible display wirelessly. It enables delivery of compressed standard or high-definition video over a wireless link125. It also may allow users to echo the display from one device onto the display of another device. Wireless link125may be a direct wireless link (e.g., peer-to-peer link125-a), or an indirect wireless link (e.g., indirect link125-b) through a Wi-Fi access point. Examples of direct wireless links include Wi-Fi Direct connections and connections established by using a Wi-Fi Tunneled Direct Link Setup (TDLS) link.

Wi-Fi Display allows users to echo the display from one device onto the display of another device by video and/or audio content streaming. The link125between the source device115and sink device135may be bi-directional. In one example, the connection between the source device115and a sink device135may also allow users to launch applications stored on the source device115via the sink device135. For example, the sink device135may include various input controls (e.g., mouse, keyboard, knobs, keys, user interface buttons). These controls may be used at the sink device135to initialize and interact during the audio/video streaming from the source through the media applications stored on the source device115.

Wi-Fi Display may use a transport stream, such as an MPEG2 Transport Stream (MPEG-TS). The content may be encoded according to a media encoding format (e.g., h.264, MPEG-4, h.265, etc.) and may be multiplexed into the transport stream with other information (e.g., error correction, stream synchronization, etc.) for transmission to the sink device135. The system100including the source device115may be configured to dynamically switch between UDP and TCP to stream media content such as audio and/or video data to the sink device135. For instance, during a Wi-Fi peer-to-peer remote display session associated with the real-time media content (e.g., live streaming, gaming, or display of interactive user inputs), the source device may utilize RTP over UDP to stream the media content to the sink device. Conversely, when the media content is not latency critical, such as playback of stored media (e.g., movie), the source device may dynamically switch to RTP over TCP in order to provide reliable data transmission. Adding the capabilities of TCP transport for carrying media data in a Wi-Fi peer-to-peer display session allows a source device and the sink device to select the best transport mechanism based on the type of content, network conditions, or the other implementation factors such as available buffering. Because the connection-oriented TCP is suited to adapt to link conditions, utilization of TCP may substantially improve the quality of media playback in congested network environments.

In some examples, before or after a Wi-Fi peer-to-peer remote display connection has been established between the source device115and the sink device135, the devices may engage in a capability negotiation procedure. As part of the capability negotiation procedure, the source device115may query the sink device135for various information regarding the capabilities of the sink device135to establish a Wi-Fi remote display connection. The source device115may query the sink device135by sending Real-Time Streaming Protocol (RTSP) request messages. In one example, the source device115may query the sink device135using RTSP request messages (e.g., Get_Parameter request message) to determine whether the sink device135is capable of supporting the use of TCP transport for media data, including a dual-mode communication, wherein the dual-mode communication may comprise supporting RTP media content transport over UDP and/or TCP.

In one example, during the capability negotiation procedure, the sink device135may generate a response to the queries to indicate whether the sink device135is capable of supporting dual-mode communication (i.e., UDP and TCP). Once the capability negotiation procedure has concluded, the source device115may establish communication with the sink device135to stream RTP media content to the sink device via either UDP or TCP.

Referring now toFIG. 2Ais a block diagram200-aillustrating a source device115-ain accordance with various embodiments. The source device115-amay be an example of one or more aspects of one of the source devices115described with reference toFIG. 1. The source device115-amay also be a processor. The source device115-amay include a source receiver205, a communication management component210, and a source transmitter215. Each of these components may be in communication with each other.

The components of the source device115-amay, individually or collectively, be implemented with one or more application-specific integrated circuits (ASICs) adapted to perform some or all of the applicable functions in hardware. Alternatively, the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits. In other embodiments, other types of integrated circuits may be used (e.g., Structured/Platform ASICs, Field Programmable Gate Arrays (FPGAs), and other Semi-Custom ICs), which may be programmed in any manner known in the art. The functions of each unit may also be implemented, in whole or in part, with instructions stored in a memory, formatted to be executed by one or more general or application-specific processors. In one embodiment, the components as shown inFIG. 2Aeach may include a circuit or circuitry for performing the functions described herein.

The source receiver205may receive communications from a sink device, such as one or more of the sink devices135described with reference toFIG. 1via one or more signals202. The received communications may be any suitable form of data. The source receiver205may receive these communications via a wireless (e.g., Wi-Fi) peer-to-peer connection that is established between the source device115-aand the sink device135described with reference toFIG. 1. The communication management component210may manage communications received by the source device115-avia one or more signals204. Additionally, the communication management component210may manage, via one or more signals206, communications transmitted from the source device115-ato the sink device(s). Further, the communication management component210may process the data to control or otherwise manage aspects of the source device115-afor providing audio and/or video streams, graphics resources, communication, and/or control instructions to one or more sink devices135.

The source transmitter215may transmit communications to a sink device, such as one or more of the sink device135described with reference toFIG. 1via one or more signals208. The transmitted communications may include data such as graphics resources, audio and/or video streams, and/or communication instructions. The source transmitter215may transmit these communications via a wireless (e.g., Wi-Fi) peer-to-peer connection that is established between the source device115-aand the sink device135. Details regarding the communication management component210will be described below.

FIG. 2Bis a block diagram200-billustrating a source device115-bin accordance with various embodiments. The source device115-bmay be an example of one or more aspects of one of the source devices115described with reference toFIGS. 1 and/or 2A. The source device115-bmay also be a processor. The source device115-bmay include a source receiver205-a, a communication management component210-a, and a source transmitter215-a. Each of these components may be in communication with each other.

The components of the source device115-bmay, individually or collectively, be implemented with one or more ASICs adapted to perform some or all of the applicable functions in hardware. Alternatively, the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits. In other embodiments, other types of integrated circuits may be used (e.g., Structured/Platform ASICs, FPGAs, and other Semi-Custom ICs), which may be programmed in any manner known in the art. The functions of each unit may also be implemented, in whole or in part, with instructions stored in a memory, formatted to be executed by one or more general or application-specific processors. In one embodiment, the components as shown inFIG. 2Beach may include a circuit or circuitry for performing the functions described herein.

The source transmitter215-amay be configured as previously described with respect toFIG. 2A. The source receiver205-amay also be configured as previously described with respect toFIG. 2A. The communication management component210-amay include a multi-protocol communication component220, a capability analysis component225, and a playback control component230.

In one embodiment, the multi-protocol communication component220may be configured to allow the source device115-bto support communication with the sink device over multiple protocols, such as UDP and TCP. In one example, the multi-protocol communication component220may initialize communication with the sink device using UDP during initial session to transmit audio/video data. In another example, the multi-protocol communication component220may select transmission over TCP as a default initial setting to communicate with the sink device. The selection of one of a plurality of transport protocols supported by the source device115-bmay be based in part on the capabilities of the sink device, type of media content, network conditions, and/or available buffering at the sink device. Yet further, the multi-protocol communication component220may be configured to allow the source device115-bto dynamically switch between UDP and TCP based on predetermined factors.

Capability analysis component225may assist the multi-protocol communication component220in selection of the appropriate transport protocol by querying the capabilities of the sink device and determining which one of the plurality of transport protocols may be best suited for streaming media content. The capability analysis component225, during the Wi-Fi peer-to-peer remote display session, may generate one or more queries for the sink device. In one example, the capability analysis component225may query the sink device to determine whether the sink device supports UDP and/or TCP protocols, and associated ports information to be used by the sink device. In yet another example, the capability analysis component225may additional query the sink device for its decoder latency and initial buffer size that the source device may be able to allocate. In response to the transmitted queries, the capability analysis component225may receive one or more messages from the sink device indicating its capabilities to the source device115-b.

In one example, a playback control component230may issue control commands (e.g., flush buffer, volume control etc.) to the sink device. In some examples, it may be necessary to flush the buffer of the sink device in order to allow the sink device to start buffering new data without playing previously buffered data. Similarly, the playback control component230may issue volume control commands in order to adjust the volume of the media content at the sink device. The volume control commands may adjust the volume at the sink device irrespective of the volume level of the audio in the stream transmitted from the source device. Some examples of volume control may include volume up, volume down, mute and/or unmute. The playback control component230may issue control commands while transmitting media content over either UDP or TCP. It should be understood by those of ordinary skill in the art that the playback control component230commands are not limited exclusively to flush buffer and/or volume control.

Referring now toFIG. 2Cis a block diagram200-cillustrating a source device115-cin accordance with various embodiments. The source device115-cmay be an example of one or more aspects of one of the source devices115described with reference toFIGS. 1, 2A and/or 2B. The source device115-cmay also be a processor. The source device115-cmay include a source receiver205-b, a communication management component210-b, and a source transmitter215-b. Each of these components may be in communication with each other.

The components of the source device115-cmay, individually or collectively, be implemented with one or more ASICs adapted to perform some or all of the applicable functions in hardware. Alternatively, the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits. In other embodiments, other types of integrated circuits may be used (e.g., Structured/Platform ASICs, FPGAs, and other Semi-Custom ICs), which may be programmed in any manner known in the art. The functions of each unit may also be implemented, in whole or in part, with instructions stored in a memory, formatted to be executed by one or more general or application-specific processors. In one embodiment, the components as shown inFIG. 2Ceach may include a circuit or circuitry for performing the functions described herein.

The source transmitter215-bmay be configured as previously described with respect toFIGS. 2A and/or 2B. The source receiver205-bmay also be configured as previously described with respect toFIG. 2A and/or 2B. The communication management component210-bmay include a multi-protocol communication component220-a, capability analysis component225-a,and a playback control component230-aas described with reference toFIG. 2B.

In one embodiment, the multi-protocol communication component220-amay further comprise TCP port component235, UDP port component240, and dynamic switching component245. The multi-protocol communication component220-amay utilize TCP port component235to transmit RTP media content over TCP to the sink device. In some examples, TCP port component235may provide communication service at an intermediate level between the application and the Internet Protocol (IP) layer. The TCP port component235may open or create a TCP port when communications that are latency tolerant (e.g., playback stored media) are to be transmitted over the wireless link to the sink device. Similarly, the UDP port component240may be utilized to transmit RTP media content over UDP from the source device115-cto the sink device. In some examples, UDP port component240may transmit messages and/or datagrams to at least one sink device on the direct wireless link during a Wi-Fi peer-to-peer remote display session. The UDP port component240may create or open a UDP port when latency intolerant communications, such as real-time streaming media, are to be transmitted across the wireless link to the sink device.

In yet further example, dynamic switching component245may switch transport protocol from a first protocol to a second protocol based in part on the capabilities of the sink device, type of media content, network conditions, and/or available buffer space as determined by the capability analysis component225-a. The first and second protocol may be either UDP and/or TCP. The capability analysis component225-amay be configured as previously described with respect toFIG. 2B. In one example, upon receiving a response from the sink device, the dynamic switching component245may initiate switching utilization of transport protocol from a first protocol to a second protocol by issuing an RTSP request message (e.g., RTSP SET_PARAMETER request) comprising profile and port information. In some instances, the first protocol may be UDP and the second protocol may be TCP. Alternatively, in other instances, the first protocol may be TCP and the second protocol may be UDP.

In one example, the source device115-c, upon receiving an affirmative response to the issued RTSP SET_PARAMETER request for setting up TCP transport for media data, may establish communication with the sink device. In some examples, the sink device may be configured as a TCP server, while the source device115-coperates as a TCP client. In one example, it may be necessary to change audio and/or video formats either before, during, and/or after switching from a first protocol to a second protocol, wherein the first or the second protocol may be either UDP or TCP.

In some examples, the dynamic switching component245may further request to allocate a specified amount of buffer space in the sink device prior to dynamically switching from the first protocol to the second protocol. The source device115-cmay include initial buffer size as a parameter to the RTSP SET_PARAMETER request message that it may send to the sink device. Allocation of a predetermined minimum buffer space may be necessary because Wi-Fi link between the source device115-cand the sink device may be jittery and/or error-prone. As a result, a predetermined minimum amount of buffering may be provided at the sink device to smooth the jitter and packet latency caused by errors in the channel (e.g., retransmissions of data, etc.) and to maintain good quality of video rendered at the sink device. In some examples, the sink buffer size may be dynamically controlled by the source device115-cbased on a type of application for a media stream being transmitted from the source device115-cto the sink device for presentation. For example, the techniques may select a buffer size that is smaller for gaming applications, larger for interactive media applications (e.g., interactive computing, presentations, bi-directional communication, etc.), and even larger for non-interactive media types (e.g., streaming video, static images, etc.).

Source device115-cmay further include user settings that allow the user to select preferences related to the amount of buffering at the sink device135. For example, the user may be able to choose a preference between lower latency or higher reliability display of the media stream and the source device115-cmay adjust the sink buffer size accordingly. The user may further be able to set sink buffer size preferences based on the use categories, for individual applications, or for individual media streams. In some examples, the user may be able to set the sink buffer size directly (e.g., in ms, etc.) prior to switching transmission from a first protocol to a second protocol.

In yet another example, a playback control component230-amay determine whether to issue control commands (e.g., flush buffer, volume control etc.) to the sink device. Flush control component250may determine whether to flush the buffer of the sink device based in part on a query requesting the current presentation time and amount of buffered data of the audio and/or video stream at the sink device. Table 1 illustrates one example of a message exchange between the source device115-cand the sink device with respect to a request for audio and video timing and buffer status.

Based at least in part on the response from the sink device, the flush control component250may flush the buffer of the sink device in order to allow the sink device to start buffering new data without playing previously buffered data. The flush control component250may issue a RTSP request message (e.g., RTSP SET_PARAMETER request) to sink device to flush the buffer of the sink device to a specified presentation time stamp (PTS) and/or decode time stamp (DTS) values. In some examples, the flush control component250may pause streaming of the media content prior to issuing the flush command to the sink device with a PTS value associated with the largest PTS transmitted from the source device115-cto the sink device. Upon receiving either a successful RTSP response or a failure message in response from the sink device, the source device115-cmay resume streaming the RTP data over either UDP or TCP as previously scheduled.

Similarly, a volume control component255of the playback control component230-amay issue volume control commands in order to adjust the volume of the media content at the sink device. The volume control commands may adjust the volume at the sink device irrespective of the volume level of the audio in the stream transmitted from the source device115-c. Some examples of the volume control may include volume up, volume down, mute and/or unmute.

Referring now toFIG. 3A, a block diagram300-aillustrates a sink device135-ain accordance with various embodiments. The sink device135-amay be an example of one or more aspects of one of the sink devices135described with reference toFIGS. 1, 2A, 2B, and 2C. The sink device135-bmay also be a processor. The sink device135-bmay include a sink receiver305, a communication establishment component310, and a sink transmitter315. Each of these components may be in communication with each other.

The components of the sink device135-amay, individually or collectively, be implemented with one or more ASICs adapted to perform some or all of the applicable functions in hardware. Alternatively, the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits. In other embodiments, other types of integrated circuits may be used (e.g., Structured/Platform ASICs, FPGAs, and other Semi-Custom ICs), which may be programmed in any manner known in the art. The functions of each unit may also be implemented, in whole or in part, with instructions stored in a memory, formatted to be executed by one or more general or application-specific processors. In one embodiment, the components as shown inFIG. 3Aeach may include a circuit or circuitry for performing the functions described herein.

The sink receiver305may receive communications from one or more source devices, such as the source devices115described with reference toFIGS. 1, 2A, 2B, and/or2C, via one or more signals302. As described herein, the communications may include audio and/or video streams, graphics resources, and/or rendering instructions. The sink receiver305may receive these communications via a wireless (e.g., Wi-Fi) peer-to-peer connection that is established between the sink device135-aand the one or more source devices115. The communication establishment component310may manage such communications received by the sink device135-avia one or more signals304. Additionally, the communication establishment component310may manage, via one or more signals306, communications transmitted from the sink device135-ato the source device(s). As described herein, these communications may include data representing user input at the sink device135-afor interacting with the source device(s) and/or one or more applications running on the source device(s). The sink transmitter315may transmit such data from the sink device135-avia the Wi-Fi connection, via one or more signals308. Further details regarding the communication establishment component310will be described below.

FIG. 3Bis a block diagram300-billustrating a sink device135-bin accordance with various embodiments. The sink device135-bmay be an example of one or more aspects of one of the sink devices135described with reference toFIGS. 1, 2A, 2B, 2C, and/or3A. The sink device135-bmay also be a processor. The sink device135-bmay include a sink receiver305-a,a communication establishment component310-a,and a sink transmitter315-a. Each of these components may be in communication with each other.

The components of the sink device135-bmay, individually or collectively, be implemented with one or more ASICs adapted to perform some or all of the applicable functions in hardware. Alternatively, the functions may be performed by one or more other processing units (or cores), on one or more integrated circuits. In other embodiments, other types of integrated circuits may be used (e.g., Structured/Platform ASICs, FPGAs, and other Semi-Custom ICs), which may be programmed in any manner known in the art. The functions of each unit may also be implemented, in whole or in part, with instructions stored in a memory, formatted to be executed by one or more general or application-specific processors. In one embodiment, the components as shown inFIG. 3Beach may include a circuit or circuitry for performing the functions described herein.

The sink receiver305-aand the sink transmitter315-amay be configured as previously described with reference toFIG. 3A. The communication establishment component310-amay include a dual-mode communication component320, a buffer management component325, a query response component330, and a playback control reception component335.

The dual-mode communication component320may be configured to decode media content received from the source device via either UDP or TCP transport protocol. In such instances, the dual-mode communication component320may identify its capabilities to the source device by providing indication to the query response component330to transmit a response to the source device that the sink device135-bis configured to receive media content over both UDP and TCP.

The buffer management component325may be configured to manage the buffer of the sink device135-b. In some examples, the source device may request to allocate a specified amount of buffer space in the sink device135-bto smooth the jitter and packet latency caused by errors in the channel (e.g., retransmissions of data, etc.) and to maintain good quality of video rendered at the sink device135-b. In some examples, the sink buffer size may be dynamically allocated by the source device based on a type of application for a media stream being transmitted from the source device to the sink device for presentation. As a result, the buffer management component325may be configured to buffer a predetermined amount of media content in the buffer prior to rendering the media content on the display device (not shown). The buffer management component325may further respond to flush commands issued by the source device. In such instances, the buffer management component325may flush the buffer to a specified PTS value. In some examples, the buffer management component325may further provide critical information to the query response component330in response to queries issued by the source device. Critical information may comprise initial buffer size, current presentation time of the audio and/or video stream being played, and available buffer space.

The query response component330may collect information from the dual-mode communication component320and buffer management component325to generate messages to the source device in response to issued queries requesting information from the sink device135-b. In yet further example, playback control reception component335may be configured to receive control information such as volume control to adjust the volume of the media content streamed to the sink device135-bfrom the source device. In response, the playback control reception component335may adjust the volume at the sink device irrespective of the volume level of the audio in the stream transmitted from the source device. In some examples, the volume control may include volume up, volume down, mute and/or unmute.

FIG. 4is a block diagram400illustrating a device115-daccording to various embodiments. The device115-dmay be an example of one or more aspects of one of the source devices115described with reference toFIGS. 1, 2A, 2B, and/or2C. The device115-dmay be configured to participate in Wi-Fi direct communications with other wireless devices (e.g., via a Wi-Fi peer-to-peer connection) to provide content for another device(s). The device115-dmay have any of various examples, such as personal computers (e.g., laptop computers, netbook computers, tablet computers, etc.), cellular telephones, PDAs, digital video recorders (DVRs), internet appliances, gaming consoles, e-readers, etc. The device115-dmay have an internal power supply (not shown), such as a small battery, to facilitate mobile operation.

The device115-dincludes antennas410, a transceiver415, memory425, and a processor435, which each may be in communication, directly or indirectly, with each other (e.g., via one or more buses455). The transceiver415is configured to communicate bi-directionally, via the antennas410, as described above. For example, the transceiver415may be configured to communicate bi-directionally with other devices115and/or135ofFIGS. 1, 2A, 2B, 2C, 3A, and/or3B. The transceiver415may include the receiver205and the transmitter215ofFIG. 2, as previously described. In one embodiment, the transceiver415may further include a modem configured to modulate packets and provide the modulated packets to the antennas410for transmission, and to demodulate packets received from the antennas410. While the device115-dmay include a single antenna, the device115-dwill typically include multiple antennas410for multiple links.

The memory425may include random access memory (RAM) and read-only memory (ROM). The memory425may store computer-readable, computer-executable software code430containing instructions that are configured to, when executed, cause the processor435to perform various functions described herein (e.g., identify/determine/obtain audio and/or video streams, graphics resources, and/or rendering instructions, receive, transmit, etc.). Alternatively, the software430may not be directly executable by the processor435but be configured to cause the computer (e.g., when compiled and executed) to perform functions described herein.

The processor435may include an intelligent hardware device, e.g., a central processing unit (CPU), a microcontroller, an ASIC, etc. According to the architecture ofFIG. 4, the device115-dmay further include a multi-protocol communication component220-b,a capability analysis component225-b,and a playback control component230-bas described with reference toFIGS. 2A and 2B. The multi-protocol communication component220-b,capability analysis component225-b,and playback control component230-bmay be a component of the device115-din communication with some or all of the other components of the device115-dvia a bus455.

FIG. 5is a message flow diagram500illustrating one example of communications between a source device115-eand a sink device135-c. The source device115-emay be an example of the devices115ofFIGS. 1, 2, and/or4. The sink device135-cmay be an example of the sink devices135illustrated inFIGS. 1 and/or 3. In one example, the source device115-eand the sink device135-cmay be connected via a Wi-Fi peer-to-peer connection. The source device115-emay have established communication with the sink device135-cutilizing UDP transport protocol. Alternately, it should be understood by those in the art that the default communication link between the source device115-eand sink device135-cis not limited to transmission of RTP media data over UDP, but may also alternatively include TCP transport protocol.

The source device115-e, upon receiving the capability information from the sink-device135-c, may reply with an RTSP “set parameter request”515to initiate switching transmission of media content from RTP over UDP to RTP over TCP. In some examples, the switching from the first transport protocol to the second transport protocol comprises an internal pause in the source and the sink device. The “set parameter request”515may comprise RTSP setup request with wfd-rtp-client-over-tcp-ports parameter containing the RTP/AVP/TCP profile and port information. The sink device135-c, in return, may execute a TCP server at the sink device135-cand return a RTSP “set parameter response”520indicating that the sink device135-chas already started its TCP server and is ready to accept connection from the source device115-e.

The source device115-e, upon receiving an affirmative “set parameter response”520may begin communicating with the sink device135-cby transmitting RTP media content over TCP525. When the RTP data is to be sent over TCP transport, each RTP packet data may be framed according to the IETF RFC 4571, “Framing Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) packets over Connection-Oriented Transport.” The sink-device135-cmay render the streamed media content530on the display of the sink-device135-c. Table 3 illustrates one example of the above illustrated message exchange between the source device115-eand the sink device135-cwith respect to switching transmission from UDP to TCP and/or TCP to UDP. It would be understood by those in the art that the illustrated examples are not limiting to the specified example.

In further examples, the source device115-e, based in part on the type of content and/or network conditions, may again seek to revert back to transmitting media content over UDP. In such an instance, the source device115-emay issue a “set parameter request”535to initiate the process of switching transport protocol from TCP to UDP. As a result, the sink device135-cmay respond with a “set parameter response”540to indicate that the sink device135-cis configured to switch from receiving RTP data over TCP to receiving RTP data over UDP. Consequently, the source device115-emay resume streaming media content over UDP545, which may be rendered and displayed550on the sink device135-c.

FIG. 6is a message flow diagram600illustrating another example of communications between a source device115-fand a sink device135-d. The source device115-fmay be an example of the devices115ofFIGS. 1, 2, 4 and/or 5. The sink device135-dmay be an example of the sink devices135illustrated inFIGS. 1 and/or 3. In one example, the source device115-fand the sink device135-dmay be connected via a Wi-Fi peer-to-peer connection. The source device115-fmay established communication with the sink device135-dutilizing either UDP or TCP transport protocol.

In some examples, the source device115-fmay issue playback control commands, such as flush command and volume control command to the sink device135-d. However, similar in the manner described with reference toFIG. 5, issuing of control commands are preceded with establishing capability negotiation procedures between source device115-fand sink device135-d. With reference toFIG. 6, the negotiation is based on RTSP “get parameter request”605and RTSP “get parameter response”610exchanged between the source device115-fand sink device135-drespectively. Thereafter, the source device115-fmay issue a flush command615identifying a PTS value of a data packet transmitted from the source device115-fto the sink device135-dto which the sink device135-dmay flush its respective buffer. Upon successful processing of the issued flush command, the sink device135-dmay respond affirming the response620. Consequently, the sink device135-dmay flush buffer625of the sink device135-dto the PTS value. Table 4 illustrates one example of flush command exchange between the source device115-fand the sink device135-d.

Similarly, the source device115-fmay issue a volume control commands630to the sink device135-dto adjust the volume of the data stream to the sink device135-d. Upon successful processing of the issued volume control command630, the sink device135-dmay respond affirming the response635. Consequently, the sink device135-dmay adjust or update the volume of the media content640at the sink device135-d. The volume at the sink device135-dmay be adjusted or updated irrespective of the volume level of the audio in the stream transmitted from the source device115-f. Some examples of the volume control may include volume up, volume down, mute and/or unmute. Table 5 illustrates one example of volume control exchange between the source device115-fand sink device135-dinitiated by the volume control component255of the source device115as described with reference toFIG. 2C.

FIG. 7illustrates a method700performed by a source device115for wireless communication with the sink device135. For clarity, the method700is described below with reference to the system100shown inFIG. 1, and/or with reference to one of the devices115described with reference toFIGS. 1, 2A, 2B, 2C, 4, 5 and/or 6. In one implementation, the communication management component210described with reference toFIGS. 2A, 2B, and/or2C may execute one or more sets of codes to control the functional elements of a source device115to perform the functions described below.

At block705of method700, the source device115initiates a media stream to a sink device over a first protocol during a Wi-Fi peer-to-peer remote display session. The operation(s) of block705may be performed by the communication management component210and/or multi-protocol communication component220as described with reference toFIGS. 2A, 2B and/or 2C.

At block710of method700, the source device115may identify a second protocol supported by the sink device135. The operation(s) of block710may be performed by the capability analysis component225as described with reference toFIGS. 2B and/or 2C. At block715, the source device115may dynamically switch to the second protocol to transmit the media stream during the Wi-Fi peer-to-peer remote display session. The operation(s) of block715may be performed by the dynamic switching component245as described with reference toFIG. 2C.

FIG. 8illustrates a method800performed by a source device115for wireless communication with the sink device135. For clarity, the method800is described below with reference to the system100shown inFIG. 1, and/or with reference to one of the devices115described with reference toFIGS. 1, 2A, 2B, 2C, 4, 5 and/or 6. In one implementation, the communication management component210described with reference toFIGS. 2A, 2B, and/or2C may execute one or more sets of codes to control the functional elements of a source device115to perform the functions described below.

At block805of method800, the source device115initiates a media stream to a sink device over a first protocol during a Wi-Fi peer-to-peer remote display session. The operation(s) of block805may be performed by the communication management component210and/or multi-protocol communication component220as described with reference toFIGS. 2A, 2B and/or 2C.

At block810of method800, the source device115may transmit a query to the sink device to determine whether the sink device can support TCP transport protocol, along with port information to be used for a TCP connection. The operation(s) of block810may be performed by the capability analysis component225as described with reference toFIGS. 2B and/or 2C. At block815, the source device115may receive a message, in response to the query, affirming that the sink device supports TCP, and associated TCP ports the sink device will be listening to. The operation(s) of block815may be performed by the source receiver205as described with reference toFIG. 2A, 2B, and/or2C.

At block820of method800, the source device115may dynamically switch to the second protocol to transmit the media stream during the Wi-Fi peer-to-peer remote display session. The operation(s) of block820may be performed by the dynamic switching component245as described with reference toFIG. 2C.

In embodiments, the components as shown inFIGS. 2A, 2B, 2C, 3A, 3B, and 4, each include a circuit or circuitry for performing the functions described herein with reference toFIGS. 2A, 2B, 2C, 3A, 3B, and 4.