Media relay

A method, system and computer program product for displaying media is provided herein. The method includes the steps of receiving, at a media server, streaming media from a source device located at a first user's premises and transmitting the streaming media from the media server to a display device at the second user's premises. The media server is located remotely from both the first user's premises and the second user's premises. The first user's premises and the second user's premises may be the same. The media server may be located at a cable television headend.

BACKGROUND OF THE INVENTION

Field of the Invention

The invention generally relates to the display of the output of a computational device, including the display of internet-based media, on a television.

Background Art

Users wishing to display the output of a computational device, such as a personal computer or laptop, on a television screen typically cannot do so without physically connecting the two via a display cable. Typically, there are multiple impediments to doing such. Users must first acquire a suitable cable. They must possess sufficient familiarity with their computational device and their television to acquire the correct cable, connect it correctly to both devices, and to be able to switch inputs on the television, as needed. If the computational device is already connected to a display, there may not be a suitable output connector available. If the television is already connected to multiple signal sources, there may not a suitable input connector available.

What is needed is a method and a system to overcome these limitations, enabling users to display the output of computational devices, including internet-based media, on their televisions, while avoiding the cost and complexity of existing systems and methods.

The present invention will now be described with reference to the accompanying drawings. In the drawings, like reference numbers may indicate identical or functionally similar elements.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments presented herein allow a source device to relay media through a media server to a display device. In an example, the source device and the display device are located in the same premises and the media server is located remote to the source device and the display device. In another example, the source device and the display device are remotely located from each other and from the media server. “Media” as referred to herein may be streaming media such as video or audio. However, it is to be appreciated that “media” may encompass any item on a source device including but not limited to images and documents of any format.

In an example, the source device may be located behind a network address translation (NAT) device. In another example both source device and the media server may be located behind a NAT device. It is to be appreciated that either the source device, the media server or both may be located behind a NAT device. It is also to be appreciated that in embodiments, either the source device or the media server or neither may be located behind a NAT device. A NAT device, may be, for example, a router, firewall or other computational device. A NAT device converts private Internet Protocol (IP) addresses of a client computational device (such as the source device or media server) that is behind it on an internal private network to one or more public IP addresses for the Internet. It may change packet headers to the converted IP address and keep track of the headers via internal tables. When packets come back from the Internet, the NAT device may use the tables to perform a reverse conversion to the IP address of the client computational device. NAT allows a single device, such as a router, to act as agent between the Internet (or “public network”) and a local (or “private”) network. In an embodiment, media is transmitted by the source device via the Internet and is received by the display device via a cable television network. The media server may be located at a cable television headend. The display device may be a television (TV). However, other display devices are contemplated. The computational device may be a user's personal computational device such as a personal computer or a laptop.

In an embodiment presented herein, media on a user's personal computational device can be viewed via the cable television network on the user's TV without directly connecting the computational device to the TV. For example, a user may launch an application on their computational device which streams the desired media to a media server that may be located at a television cable headend remote from the user's premises. The user may then tune to a particular television channel, such as a channel on a cable set-top box, dedicated for media relay. The cable television headend may transcode the received media and transmit the media via a cable television network to the user's set top box. The set top box displays the received media on the user's TV.

Embodiments presented herein further allow communication with systems behind a NAT, without using traditional NAT traversal techniques such as Transmission Control Protocol (TCP) hole punching, Universal Plug and Play (UPnP), port forwarding or a proxy server. Traditional NAT traversal techniques may require a manual setup step by a user to create a port-forwarding entry in the user's NAT to enable outside access to the streaming source device. This port-forwarding has two drawbacks. First, because of the manual setup by the user it is a potential source of failure resulting in customer support requests due to the level of complexity and knowledge required for the manual setup. Second, it is a potential security issue because it creates an outside accessible port to the source device that could be hacked by a malicious third party client. Simple solutions such as a proxy server, TCP hole-punching or UPnP are either not scalable or are not without problems on the client side.

FIG. 1illustrates an example media relay system100according to an embodiment of the invention.

Media relay system100includes a source device102, media server104, authorization server110, rendezvous server112, set top box116, and display device118. In an example, media server104may be located behind a NAT device108and source device102may be located behind a NAT device106. In another example, only media server104may be located behind NAT device108. NAT device106and NAT device108, may be, for example, routers. Source device102, set top box116, media server104, authorization server110and rendezvous server112are coupled to a network114.

In an embodiment, source device102transmits media to media server104via a real time streaming protocol (RTSP) and media server104relays media to set top box116over a cable television network via a QAM- (quadrature amplitude modulation) modulated MPEG-2 transport stream. In another embodiment, source device102transmits media to media server104via a real time protocol (RTP). Although the embodiments presented herein may use RTSP or RTP, it is to be appreciated that other protocols may be used to transmit and receive media. In an embodiment, network114may include a hybrid fiber coaxial (HFC) cable television network or a fiber to the premises (FTTP) network. In an embodiment, network114may also include the Internet. It is to be appreciated that the type of network is a design choice, and embodiments of the invention presented herein are not dependent on the type of network used.

Source device102may include a processor120coupled to a memory122. Processor120may perform the functions described therein as being performed by source device102based on instructions stored in memory122. Source device102may also include a personal computation application (PC app)124and a streamer126. Functions performed by PC app124and streamer126are described in further detail below with respect toFIG. 2. In an embodiment, PC app124and streamer126are embodied in hardware. In another embodiment, PC app124and streamer126may be software that run on processor120based on instructions stored in memory122. Display device118may be, for example, a TV or any other form of display device, such as, an Apple iPad™ or a mobile communication device, such as a smart phone.

Set top box116may include a processor128coupled to a memory130. Processor128may perform the functions described herein as being performed by set top box116based on instructions stored in memory130.

Media server104may include a processor128coupled to a memory130. Steps described herein are being performed by media server104may be performed by processor128based on instructions stored in memory130. Media server104may also include a transcoder132, a stitcher134, an Active Video Markup Language application (AVML app)136. Embodiments presented herein may use AVML, but it is to be appreciated that other languages or protocols may be used to implement the functions performed by the AVML app136. In an example, transcoder132, stitcher134and AVML app136may be implemented solely in hardware. In another embodiment, transcoder132, stitcher134and AVML app136may be implemented in hardware, software, firmware or any combination thereof. In another example, transcoder132, stitcher134and AVML app136may be implemented as software running on processor128based on instructions stored in memory130.

Authentication, authorization and accounting server (AAA)110may include a processor138coupled to a memory130. The functions described herein as being performed by authorization server110may be performed by a processor138based on instructions stored in memory140. Rendezvous server112may include a processor142coupled to a memory144. The functions performed by rendezvous server112as described herein may be performed by processor142based on instructions stored in memory144.

According to an embodiment of the invention, to view media stored or streamed by source device102on display device118, a user initiates the PC application124. In response to the user input, source device102authenticates itself with AAA server110. In response to being authenticated by AAA server, source device102, opens a transmission control protocol (TCP) session with rendezvous server112. In an embodiment, source device102may use AAA server to authenticate itself to rendezvous server112.

To view the media streamed by source device102on display device118, the user tunes to a dedicated media relay channel using set top box116. In response to the user tuning to the media relay channel, the set top box116transmits a signal to media server104to open a TCP connection with rendezvous server112. Rendezvous server112, receives a signal from media server102that indicates the media server102is ready to receive media from source device102. Upon receiving the signal from media server102, rendezvous server112transmits a connection address for media server104to source device102. Rendezvous server also notifies the media server104of a connection address for source device102. In an example, the connection address is an Internet Protocol (IP) address and a port number. It is to be appreciated that the type of connection address and protocol used is a design choice. After both the source device102and the media server104have been provided each other's connection address, rendezvous server112closes the connection with the source device102and media server104closes the connection with rendezvous server112.

Source device102, based on the connection address for media server104, connects to media server104and requests the start of an RTSP session by media server104, to stream media to media server104. In an alternate embodiment, the source device102may start the RTSP session. In response to the request, media server104grants the source device102permission to transmit media to the media server104. Streamer126streams media to the media server104. The streamed media is relayed by media server to set top box116. The relayed media may then be viewed on the on the media relay channel via display device118. The streamed media may be optionally transcoded by transcoder132prior to transmission to set top box116. Alternatively, media may be transcoded by streamer126prior to streaming the media to media server104. The steps performed by source device102, authorization110, rendezvous server112, media server104, and set top box116are further described below with respect toFIG. 2

FIG. 2illustrates an example flow diagram200that illustrates a media relay process according to an embodiment of the invention.

PC application124authenticates source device102with AAA server110using authentication message202. It is to be appreciated that any type of authentication may be used including but not limited to Rivest-Shamir-Adleman (RSA), Digital Signature Algorithm (DSA) and Secure Hash Algorithm (SHA).

AAA server110sends an authorization message204to source device102. The authorization message204may include a Transmission Control Protocol (TCP) address (tcp_ip) and a port address (pc_port).

In response to receiving authorization message204from AAA server110, PC application124sends a start streamer command206to streamer126with an IP address (rv_ip) for rendezvous server112, a port address (rv_port) for rendezvous server112and a session key (skey).

In response to the start streamer command206, streamer126opens a TCP connection with rendezvous server112using an open TCP connection command208.

Streamer126authenticates itself with rendezvous server112using an authenticate command210that includes the session key (skey). Upon successful authentication, rendezvous server112open a TCP connection with streamer126and keeps the TCP connection with streamer126open.

In response to a user tuning to a media relay channel, set top box116sends a start application command212to stitcher134that is located in media server104.

In response to the start application command212, stitcher134sends a load command214to AVML application136. In response to the load command214, AVML application136sends a Get PC command216to AAA server110. The Get PC command216includes a Media Access Control (MAC) address of the set top box116and a home ID that identifies the set top box116to AAA server110.

In response to the Get PC command216, AAA server110transmits a stream PC command218to AVML application136. The steam PC command18may include the IP address of the rendezvous server112(rv_ip), the port address of the rendezvous server112(rv_port), the session key (skey) and an alias.

In response to receiving the steam PC command218, AVML App136transmits a transcode command220to stitcher134that includes the rv_ip, rv_port and the skey. In response to a transcode command220, stitcher134sends a RTSP session start command222to transcoder132that includes the rv_ip, rv_port and the skey.

Transcoder132sends an open TCP connection command224to rendezvous server based on rv_ip and rv_port which opens a TCP connection with the rendezvous server112. Transcoder132also authenticates itself to rendezvous server112using the authenticate command226and session key (skey) received from stitcher134.

In response to opening the TCP connection with transcoder132, rendezvous server sends a notify client command228to streamer126. The notify client command228includes an IP address (tcn_ip) of transcoder132and a port address (tcn_port) of transcoder132. Rendezvous server112also transmits a notify listener command230to transcoder132that includes an IP address (pcn_ip) of media source102and a port address (pcn_port) of the media source102. Based on the notify listener command230, transcoder places the source device102in a queue. The queue may be a first in first out (FIFO) queue or it may be a queue based on programmable priority levels.

By providing media source102with the IP address and port address of transcoder132and by providing the IP address and port address of source device102, rendezvous server112bridges source device102and media server104which may be behind NAT device106and NAT device108. This allow source device102to directly connect to media server104without any further assistance from rendezvous server112. Rendezvous server112closes the TCP connection with streamer126using command234and closes the TCP connection with transcoder132using command236.

Streamer126sends an open TCP connection command236to transcoder132with pcn_ip and pcn_port as parameters. When it is the turn of source device102in the queue, transcoder132transmits a RTSP session start command238that commands streamer126to stream media to transcoder132. In response to receiving the RTSP session start command238, streamer126streams media240to transcoder132. Transcoder132transcodes media240and sends a transcoded stream242to stitcher134. Stitcher134transmits a session stream244through network114(that may include a cable television network) to set top box116for display on the display device118.

Thus, embodiments of the invention allow a user to relay media form a source device102to a display device118via a media server104. Furthermore, in the embodiments presented above, rendezvous server112acts as a intermediary or a bridge to couple source device102to media server104, one or both of which may be behind a NAT device. Typical NAT traversal techniques require an intermediary server to be active throughout the communication process thereby creating significant delays due to bottlenecks created by the intermediary server. In contrast to typical NAT traversal techniques, there is no further intervention by rendezvous server112once it has facilitated a connection between source device102to media server104.

Example Implementation

An example implementation of the media relay system100is described below. It is to be appreciated that the implementation is for example purposes only and does not limit the scope of the embodiments presented herein.

In an example, a main communication element from AAA server10to AVML APP136is a “pcinfo” XML document. This XML structure is augmented with an additional element called “rendezvous” that includes the IP address and port address of the rendezvous server as attributes. A “pc” element may include an additional attribute called “skey” that transmits the session key that is used for match making in the rendezvous server112. This attribute is mutual exclusive with an “ip address”/“port” attribute pair that is used, if no reversed connect is required. Example XML code is provided below:

Optional Universal Resource Locator (URL) code is provided below:

In an example, the rendezvous server112has to send keep-alive messages to the AAA server110which maintains a list of active rendezvous server112. These messages have to include three data elements: (1) a public IP address and port that has to be used by streamer126; (2) NATTP address and port for NAT device108that will be used by the transcoder132; and (3) A site identification that creates an association between a source device102and rendezvous server112.

In an example, one rendezvous server is used for all sites and it has a single IP address that is available for both the source device102and the transcoder132.

The AAA server110may verify the IP of the source device102based on a pool of valid IP addresses, to avoid accepting false keep-alive messages from malicious third parties that might result in invalid entries in rendezvous server112.

In an example rendezvous server112is not behind NAT device108and may use the URL:

Example Stitcher to Transcoder Communication

The stitcher134may provide the IP address and port of source device102to transcoder123as media resource information. The rendezvous server initiated process sends the rendezvous server IP address and port address as well as the session key to the transcoder132. This can be implemented using a combined “rdvs://” URL scheme instead of a “rtsp://” URL. The transcoder can then derive the connection mode from the URL protocol prefix.

Example PC-App to Streamer Communication

The PC-App124provides all required configuration values to the streamer126as command line arguments. The additional information required for the reversed connection process is thus provided as additional command line arguments. Some arguments are not used in the case of a rendezvous server112initiated connection such as:

(1) listen_port—the port the streamer126should listen for incoming streaming connections on. This argument is not needed in case of a rendezvous server112initiated connection because it is derived from the TCP connection to the rendezvous server112.

(2) connect_url—The rendezvous server URL including the session key using the “rdvs://” protocol prefix.

The streamer126can decide, based on the presence of a connect URL, whether to expose a port directly or negotiate a connection via the rendezvous server112. The validate message might not be needed in case of a rendezvous initiated connection.

Example Rendezvous Server Protocol

The rendezvous server112may use a simple text based protocol that includes the following messages:

The challenge message is sent by the rendezvous server112to a connecting source device102and may include a partially random challenge key that is encrypted with the server key of an RSA key pair. This key is decrypted and re-encrypted by the client (streamer126or transcoder132) using the other key of an RSA key pair. The source device102may verify the validity of a non-random part of the challenge key. This re-encrypted key may be sent back by the client as an authenticate key, together with a session key that is used for match making. The rendezvous server112uses its key of the RSA key pair to verify the authenticity of source device102using the original challenge key and the authenticate key. If it considers the source device102to be valid, it decrypts the session key and then attempts to find the peel of this connection in its session dictionary. If this is the first peer, it stores this connection into the session dictionary.

In another example, a less complex authentication procedure may use a salted SHA1 hash. The challenge key is a random number/string generated by the rendezvous server. The response key is the SHA1 hash generated from this random number with a pre-shared fix salt appended to this key.

Rendezvous server112may notify both source device102and media server104when a match is found by sending a notify message, that includes the public IP address and public port number of the opposite NATs. The communication from transcoder132to rendezvous server112could be as follows:

The public IP address is not required if the same IP is used for inbound traffic that is used to connect to the rendezvous server. Streamer126does not need to send its public IP and port since it can be derived from the connection. An example connection message is as follows:

The “challengeKey” and “authenticateKey” may be base64encoded binary keys, the “sessionKey” could be encrypted with a decrypted “challengeKey” which is available to rendezvous server112, streamer126and transcoder132.

Example Security Provisions

To prevent overload or crash of rendezvous server112, a third party attack such as a Denial of Service (DOS) or an attempt to compromise the rendezvous server112and/or source device102, multiple rendezvous servers112may be implemented in a cluster format. In an example, the cluster may include AAA server110. AAA server110provides the IP address of the rendezvous server112to both source device102and media server104and may perform a round robin mechanism when selecting a rendezvous server112. Rendezvous servers112may send keep-alive messages to the AAA server110in fixed time distances. If a keep-alive message is not be received by the AAA server110after three such intervals, the rendezvous server112would be considered dead and taken off the list.

A source device102that is unable to communicate with its rendezvous server112may either fail with a retry option (and thus getting a fresh rendezvous server112) or a sorted list of rendezvous servers112may be used instead of a single one, giving the source device102an option for automatic retry.

The rendezvous server112has to keep a TCP connection from source device102alive until media server104successfully connects. This may consume valuable resources such as ephemeral sockets. Clustering over several physical or virtual machines reduces the risk of running out of resources. A watchdog timer may be used to cancel sessions that wait for an extended time for their peer, e.g. the customer starts the service on the source device102but fails to start tune to a media relay channel using STB116. The watchdog time may cover a typical delay between start of PC App124and start of AVML app136. Considering that there are 30,000 ephemeral sockets on a typical server and a timeout value of 15 minutes, one server could handle a connection rate of approximately 33 session setups per second, if all connections time out. With an average delay of 5 minutes, one server would achieve approximately 100 session setups per second.

The most important defense against a denial of service attack is an early termination of an attacking connection. This has two major requirements, first detecting the attacking connection attempt and second doing this quickly. The use of a challenge and authentication key may allow rendezvous server112to detect unauthorized requests. To do this quickly rendezvous server112may use a smaller watchdog timer for the period between the challenge-request and authentication-response than the normal session timeout. Another option would be the use of signing the session keys by AAA server110. A concatenation of the source device102public IP address (available to AAA server110through a TCP connection) and the session key could be signed with the private key of the AAA server110. The rendezvous server112could then validate the request of the streamer by verifying the signature with the public key.

The transcoder132may know an IP address (but not the port) of the source device102through notification by the rendezvous server112, and is therefore able to limit the range of potential malicious attackers early, or deny it before data is received and interpreted.

The transcoder132may keep a listening port open that accepts the incoming stream connections and knows the public IP address and port pair (or port only, if it is a 1.1 port mapping NAT forward and the rendezvous server112is outside of NAT device108) that this port will be represented in the open internet. This acceptor is global to the transcoder process, thus implemented as a singleton, and maintains a dictionary of expected connections. The rendezvous handshaking sequence will insert the session key or the public IP address of the expected streamer126into this dictionary, and wait for the connection to happen. It is not possible to wait for the incoming connection directly because several connection attempts from different streamers may be pending at the same time. Therefore it is necessary to maintain a dictionary of pending connections.

The streamer126may use an instance of an RTSP Server which uses an Tcp Acceptor to open a listening port. Assuming that there is always only one connection to this RTSP Server active at any time, it is possible to derive from the RTSP Server and replace the acceptor related connection with a direct connect call after a synchronous connect and communication with the rendezvous server. The RTSP Server receives an input parameter “struct” that could transfer the rendezvous server address and the session key.

Example AAA Server

The AAA server110may send an XML file to the PC application124and AVML application136that includes an IP address of rendezvous server112and a one-time session key. AAA server110may maintain a list of active rendezvous servers112per site based on the received keep-alive messaged.

Example Rendezvous Server

The rendezvous server112may run on Linux as well as Windows™, because the physical server has not been determined yet and may have to be shared with other services. The services provided by the rendezvous server112may include: (1) announce availability to AAA server110using a keep alive-message; (2) accept incoming connections from streamer126and transcoder132; (3) provide challenge keys and validate response keys; (4) store pending connections in dictionary; (5) match connections using the dictionary; and (6) provide response to participating source device102.

Rendezvous Server Client Protocol Handler

A rendezvous server protocol is used by the streamer126and the transcoder132. The implementation is based on a ucl::ConnectionPoint/ucl::Protocol class family. Each message ends with an Embedded Open Type (EOT) character that can be used to split the incoming data stream into individual messages.

Embodiments presented herein, or portions thereof, can be implemented in hardware, firmware, software, and/or combinations thereof. The embodiments presented herein apply to any communication system that utilizes packets for data transmission.

The representative media relay functions described herein can be implemented using one or more of computer processors, such as one or more of processors120,129,138,142and128, computer logic, application specific circuits (ASIC), digital signal processors, etc., or any combination thereof, as will be understood by those skilled in the arts based on the discussion given herein. Accordingly, any processor that performs the functions described herein is within the scope and spirit of the embodiments presented herein.

Further, the media relay functions described herein could be embodied by computer program instructions that are executed by a computer processor, for example one or more of processor120,129,138,142and128, or any one of the hardware devices listed above. The computer program instructions cause the processor to perform the instructions described herein. The computer program instructions (e.g. software) can be stored in a computer usable medium, computer program medium, or any storage medium that can be accessed by a computer or processor. Such media include a memory device, such as memory122,140,144,131and130, a RAM or ROM, or other type of computer storage medium such as a computer disk or CD ROM, or the equivalent. Accordingly, any computer storage medium having computer program code that cause a processor to perform the functions described herein are within the scope and spirit of the embodiments presented herein.

CONCLUSION

The embodiments presented herein have been described above with the aid of functional building blocks and method steps illustrating the performance of specified functions and relationships thereof. The boundaries of these functional building blocks and method steps have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. Any such alternate boundaries are thus within the scope and spirit of the claimed embodiments. One skilled in the art will recognize that these functional building blocks can be implemented by discrete components, application specific integrated circuits, processors executing appropriate software and the like or any combination thereof. Thus, the breadth and scope of the present embodiments should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.