System and method for providing real time connectionless communication of media data through a firewall

A method for communicating real time media data between a first client and a second client across a packet switched data network is provided. The method includes receiving an indication of a first client network address for use as a destination network address for sending media datagrams to the first client. A media datagram originated by the first client is also received. A comparison between the first client network address and a source network address extracted from the media datagram originated by the first client is made. A media datagram is sent to the first client using the source network address if the source network address and the first client network receiving address are not the same and the media datagram is sent to the first client using the first client network address if the source network address and the first client network address are the same.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation in part of U.S. patent application Ser. No. 09/788,865 entitled Method for Communicating Audio Data in a Packet Switched Network filed on Feb. 20, 2001 and is a continuation in part of U.S. patent application Ser. No. 09/819,492 entitled System and Method for Determining a Connectionless Communication Path for Communicating Audio Data through an Address and Port Translation Device filed on Mar. 28, 2001. Both of the above referenced patent application are incorporated herein in their entirety.

TECHNICAL FIELD

The present invention relates to communicating media data in a packet switched data network and, more specifically, to establishing and maintaining real time media data sessions through a firewall.

BACKGROUND OF THE INVENTION

For many years voice telephone service was implemented over a circuit switched network commonly known as the public switched telephone network (PSTN) and controlled by a local telephone service provider. In such systems, the analog electrical signals representing the conversation are transmitted between the two telephone handsets on a dedicated twisted-pair-copper-wire circuit. More specifically, each telephone handset is coupled to a local switching station on a dedicated pair of copper wires known as a subscriber loop. When a telephone call is placed, the circuit is completed by dynamically coupling each subscriber loop to a dedicated pair of copper wires between the two switching stations.

More recently, the copper wires, or trunk lines between switching stations have been replaced with fiber optic cables. A computing device digitizes the analog signals and formats the digitized data into frames such that multiple conversations can be transmitted simultaneously on the same fiber. At the receiving end, a computing device reforms the analog signals for transmission on copper wires. Twisted pair copper wires of the subscriber loop are still used to couple the telephone handset to the local switching station.

More recently yet, voice telephone service has been implemented over the Internet. Advances in the speed of Internet data transmissions and Internet bandwidth have made it possible for telephone conversations to be communicated using the Internet's packet switched architecture and the TCP/IP protocol.

Software is available for use on personal computers which enable the two-way transfer of real-time voice information via an Internet data link between two personal computers (each of which is referred to as an end point or client), each end point computer includes appropriate hardware for driving a microphone and a speaker. Each end point operates simultaneously both as a sender of real time voice data and as a receiver of real time voice data to support a full duplex voice conversation. As a sender of real time voice data, the end point computer converts voice signals from analog format, as detected by the microphone, to digital format. The software then facilitates data compression down to a rate compatible with the end point computer's data connection to an Internet Service Provider (ISP) and facilitates encapsulation of the digitized and compressed voice data into a frame compatible with the user datagram protocol and internet protocol (UDP/IP) to enable communication to the other end point via the Internet.

As a receiver of real time voice data, the end point computer and software reverse the process to recover the analog voice information for presentation to the operator via the speaker associated with the receiving computer.

To promote the wide spread use of Internet telephony, the International Telephony Union (ITU) had developed a set of standards for Internet telephony. The ITU Q.931 standard relates to call signaling and set up, the ITU H.245 standard provides for negotiation of channel usage and compression capabilities between the two endpoints, and the ITU H.323 standard provides for real time voice data between the two end points to occur utilizing UDP/IP to deliver the real time voice data.

Additionally, the Internet Engineering Task Force (IETF) has developed a set of standards for initiating real time media data sessions known as the Session Initiation Protocol (SIP). SIP provides for UDP/IP messages to be exchanged between the two endpoints (or between the two endpoints and multiple proxy servers) to provide for call signaling and negotiation of compression capabilities.

A problem associated with standard ITU Internet telephony and with SIP Internet telephony is that network address translation (NAT) firewalls prevent the transmission of UDP/IP frames from an endpoint computer outside the firewall to an endpoint computer on a private network inside the firewall.

More specifically, both the ITU Internet telephony standards and the SIP standards provide for each endpoint to designate a real time protocol (RTP) channel, which comprises and IP address and port number, for receipt of media datagrams and to provide that RTP channel to the other end point.

Because the private network client does not have a globally unique IP address, a frame sent to such non-globally unique IP address can not be routed on the Internet and will be lost. Further, even if the private network client were able to identify and designate the IP address of the NAT firewall server, the private network client has no means for establishing a port on the NAT firewall for receipt of media datagrams.

Because of the wide spread use of NAT firewalls which typically provide both IP address translation and port translation of all frames sent from the private network to the Internet, what is needed is a system and method for establishing and maintaining Internet telephony conversations between two clients, both of which are located on private networks behind NAT firewalls. What is also needed is a system and method for establishing and maintaining Internet telephone conversations between a client located on a private network behind a NAT firewall and a client with an Internet routable IP address (e.g. public IP address on the Internet) that operates a receiving UDP channel that is different from its sending UDP channel.

SUMMARY OF THE INVENTION

A first aspect of the present invention is to provide a method of communicating real time media data between a first client and a second client. Both the first client and the second client may be located on separate private networks behind NAT firewalls and both may use the same UDP channel for both sending and receiving media datagrams. The method may be performed by an intermediary server with a public IP address on the Internet. The method comprises extracting a first client source network address from a first media datagram originated by the first client and extracting a second client source network address from a second media datagram originated by the second client. A third media datagram, that includes media data received from the second client, is sent to the first client source network address and a fourth media datagram, that includes media data sent by the first client, is sent to the second client source network address.

The first client source network address may comprise an Internet Protocol address and a port number of a firewall server supporting the first client and the second client source network address may comprise an Internet Protocol address and a port number of a firewall server supporting the second client. As such, the step of sending a third media datagram to the first client source network address may include sending the third media datagram to the Internet Protocol address and port number extracted from the first media datagram and the step of sending a fourth media datagram to the second client source network address may include sending the fourth media datagram to the Internet Protocol address and port number extracted from the second media datagram.

The method may further include establishing a first port number for receipt of the first media datagram and establishing a second port number for receipt of the second media datagram. The first port number and the second port number may be the same. An indication of the first port number may be provided to the first client and an indication of the second port number may be provided to the second client. The third media datagram may include the first port number as a source port number and the fourth media datagram may include the second port number as a source port number.

A second aspect of the present invention is to provide a method of communicating real time media data between a first client and a second client. Either (or both) of the first client and the second client may be behind firewall servers. Additionally, if either (or both) of the first client and the second client are not behind a firewall server, such client may use the same network address and port number (real time protocol channel) for both sending and receiving or may use separate real time protocol (RTP) channels for sending and receiving. The method of the second aspect will work with all such permutations.

The method comprises receiving an indication of a first client network address which includes an Internet Protocol address and port number for receiving media datagrams, receiving a first media data gram originated by the first client, and receiving a second media data originated by the second client. The first client network address may be compared to a first client source network address extracted from the first media datagram. If the Internet Protocol address of the first client network address is not the same as an Internet Protocol address from the network address extracted from the first media datagram, it can be concluded that the first client is behind a firewall server and that the network address extracted from the first media datagram is that of the firewall server. As such, a third media datagram, that includes media data originated by the second client, is sent to the first client using the first client source network address as a destination network address of the third media datagram.

Additionally, if the Internet Protocol address of the first client network address is the same as the Internet Protocol address from the network address extracted from the first media datagram, it can be concluded that the first client is not behind a firewall server and has a public IP address. As such, the method may further include sending a third media datagram, that includes media data originated by the second client, to the first client using the first client network address if the source network address and the first client network address are the same.

The method may also include establishing a first port number for receipt of the first media datagram. An indication of the first port number may be provided to the first client. The third media datagram may include the first port number as a source port number.

A third aspect of the present invention is to provide a device for relaying real time media data between a first client and a second client, both of which may be behind firewall servers. The device comprises a network interface circuit for communicating with each of the first client and the second client via a data network and a media communication application operatively coupled to the network interface circuit. The media communication application provides for extracting a first client source network address (Internet Protocol address and port number) from a media datagram originated by the first client and extracting a second client source network address (Internet Protocol address and port number) from a media datagram originated by the second client, both of which may be received by the network interface circuit. The media communication application further provides for driving the network interface circuit to send a third media datagram, that includes media data received from the second client, to the first client source network address and to send a fourth media datagram, that include media data received from the first client, to the second client source network address.

The first client source network address may comprise an Internet Protocol address of a firewall server supporting the first client and the second client source network address may comprise an Internet Protocol address of a firewall server supporting the second client.

The media communication application may further provide for establishing a first port number for receipt of the first media datagram and establishing a second port number for receipt of the second media datagram. The first port number and the second port number may be the same. The media communication application may drive the network interface circuit to provide an indication of the first port number to the first client and an indication of the second port number to the second client. The third media datagram may include the first port number as a source port number and the fourth media datagram may include the second port number as a source port number.

A fourth aspect of the present invention is to provide a device for relaying real time media data between a first client and a second client, independent of whether either or both of the first client and the second client are behind firewall servers, and, if located directly on the Internet, independent of whether such client uses the same RTP channel for both sending and receiving media datagrams. The device comprises a network interface circuit for communicating with each of the first client and the second client via a data network and a media communication application operatively coupled to the network interface circuit. The media communication application provides for: i) obtaining a first media datagram originated by the first client and received by the network interface circuit; ii) obtaining an indication of a first client network address that includes an Internet Protocol address and port number for use as a destination network address for sending media datagrams to the first client; iii) obtaining a second media datagram originated by the second client and received by the network interface circuit; iv) comparing the first client network address to a first client source network address extracted from the first media datagram; and v) driving the network interface circuit to send a third media datagram that includes media data originated by the second client using the first client source network address as a destination network address of the third media datagram if the Internet Protocol address of the first client network address and the Internet Protocol address of the first client source network address are not the same.

Further, the media communication application may provide for driving the network interface circuit to send a third media datagram that includes media data originated by the second client using the first client network address as a destination network address of the third media datagram if the Internet Protocol address of the first client network address and the Internet Protocol address of the source network address are the same.

The media communication application may further provide for establishing a first port number for receipt of the first media datagram and may drive the network interface circuit to provide an indication of the first port number to the first client. The third media datagram may include the first port number as a source port number.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

The present invention will now be described in detail with reference to the drawings. In the drawings, each element with a reference number is similar to other elements with the same reference number independent of any letter designation following the reference number. In the text, a reference number with a specific letter designation following the reference number refers to the specific element with the number and letter designation and a reference number without a specific letter designation refers to all elements with the same reference number independent of any letter designation following the reference number in the drawings.

It should also be appreciated that many of the elements discussed in this specification may be implemented in a hardware circuit(s), a processor executing software code, or a combination of a hardware circuit(s) and a processor or control block of an integrated circuit executing machine readable code. As such, the term circuit as used throughout this specification is intended to encompass a hardware circuit (whether discrete elements or an integrated circuit block), a processor or control block executing code, or a combination of a hardware circuit(s) and a processor and/or control block executing code.

Referring toFIG. 1, a block diagram of a real time media communication network10is shown. The real time media communication network10includes a network12comprising a plurality of interconnected routers16(a)–16(c) interconnecting a plurality of network devices. The network12may be the Internet. Throughout this application, the network12may be referred to as the “Internet”, however, it should be appreciated this is for illustrative purposes only and does not limit the network12to the Internet or similar networks.

Coupled to the Internet12, or more specifically coupled to one of the routers16(a)–16(c), are a plurality of network devices which for purposes of this invention include real time media communication clients30(g) and30(h), proxy servers14(a) and14(b), a call control manager server (CCM)18, a directory server20, and a plurality of firewall servers32(a) and32(b).

Each of the network devices coupled to the Internet12is assigned a public Internet Protocol (IP) address. Frames of data are communicated between the various devices utilizing each devices IP address for routing the frames from a source device to a destination device. More specifically, a suite of protocols known as TCP/IP enables devices to set up TCP logical connections, and/or UDP logical channels, with each other utilizing each devices IP address and logical port number for purposes of exchanging data.

Each firewall server32(a) and32(b) may be a network address translation (NAT) server that operates as an IP layer proxy for devices coupled to each of a private network34(a) and34(b) respectively. Throughout this application, the firewall servers32(a) and32(b) may be referred to as “NAT Servers”, however, it should be appreciated this is for illustrative purposes only and does not limit the structure of the firewall servers32(a) and32(b).

Each private network34(a) and34(b) may function in a similar manner to the Internet12using the TCP/IP protocols. However, the IP address assigned to each client30(a)–30(f) on each of the private networks34(a) and34(b) may be an IP address selected from a block of IP addresses reserved for private networks. Addresses within the block of private network addresses are routable on the private network34(a) [or34(b)] but are not routable on the Internet12.

IP frames on the private network34(a) [or34(b)] are routed to the appropriate device on private network34(a) [or34(b)] when the destination address is within the block of private network IP addresses. When the destination address is a “public” IP address on the Internet12, the IP frame on the private network34(a) [or34(b)] is routed to the NAT server32(a) [or32(b)]. The NAT server32(a) [or32(b)] then emulates the destination device when communicating data over TCP/IP connections with the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] and operates as an IP layer proxy, by performing both address translation and port translation, to exchange data using TCP/IP connections with the destination device, on behalf of the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)], over the Internet12.

Further, the NAT server32(a) [and32(b)] may also be capable of translating connectionless datagrams sent by the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] and forwarding such connectionless datagrams to the destination device over the Internet12. And, if a connectionless datagram were to be replied to by the destination device and the reply datagram is 1) received at the NAT server32(a) [or32(b)] on the same port number as the NAT server32(a) [or32(b)] utilized when translating the connectionless datagram; 2) includes a source IP address and port number which matches the destination IP address and port number of the connectionless datagram; and 3) is received within a predefined time window following when the NAT server32(a) [or32(b)] sent the connectionless datagram, then the response datagram may be routed back to the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] on the private network34(a) [or34(b)].

The NAT server32(a) and32(b) may maintain a translation table that maps the source address and port number of the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] to the corresponding translated source address and port number for each TCP/IP connection opened (and UDP/IP connectionless datagram sent) by NAT server32(a) [or32(b)] on behalf of each initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)]. As such, the NAT server32(a) [and32(b)] may utilize the translation table to relay a reply frame received over the Internet12back to the appropriate initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] on the appropriate port number.

For added security, each entry in the translation table may include the destination IP address and port number to which the translated frame was sent over the Internet12. As such, the NAT server32(a) [32(b)] is capable of verifying that a frame addressed to the translated IP address and port number is truly a reply frame from the device to which the translated frame was addressed.

Upon receipt of any frame from the Internet, the NAT server32(a) [32(b)] will locate the one of the entries in the translation table to which the received frame corresponds utilizing the frames destination IP address and port number. The NAT server32(a) [32(b)] may then verify that the frame is truly a reply frame by comparing the frame's source address and port number with the destination IP address and port number in the entry. If there is a match, the NAT server32(a) [32(b)] will generate a reverse translated frame and forward the reverse translated frame to the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] on the private network34(a) [34(b)]. The reverse translated frame is the same as the reply frame except the destination IP address and port number are replaced with the initiating client30(a),30(b), or30(c) [30(d),30(e) or30(f)] private network IP address and port number.

Each client30(a)–30(h) whether on a private network34(a) or34(b) or directly coupled to the Internet12is configured to initiate (e.g. place) and terminate (e.g. receive) real time media communication sessions such as Internet telephony calls with a second client30(a)–30(h).

More specifically, each client30(a)–30(h) remains registered with one of the proxy servers14(a) and14(b). When initiating a media session the first client, client30(a) for example, may provide the proxy server with which the first client30(a) is registered, proxy server14(a) for example, with identity of the second client to which it would like initiate a media session, client30(d) for example. The proxy server14(a) then interrogates the directory server20to determine with which proxy server the second client30(d) is registered, proxy server14(b) for example. The two proxy servers14(a) and14(b) then facilitate the exchange of messages for setting up the media session, for communicating other messages representing media session negotiation between each of the first client30(a) and the second client30(d), and for directing each of the first client30(a) and the second client30(d) to route media datagrams to the CCM server18during the media session. During the media session, the CCM server18operates to relay real time media data between the first client30(a) and the second client30(b).

Referring to the ladder diagram ofFIG. 2in conjunction withFIG. 1, more detailed exemplary steps for real time media communication between the first client30(a) and the second client30(d) are shown.

Step36represents the first client30(a) sending a media session signaling message to the proxy server14(a) to which the first client30(a) is registered. The media session signaling message may be a SIP compliant “Invite” message and may identify: 1) the second client30(d) as the client with which the first client30(a) would like to initiate a media session, and 2) a first client network address established by the first client30(a) for receipt of media datagrams during the media session. The first client network address may include the IP address of the first client30(a), which is a private network address, and a first client port number assigned to the media session by the first client30(a).

It should be appreciated that a network address that includes an Internet Protocol address and a port number may be referred to as a real time protocol (RTP) channel. Throughout this specification, the terms network address and real time protocol channel are used interchangeably.

After receipt of the media session signaling message at step36, step38represents the proxy server14(a) providing the first client RTP channel to the CCM server18and requesting a CCM RTP channel for the media session from the CCM server18. The CCM RTP channel may include the IP address of the CCM server18and a port number of the CCM server18assigned to the media session. Step40represents the CCM server18providing the CCM RTP channel to the proxy server14(a).

At step42the proxy server14(a) initiates a media session signaling message to the proxy server14(b) with which the second client30(d) is registered. The media session signaling message of step42again may be a SIP compliant Invite message and may identify: 1) second client30(d) as the client with which the first client30(a) would like to initiate a media session, and 2) the CCM RTP channel.

After the proxy server14(b) receives the media session signaling message at step42, the proxy server14(b) sends a media session signaling message to the second client30(d) at step44. The media session signaling message sent to the second client identifies: 1) second client30(d) as the client with which the first client30(a) would like to initiate a media session, and 2) the CCM RTP channel.

It should be appreciated that because the second client30(d) is on the private network34(b) and can only communicate with the proxy server14(b) through the NAT server32(b), the proxy server14(b) may not be able to initiate the sending of the media session signaling message to the second client30(d) but instead may have to wait for the second client30(b) to poll the proxy server14(b) for the media session signaling message or to update its registration with the proxy server14(b) such that the proxy server14(b) can initiate the media session signaling message as a reply frame to the registration update frame such that the media session signaling message will be routed to the second client30(d) through the NAT server30(d).

Step46represents the second client30(d) sending a response message back to the proxy server14(b). The response may be a SIP compliant “Ringing” message and may identify a second client RTP channel for use sending media datagrams to the second client30(d).

After receipt of the response message by the proxy server14(b) at step46, step48represents providing the CCM server18with the second client RTP channel and requesting a CCM RTP channel for the media session. Step50represents the CCM server18providing the CCM RTP channel to the proxy server14(b).

At step52, the proxy server14(b) sends the response message back to the proxy server14(a). This response message again may be an SIP compliant “Ringing” message and may identify the CCM RTP channel. The proxy server14(a) then sends a response message, again identifying the CCM RTP channel, back to the first client30(a) at step54.

Steps56,58, and62represent the communication of datagrams representing additional media session negotiation between each of the first client30(a) and the second client30(d).

The steps within block64represent the exchange of media datagrams during the media session between each of the first client30(a) and the second client30(b) with the CCM server18operating as a relay server there between. More specifically, step66represents a first media datagram, addressed to the CCM RTP channel, being originated by the first client30(a) and received by the CCM server18. Because the first client30is coupled to the Internet12through the NAT server32(a), the NAT server32(a) translates the source IP address of the datagram from the private network IP address of the first client30(a) to the public IP address of the NAT server32(a) and translates the port number from the port number assigned by the first client30(a) to a port number assigned by the NAT server32(a).

After receipt of the first media datagram originated by the first client30(a) at step66, the CCM server18extracts a first client source network address (source Internet Protocol address and source port number). Because the CCM server18is not aware that the first client30(a) is behind a NAT server32(a) and because the CCM server18is able to work with clients that use different RTP channels for sending and receiving, the CCM server18may compare the first client source network address (specifically the source Internet Protocol address) to the network address from the Internet Protocol address of the first client RTP channel. Because of the translation, the extracted source Internet Protocol address will not match the Internet Protocol address of the first client RTP channel. At step68and step74, media datagrams sent by the CCM to the first client30(a) (referred to as third media datagrams for the sake of clarity) will be sent on an RTP channel defined by the extracted source network address such that the media datagrams will be routed to the first client30(a) by the NAT server32(a).

Similarly, step70represents a media datagram, addressed to the CCM RTP channel, being originated by the second client30(d) and received by the CCM server18. To distinguish this media datagram of step70from the media datagram of step66, this media datagram of step70will be referred to as the second media datagram. However, it should be appreciated that the labeling of first and second is for clarity only and does indicate a time order. It is possible for the second media datagram of step70to occur prior to the first media datagram of step66.

Because the second client30(d) is coupled to the Internet12through the NAT server32(b), the NAT server32(b) translates the source network address of the datagram from the private network IP address of the second client30(d) to the public IP address of the NAT server32(b) and translates the port number from the port number assigned by the second client30(d) to a port number assigned by the NAT server32(b).

After receipt of the second media datagram at step70, the CCM server18extracts a second client source network address comprising the source network Internet Protocol address and source port number from the media datagram and compares the extracted Internet Protocol address to the Internet Protocol address from the second client RTP channel. Again, because of the translation, the extracted source Internet Protocol address will not match the Internet Protocol address of the second client RTP channel. At step72and step76, media datagrams sent by the CCM to the second client30(d) (referred to as fourth media datagrams for sake of clarity) will be sent on an RTP channel defined by the extracted source network address such that the fourth media datagrams will be routed to the second client30(d) by the NAT server32(b).

Referring toFIG. 3, a block diagram of an exemplary client30is shown. The client30may include a desk top computer78and a traditional plain old telephone server (POTS) telephone80coupled thereto. The desk top computer78may include a processing unit82for operating a POTS emulation circuit84, a network interface circuit86, a driver88for the POTS emulation circuit84, a driver90for the network interface circuit86, and a media communication application92. Each of the POTS emulation circuit84and the network interface circuit86may be cards that plug into the computer expansion slots.

Alternatively, other configurations of a client30are envisioned which include all of the above systems embedded therein. Other configurations include, but are not limited to, an Internet telephony appliance structured as a network interface home telephone, a gaming device, or another consumer product with Internet telephony capabilities coupled to the Internet12(FIG. 1) via a wired or wireless connection such as the cellular telephone network, the PCS network, or other wide area RF network.

In the exemplary embodiment, the network interface circuit86and the network interface driver90together include the hardware and software circuits (including the IP stack) for operating the TCP/IP and UDP/IP protocols for communication on a TCP/IP compliant network.

The POTS emulation circuit84includes an RJ-11 female jack94for coupling the POTS telephone80to the POTS emulation circuit84. The POTS emulation circuit84comprises a tip and ring emulation circuit96for emulating low frequency POTS signals on the POTS tip and ring lines for operating the telephone80. The POTS emulation circuit84further includes an audio system98for interfacing the tip and ring emulation circuit96with the media communication application92. More specifically, the audio system98provides for digitizing analog audio signals generated by the microphone in the telephone80(and provided to the POTS emulation circuit84on the tip and ring lines) and presenting a digital audio signal to the media communication application92(preferably by writing the digital audio data to memory using direct memory access systems). The audio system98simultaneously provides for receiving a digital audio signal from the media communication application92(representing audio data received from a remote client via the CCM server18(FIG.1)), converting the digital audio signal to an analog audio signal, and coupling the analog audio signal to the tip and ring emulation circuit96. The tip and ring emulation circuit96modulates the tip and ring lines for driving the speaker of the telephone80in accordance with the analog audio signal generated by the audio system98.

Referring to the flowchart ofFIG. 4in conjunction with the block diagram ofFIG. 3, exemplary steps performed by the client30for media communication in accordance with this invention is shown. Step100represents registering with one of the proxy servers, proxy server14(a) for example. Step100may also represent poling the proxy server14(a) for any media session signaling messages from remote clients. The loop formed by steps100,102, and104represents waiting for either receipt of a media session signaling message at step102from the proxy server14(a) or an instruction to initiate a media session from the operator at step104.

If a media session signaling message is received at step102, step116represents establishing the an RTP channel for both sending and receiving media datagrams during the media session and step118represents sending a response message back to the proxy server14(a) which identifies the RTP channel. Step118corresponds to step46ofFIG. 2.

Step112then represents additional session negotiation via the proxy server14(a). Step112corresponds to steps68and62ofFIG. 2. Thereafter, step114represents participating in the media session. More specifically, at step114, the client30sends media datagrams (corresponding to the second media datagram of step70ofFIG. 2and the additional media datagrams of step76ofFIG. 2) to the CCM RTP channel provided to the client30in the media session signaling message received at step102. Further, at step114, the client receives media datagrams (corresponding to the fourth media datagram of step72ofFIG. 2and the additional media datagrams of step76ofFIG. 2) on the RTP channel established in step116.

If an instruction to initiate a media session is received at step104, step106represents establishing a RTP channel for both sending and receiving datagrams during the media session. Step108represents sending the invite message to the proxy server14(a) identifying the remote client with which the operator desires to initiate a media session and identifying the RTP channel. Step106corresponds to step36ofFIG. 2.

Step110represents receiving a response message back from the proxy server14(a) which identifies the CCM RTP channel. Step110corresponds to step54ofFIG. 2. Thereafter, when the client30is the initiating client, step112then represents communicating additional session negotiation messages via the proxy server14(a). Step112corresponds to steps56and58ofFIG. 2. Step114represents participating in the media session. More specifically, at step114, the client30sends media datagrams (corresponding to the first media datagram of step66ofFIG. 2and the additional media datagrams of step74ofFIG. 2) to the CCM RTP channel provided to the client30in the media session signaling message received at step102. Further, at step114, the client receives media datagrams (corresponding to the third media datagram of step68ofFIG. 2and the additional media datagrams of step74ofFIG. 2) on the RTP channel established in step116.

Referring toFIG. 5, in conjunction with the block diagram ofFIG. 1, exemplary steps performed by the proxy server14(a) and14(b) for media communication in accordance with this invention are shown.

The loop formed by steps116and118represents waiting for either a media session signaling message from a supported client at step116(corresponding to step36ofFIG. 2) or a media session signaling message from a remote proxy server at step118(corresponding to step42ofFIG. 2).

If a media session signaling message is received at step116, step120represents providing the RTP channel from the media session signaling message to the CCM server18and requesting a CCM RTP channel. Step120corresponds to step38ofFIG. 2. Step122represents receiving the CCM RTP channel from the CCM server and step124represents sending a signaling request to the proxy server with which the remote client (identified in the media session signaling message received at step116) is registered. Step112corresponds to step40ofFIG. 2and step124corresponds to step42ofFIG. 2.

Step126represents receiving a response message back from the remote proxy server and step128represents sending a response message back to the client. Step126corresponds to step52ofFIG. 2and step128corresponds to step54of FIG.2. Step138then represents exchanging additional session negotiation messages. Step138corresponds to steps56and58ofFIG. 2.

If at step118a media session signaling request is received from a remote proxy, step130represents forwarding the media session signaling request to the client identified in the media session signaling request received at step118. Step118corresponds to step42ofFIG. 2and step130corresponds to step44ofFIG. 2. Step131represents receipt of a response message back from the client30, corresponding to step46ofFIG. 2.

Step132represents providing the client RTP channel from the response message received at step131to the CCM server18and requesting a CCM RTP channel. Step134represents receiving the CCM RTP channel. Step132corresponds to step48ofFIG. 2and step134corresponds to step50ofFIG. 2.

Step136represents sending a response message that includes the CCM RTP channel back to the proxy server from which the media session signaling message was received at step118. Step136corresponds to step52ofFIG. 2.

Step138again represents exchanging session negotiation messages and corresponds to steps58and62ofFIG. 2.

Referring toFIG. 6aandFIG. 6b, in conjunction with the block diagram ofFIG. 1, operation of the CCM server18for media communication in accordance with this invention is shown.

The CCM server18comprises a network interface circuit19for communicating datagrams with other network devices coupled to the Internet12and with the clients30(a)–30(f) coupled to one of the private networks34(a) and34(b) through one of the NAT servers32(a) and32(b) respectively. A media communication application is operatively coupled to the network interface circuit. In operation, the CCM server18operates in accordance with the flow charts ofFIG. 6aandFIG. 6b.

Step140represents receiving a first client RTP channel and a request for a CCM RTP channel for a media session from a proxy server, proxy server14(a) for example. Step140corresponds to step38ofFIG. 2. Step142represents establishing a CCM RTP channel (including the public IP address of the CCM server18and a port number) for the media session and step144represents providing the CCM RTP channel back to the proxy server14(a). Step144corresponds to step46ofFIG. 2. The CCM RTP channel established at step142will be used for receiving the second media datagram at step70(and for sending the fourth media datagram at step72ofFIG. 2) and for simplicity can be referred to as a second CCM RTP channel (and the port number as a second port number).

Step146represents receiving a second client RTP channel and a request for a CCM RTP channel for the media session. Step148represents establishing a CCM RTP channel (including the IP address of the CCM server18and a port number) and step150represents providing the second CCM RTP channel to the proxy server making the request, proxy server14(b) for example. Step146corresponds to step48ofFIG. 2and step148corresponds to step50ofFIG. 2. Similarly, the CCM RTP channel established at step148will be used for receiving the first media datagram at step66ofFIG. 2(and sending the third media datagram at step68ofFIG. 2) and for simplicity can be referred to as a first CCM RTP channel (and the port number as a first port number). The first port number and the second port number may be the same.

Step154represents the beginning of the media session corresponding to block65ofFIG. 2. Step156represents receiving the first media datagram from the first client30(a) and step158represents extracting the first client source network address (source Internet Protocol address and source port number) from the first media datagram. Step160represents comparing the extracted source network address to the first client network address received as part of the first client RTP channel in step140from the proxy server14(a). If the source network address does not match the first client network address, a sending address for sending media datagrams to the first client30(a) is established to be the extracted RTP channel (extracted network address and extracted source port number).

Alternatively, if the extracted network address matches the network address from the first client RTP channel received at step140, the sending address for sending media datagrams to the first client30(a) is established to be the first client RTP channel.

During the same period of time when the CCM server18is performing steps156through164with respect to the first client30(a), it is also performing steps168through176with respect to the second client30(d).

Step168represents receiving the second media datagram from the second client30(d) and step170represents extracting the source network address and source port number from the second media datagram. Step172represents comparing the extracted source network address to the network address from the second client RTP channel received in step146from the proxy server14(b). If the source network address does not match the second client network address, a second client sending address for sending media datagrams to the second client30(d) is established to be the extracted RTP channel (extracted network address and extracted source port number).

Alternatively, if the extracted network address matches the network address from the second client RTP channel received at step146, the second client sending address for sending media datagrams to the second client30(d) is established to by the second client RTP channel.

Thereafter at step166the media session is maintained with the CCM server18operating as a relay. More specifically, media datagrams are received from the first client at step178and are sequenced and written to memory at step180. During the same time period, media datagrams are received from the second client at step198and are sequenced and written to memory at step190.

Also occurring during the same time period, media data received from the second client30(d) is retrieved from memory and sent to the first client30(a) using the first client sending channel which, in the case of client30(a) will be the extracted RTP channel, at step186. Similarly, media data received from the first client is retrieved from memory and sent to the second client30(d) using the second client sending channel which, in the case of client30(d) will be the extracted RTP channel, at step196.

It should be appreciated that because the CCM server18includes step160and step172which provide for comparing the extracted network address to the network address of the RTP channel provided by each clients corresponding proxy server, the CCM server18will operate with a client that has a public IP address and does not use the same RTP channel for both sending and receiving media datagrams during the media session. For example, assume a client, client30(h) for example, with a public IP address uses a first port for sending media datagrams and a second port for receiving media datagrams. As such, the RTP channel provided to the CCM server18will include the public IP address and the second port. However, the extracted RTP channel will include the public IP address and the first port. Because the IP address match, the CCM server will establish the client sending channel as the RTP channel provided to the CCM server18such that the client will be able to receive media datagrams on the second port.

More specifically, referring to the ladder diagram ofFIG. 2bin conjunction withFIG. 1, exemplary steps for media communication between a first client supported by a NAT server, client30(a) for example, and a second client30(h) that is directly coupled to the Internet12and may (or may not) use the same RTP channel for sending and receiving. It can be seen that steps36through62are the same as discussed with reference toFIG. 2(a).

However, the steps within block64′ will be different because the CCM server will detect that the IP address extracted from the second media datagram received at step70′ will be the same IP address as that provided to the CCM server at step48. As such, the fourth media datagram at step72′ and the additional media datagrams at step76′ will be sent to the RTP channel provided to the CCM server at step48.

Similarly, the ladder diagram ofFIG. 2crepresents exemplary steps for media communication between a first client directly coupled to the Internet12, client30(g) for example, and a second client supported by a NAT server, client30(b) for example, are shown. Again, client30(g) may (or may not) use the same RTP channel for both sending and receiving. Again, it can be seen that steps36through62are the same as discussed with reference toFIG. 2(a) and again it will be seen that the steps within block64″ will be different because the CCM server will detect that the IP address extracted from the first media datagram received at step66″ will be the same IP address as that provided to the CCM server at step38. As such, the third media datagram at step68″ and the additional media datagrams at step74″ will be sent to the RTP channel provided to the CCM server at step48.

In summary, the above described systems and methods provide for real time media communication between two clients if one or both of the clients have a private network address and are coupled to the Internet by a firewall server performing address translation and port translation. Although the invention has been shown and described with respect to certain preferred embodiments, it is obvious that equivalents and modifications will occur to others skilled in the art upon the reading and understanding of the specification. The present invention includes all such equivalents and modifications, and is limited only by the scope of the following claims.