Device and method for configuring a target device

An interfacing module connected to a network. A target network device connected to the network. An interfacing module is associated with a target network device. An interfacing module is coupled to a peripheral port of a target network device. Interfacing module identification data is stored in a memory portion of the target network device. The network address of the target network device is determined. The network address of the interfacing module is determined. At a remote location the network address of the target network device is used to read the interfacing module identification data from the memory portion of the target device via the network.

FIELD OF THE INVENTION

This relates to a virtualized media system with high-quality audio-video performance and high-performance virtual USB extension.

INTRODUCTION

This system is a platform for the creation of an Ethernet-based IP KVM system with high-quality audio-video performance and high performance virtual USB “extension.” The goal of the system is to enable a high-quality desktop experience for back-racked client target devices. Target devices are typically PCs and servers. The system architecture is focused on providing high-quality digital extension and back-racking. This includes extending KVM devices and USB peripherals, such as mass storage devices, over a network. The system is typically employed over a LAN, but as the underlying connection is IP-based it can be extended to work over a WAN. The system is designed to be implemented with minimal user initial configuration. It is an innovative solution to the weakness of current Thin-Clients (based on RDP or ICA) by ensuring constant video and audio experience no matter what the user is doing on the computer. Further, the system uses hardware engines that enable the full Ethernet pipe to be filled whether 10 Mbps, 100 Mbps or 1 Gbps. This is different from prior art KVM over IP implements in which this layer is performed in a software stack.

FIG. 1ais a schematic summarizing of the operation of remote virtualization of the system. The system creates a mechanism to virtualize so that a device driver on a host does not realize the device is remote and being accessed over a network. The system is realized with a combination of software and hardware.

The system is composed of two components—the host interface mechanism and the remote interface mechanism. The host interface mechanism is connected to the software driver layer in a host. This connection is over the normal host interfaces such as PCI or PCIe. The host interface mechanism looks to the software driver as the device that is being virtualized. For example, for virtualization of a USB Host Controller it provides the same register set and completes operations as if the actual USB Host Controller was there—the difference is the Host Interface Mechanism has to handle the fact that the action is actually happening in the Remote Interface Mechanism. The Remote Interface Mechanism actually drive the device being virtualized. The Remote Interface Mechanism receives commands from the Host Interface Mechanism and sends back replies to Host Interface Mechanism.

The host interface mechanism and remote interface mechanism are unique to the class of device being virtualized. For example, they would be unique remote and host interface mechanism for virtualizing a USB Host Controller chip and Audio chip. When virtualizing devices presenting industry standard interfaces the Remote Interface Mechanism and Host Interface mechanisms exchange capabilities and can auto negotiate to agree the virtualization capability set. In this way Remote Interface Mechanisms supporting different physical devices can interoperate with a specific Host Interface Mechanism.

The system is designed to virtualize a device so that it can be remotely accessed. This requires mechanisms to handle remoting of the interface mechanisms. This invention allows this virtualizing of a device to be performed without the software driver being aware of it.

The Host interface Mechanism and Remote Interface Mechanism interact with the host side device drivers and the remote virtualized device to make the host driver believe it is communicating with a local device. The Host Interface Mechanism and Remote Interface Mechanism are each composed of three core functional components.

The Host Device Interface is the device interface presented to the Host system (typically across a PCI or PCIe bus). The Host Device Interface is identical in register set and behavioural model to the device being virtualized. The Host system uses the same drivers and applications used with the real device to communicate across the Host Device Interface.

The Host Device Virtualization Engine maps the Interface presented to the Host to the Device Virtualization Protocol for communication with the Remote Interface Mechanism. The Host Virtualization Engine is aware of the remoting sensitivities of a specific device and applies a device specific virtualization algorithm that interacts with a compatible Remote Virtualization Engine in the Remote Interface Mechanism to “hide” the remoting of the real device from the Host system.

The Host Device Virtualization Engine and Remote Device Virtualization Engine communicate using the transport independent Device Virtualization Protocol.

The Remote Virtualization Engine interacts with the real device through the Remote Device Interface which communicates with the actual Virtualized device.

The Device Virtualization Engines operate with awareness of the device functionality and host system driver requirements to effectively virtualize devices over remote links. The associated algorithms are specifically optimized to accommodate bandwidth and latency characteristics of remote links. For example a set of specific algorithms accommodate virtualization of a USB 2.0 Host Controller, each algorithm focused on specific remoting aspect of a the USB 2.0 Host Controller Interface.

Alternative approaches used to solve this problem have been to emulate a device at both ends. The system described herein enables the virtualized device to behave and function as if locally connected using its native device drivers and exploiting the devices full functionality. This approach is not concerned with remoting the output from a specific hardware device being virtualized and as such does not have to content with resultant data transformation performed by the device. Such transformations are optimized for communicating with local peripherals and devices. This approach leads to an efficient and true virtualization of the device.

FIG. 1bshows a general exemplary configuration of a virtualized media system and illustrates the main components in the system and basic connections between components.FIG. 1bis an exemplary embodiment of the schematic of1a. The purpose of the system is to allow peripherals (KVM peripherals and a wide range of USB based peripherals) to access a target device across a network. In the exemplary embodiment there are four types of information that are transmitted across network100. The four types of information are: video information, keyboard/mouse information, audio information, and media information. Each type of information is transmitted in a media stream. There are three main components of the system connected to network100: Digitalizing Interface Pod (DIP)400, Digital User-Station (DUS)500, and Management Application (MgmtApp)200. Network100is typically an IP-based LAN/WAN network. It should be noted, components will be able to work without issue if Network Address Translation (NAT) devices are used on network100. In the exemplary embodiment, DUS500and DIP400will be hardware units and MgmtApp200will be a software application. In alternative embodiments, this could change so DIP400and DUS500are software embodiments and MgmtApp200embodied in hardware.

DIP400connects to the peripheral interfaces of a target device300. DIP400transports the I/O streams between a target device300and network100. Target devices300are typically back-racked PCs or servers, as shown inFIG. 1b, but are not limited to such. Target devices300can include any device with peripheral ports (e.g. media player, PDA, Set top box, etc.).

DUS500is essentially the inverse of a DIP400. DUS500connects to various peripherals at a user station. DUS500transports I/O streams between network100and connected peripherals. Exemplary peripherals connected to DUS500shown inFIG. 1include: PS/2 keyboard800, PS/2 mouse900, monitor600, audio device700, and USB peripherals1000. USB peripherals1000can include any type of USB device including mass storage devices, a USB mouse and a USB keyboard.

Any number of combinations of DIPs400and DUSs500can be used in the system. However, in some instances, there is a single DUS500and DIP400pair. In such instances, network100can comprise a single cable directly connecting the pair or an IP subnet. Exemplary DUS500and DIP400will have network interfaces that enable them to operate over a 100 BT or 1 Gig copper cabled Ethernet network.

MgmtApp200is a software application that provides authentication, access control, and accounting services for a network of DIPs400and DUSs500. MgmtApp200includes a database that stores information about each DIP400and DUS500connected to the network100. It should be noted that MgmtApp200is not necessary when there is a single DUS500and DIP400pair. The single DUS500and DIP400pair is described in greater detail in accordance withFIG. 15. This MgmtApp200can be server based as shown or integrated into a network switch or appliance.

An exemplary DIP400is shown inFIG. 2. DIP400is typically implemented as an external dongle that connects to the peripheral ports of a target device300. However, a DIP400can also be embedded inside a target device300. This may be in the form of a PCI card or as an embedded chip-set to be integrated by OEMs. Exemplary DIP400presents a target device300with a video connection, an audio connection, and a pair of USB connections.

For descriptive purposes, exemplary DIP400is shown as being comprised of three hardware functional blocks: target device interface hardware410, media stream processing hardware420, and network communications hardware430. Each of the hardware functional blocks interacts with a software layer440. Functional hardware blocks are typically embodied by an FGPA, but can also be implemented using an ASIC or other hardware implementations or combinations thereof. Using three functional blocks to describe DIP400is not intended to limit the ways in which a DIP400can be physically implemented.

Target device interface hardware410interfaces a target device's peripheral ports and media stream processing hardware420. Target device interface hardware410converts data communicated between target device's peripheral ports and media stream processing hardware420. An example of such a conversion is converting analog data output from a target device into a digital data form that can be processed by media stream processing hardware420. Target device interface hardware410includes a USB peripheral controller412, an audio codec416, and a video receiver418, each of which is described in greater detail with their respective media stream. Although not shown inFIG. 2, target device interface hardware410can also provide a serial connection for interfacing a target device's serial port.

Media stream processing hardware420interfaces target device interface hardware410and network communication hardware430. Media stream processing hardware420packetizes/depacketizes data communicated between target device interface hardware420and network communication hardware430. Media stream processing hardware420includes a keyboard/mouse engine422, a mass storage engine424, an audio engine426, and a video engine428, each of which is described in greater detail with their respective media stream.

Network communications hardware430interfaces network100and media stream processing hardware420. Network communication hardware430receives data packets from media stream processing hardware420and converts the data into a form compatible with network100. Network communication hardware430also receives data from network100and converts it into a form compatible with media stream processing hardware420. Network communications hardware430includes management port432, network engine434, and SSL/encryption engine436, each of which is described in greater detail in accordance withFIGS. 5 and 6.

An exemplary DUS500is shown inFIG. 3. Exemplary DUS500presents the following peripheral interfaces: a video connection (e.g. VGA and/or DVI), audio in/out connectors, PS/2 keyboard and mouse connectors, USB connectors for USB peripherals (e.g. keyboard, mouse, mass storage, and other USB peripherals), and a serial interface (e.g. RS232).FIG. 4shows an exemplary DUS500panel illustrating interfaces.

For descriptive purposes, exemplary DUS500is shown as being comprised of three hardware functional blocks: peripheral interface hardware510, media stream processing hardware520, and network communications hardware530. Each of the hardware functional blocks interacts with a software layer540. DUS500also includes an on screen display (OSD) hardware550and universal asynchronous receiver/transmitter (UART) hardware560. Using three functional blocks to describe DUS500is not intended to limit the ways in which a DUS500can be physically implemented.

Peripheral interface hardware510interfaces peripherals and media stream processing hardware520. Peripheral interface hardware510converts data communicated between peripherals and media stream processing hardware520. Peripheral interface hardware510includes USB controller512, PS/2 interface514, audio codec516, VGA Transmitter518, and DVI transmitter519, each of which is described in greater detail with their respective media stream.

Media stream processing hardware520interfaces target device interface hardware510and network communication hardware530. Media stream processing hardware520packetizes/depacketizes data communicated between target device interface hardware420and network communication hardware430. Media stream processing hardware520includes a keyboard/mouse engine522, a mass storage engine524, an audio engine526, and a video engine528, each of which is described in greater detail with their respective media stream.

Network communications hardware530interfaces network100and media stream processing hardware520. Network communication hardware530receives data packets from media stream processing hardware520and converts the data into a form compatible with network100. Network communication hardware530also receives data from network100and converts it into a form compatible with media stream processing hardware520. Network communications hardware530includes management port532, network engine534, and SSL/encryption engine536, each of which is described in greater detail in accordance withFIGS. 5 and 6.

OSD hardware550provides a GUI that allows a local user to configure DUS500.

UART560interfaces a serial port on DUS500and that allows data to be transferred to DUS500. For example, a firmware upgrade of a DUS500can be accomplished by connecting a PC using XMODEM over a local RS232 port.

In the exemplary embodiment network communication hardware blocks430and530will transmit and receive up to four AES encrypted SSL/TPCIP streams. These streams are used to transport video, mass storage, audio, keyboard and mouse data between DUS500and DIP400. The media stream processing hardware blocks420and520contains specific media processing hardware engines that source and sink data over the SSL/TCPIP streams. The four SSL streams shall be used as follows by the media processing engines:

The DUS500and DIP400require minimal software involvement to transfer the associated data streams. The hardware and software involvement in the data stream transport is described in greater detail below, in accordance the descriptions architecture and processing required for each media stream (video, mass storage, keyboard/mouse, audio). All other network communications go through the management ports432and532.

The top level collaboration of software440and540, media stream processing hardware430and530and network communication hardware components (i.e. management ports432and532, network engines434and534, and SSL engines436and536) and the role they play in the establishment and teardown of a media transfer session (Video, Keyboard/Mouse, Audio, vMedia) for an initialized DIP/DUS pair is illustrated inFIGS. 5-7.

FIG. 5illustrates an exemplary management port432. The management port432presents a MAC level device interface to the software440. All packets except those associated with TCP ports active on the network engine434are received by management port432and presented to software440. The management port432has a receive packet FIFO buffer460and transmit packet buffer462. In the exemplary embodiment, receive packet FIFO buffer460is an 8 Kbyte buffer and transmit packet buffer462is a 1.5 Kbyte buffer. All LAN packets addressed to the unicast MAC address programmed on the network engine434and broadcast IP and ARP packets are received and placed in the receive FIFO460. Multicast packets are not received. A high watermark system will be used whereby if receive buffer460is filled over a specific percentage (e.g. 50%) all broadcast packets are discarded and counted until the receive buffer level reduces. If the receiver FIFO buffer460fills, unicast packets are also discarded and counted. Transmit buffer462stores packets before they are transmitted to network100. The management port432has little explicit configuration other than that packet reception can be enabled/disabled and can be configured to interrupt when a packet transmit is complete or when a packet is in the receive FIFO buffer460. It should be noted that management port532is similar to management port432and for the sake of brevity is not described herein.

In the exemplary embodiment, software440and540creates a TCP control channel between a DUS500and a DIP400, after which the network engines434and534are configured at each end (a TCP port for each media stream) and TCP data transfer commences abruptly on the media session TCP ports (no SYN or FIN phases in media session establishment or teardown).

The exemplary embodiment uses an Avocent (Huntsville, Ala.) commercially available AVSP protocol format. It should be noted that although the exemplary embodiment is described using AVSP, other protocols can be used. An AVSP control channel is used to establish the session. The AVSP session is used for control of media sessions using a small subset of the AVSP message set. The media (video, keyboard, mouse, etc) transfer messages of AVSP are not used in the exemplary embodiment, all media transfer flows through the network hardware engines434and534using the specific media transfer sessions. The network engines434and534are architected to provide efficient data transfer using TCPIP sessions. The establishment and teardown of the TCPIP sessions is achieved in cooperation with a software SSL/TCP stack. The control channel is used for out of band media session control and session monitoring. All messages on this link use the AVSP protocol format. The exemplary embodiment uses a small and extended subset of the available AVSP messages, it does not use the media/data transfer AVSP messages, and the hardware engines use their own protocol. The channel is used to exchange session control messages between the DUS500and DIP400(changing media session properties etc).

FIG. 6illustrates the process that is used for establishment of a media session. As shown inFIG. 6, the process is described as consisting of the following five steps: (1) Establish SSL TCP Session; (2) Initialize Network Engines; (3) Initialize Encryption Engines; (4) Initialize Media Stream Processing Hardware Engines; and (5) Enable Media Data Path.

In the Establish SSL TCP Session step, the DUS500initiates the connection to the AVSP server port on the DIP400(e.g. port2068) and establishes an AVSP SSL connection with the DIP400. The exemplary system shall only allow one AVSP session on a DIP400. The session is authenticated using the session certs on the DUS500and AVSP keep alive between DUS500and DIP400.

After the SSL TCP session is established, the Initialize Network Engines step occurs. Network engines434and534need to be configured with MAC and IP addressing information as well has having the TCPIP windowing and general operating parameters configured. If exponential back off is required, then software440and540must set the timeout accordingly.

The following core communication parameters must be configured on the hardware network engines434and534:

MAC Data

Source MAC (L-MAC): This is the MAC address of the appliance (i.e. DIP400or DUS500). It is available in persistent storage of the appliance.

Dest. MAC (D-MAC): The address is retrieved from the ARP table of the software stack after the DUS-DIP AVSP control session is established. This may not necessarily be the MAC address of the partner appliance involved in the connection.

IP Data

Source IP Address (L-IP): Obtained through DHCP or via static configuration of the IP address information on the appliance.

Destination IP Address (D-IP): Provided to the Appliance in the Session Cert. for the connection.

TOS: Type of Service field, describing the quality of service requested by the sender. The byte is composed of:Precedence field (provides an indication of the priority): 0=normal, 1=priority, 2=immediate, 3=Flash, 4=Flash Override, 5=Critical, 6=Interwork Control, 7=Network Control. Recommend Setting to 5.Delay Bit: 0 indicates can be delayed, 1 cannot be delayed. Recommend setting to 1.Throughput Bit: 0=normal, 1=high, recommend setting to 1Reliable bit: specifies if a reliable sub network is required. 0=not, 1=yes, set to yes.

TTL: Time to live. Number of Hops for IP packet, the following default value is recommended255.

TCP Data

Port # (D-Port, L-Port): The following port numbers are recommended for the media transfer protocols:Video: 4459Audio: 4460Keyboard/Mouse: 4461Mass Storage: 4462

Max Segment Size (L-MSS): The largest block of data TCP will send to the other side. The recommended value for exemplary embodiment is 1460 bytes, the largest value that will keep each TCP transaction within one Ethernet packet so that the need to segment and reassemble the TCP packet is not required.

Advertised Window Size and Scale (AW and AWS): The size and scale of the TCP window that will be advertised to the other side. This effectively tells the other side of the connection how much data this unit can buffer and thus how much the other side can send. The optimum values can be determined through performance analysis of the system.

Congestion Window Size (CGW): If software440and540wishes to perform congestion control (the connection is frequently retransmitting packets and there may be congestion) it can use the “Congestion window size” to reduce the amount of data transmitted. When there is no congestion the congestion window is set to be the receivers'advertised window size. That is, a transmitter uses the receiver's advertised window to determine how much data to send. If it is determined that congestion is being experienced, the congestion window size can be reduced while congestion is being experienced, this reduces the window of data transmitted. A transmitter determines amount of data to transmit as being the smaller of the window advertised by the receiver and the local congestion window setting.

Retransmission Timeout Value: This value controls that retransmission and retransmission backoff. The retransmission timeout value is generally changed (using formula) based on the estimated Round Trip Time (RTT) for a connection. If variable retransmission timers and backoff are required, then software440and540needs to request the network engine434and534to calculate the RTT at intervals, and then use a smoothed average (SRTT) and the variance (RTTVAR) of the RTT samples to calculate a suitable timeout value as follows: Timeout=SRTT+4*RTTVAR.

Software440and540can also elect not to dynamically modify the timeout value, and set the retransmission timeout to a conservative value (possibly based on the initial value of RTT read) and maintain for the duration of the connection. This will be sufficient in well defined and controlled networks.

In addition to setting the parameters described above, software440and540needs to do the SYN transactions to establish the TCPIP session, and then transfer the sequence numbers (NextRxSeq#, NextTx#, and RxAckSeq#) to the network engine434and534for the data transfer.

After the network engines434and534are initialized, the Initialize Encryption Engine step occurs. DIP400and DUS500initialize the encryption engines436and536associated with each media stream that is enabled in the session cert. The encryption engines436and536encrypt and decrypt data transported by the TCPIP network engines434and534. The engines436and536use 128 bit AES encryption on the media streams. The engine436and536will be used to encrypt video, mass storage, keyboard and mouse streams. The encryption must be configured with the 128 bit cipher key. This cipher key obtained as a result of the creation of the DUS-DIP AVSP control session SSL connection is used to seed the encryption of each media session. Thus, the key required to seed the SLL hardware encryption engines436and536is the same key that was generated after software SSL session negotiation. This guarantees that the key changes with each new connection.

After encryption engines436and536are initialized, the Initialize Media Stream Processing Hardware Engines step occurs. The DIP400and DUS500initialize the media application engines within media stream processing hardware420and520. The DIP400and DUS500respective media engines (e.g. video processing engines428and528) “tunnel” application specific control, setup, and session management data (known sequence numbers) through the AVSP control channel to enable setup to the specific media stream.

Once the respective media engines are initialized, The Enable Media Data Path step occurs. A connection is established between DUS500and DIP400and data streams are transferred by the hardware TCP “lite” engine called TOELite. The “lite” is used to refer to some basic optimizations made on the network engines434and534due to the fact that the system is optimized to run over a LAN.

On connection establishment, an external observer will see the media carrying TCP sessions start by transferring data without a TCPIP setup (SYN) or teardown (FIN) stage. Transferring data without a SYN or FIN stage will be tolerated by layer 3 switches and routers. However, going through layer 4 aware equipment like firewalls is an issue unless the specific ports are opened up on the firewall. The system of the exemplary embodiment will use fixed port numbers for each media stream on the DUS500and DIP400, this way the specific ports can be opened on firewalls. The exemplary embodiment only uses the facility to enable keyboard, Mouse and vMedia session control entities on the DUS500and DIP400and to pass descriptors between the DUS500and DIP400. However, it is available should other media applications require specific DUS-DIP communications.

FIG. 7illustrates the process that is used for the teardown of a media session. As shown inFIG. 7, the process is described as consisting of the following three steps: (1) Stop Application; (2) Stop SSL Session; and (3) Disable connection. Software540initiates the FIN transactions to tear down the TCP session, again extracting the sequence number from the network engines434and534. To the outside world the TCPIP session looks like one seamless session.

InFIG. 7, the Stop Application Step involves DUS500commanding DIP400to close a specific media session its specified port. After the application is stopped, the SSL session is stopped at the Stop SSL Session step. After the SSL session is stopped, that is transmission of data is stopped, the Disable connection step disables the transport connection.

The system components and behavior with respect to a video media stream are illustrated inFIG. 8.

DIP400presents a video interface to a target device300. In the exemplary embodiment the video interface is a DVI-I interface. However, the video interface is not limited to a DVI-I interface and can include any number of video interfaces including analog (e.g. component video, S-video, etc.) and digital interfaces or combinations thereof. The interface is connected to video receiver418. Exemplary video receiver418is a DVI/VGA(RGB) video receiver capable of receiving both DVI and VGA video data. Video receiver418can be adapted to receive various types of analog or digital video data.

On target device300power-up, target device300uses DDC to read the EDID table on the DIP400. The DIP400has a hard-coded EDID table advertising the capabilities of the system.

Video receiver418can receive RGB video and supports resolutions up to at least 1280×1024@75 Hz. When video receiver418receives RGB or another type of analog video, video receiver418will digitize the video and forward it to the video processing engine428. Software440configures video receiver418by specifying the optimum sampling of the VGA input signal (sampling phase). Software440adjusts the phase of the sampling clock used by the video receiver418to optimize the sampling of the analog input. It does this by continuously adjusting the phasing and looking at the data reported by the video engine428to get the optimum setting.

Video receiver418will capture digital video data from the DVI interface with support for resolutions up to at least 1280×1024@60 Hz. The digital video data will be forwarded to the video processing engine428.

When DIP400presents a DVI-I connector to a target device300, the DVI I2C DDC interface is routed so that the serial interface is made available to software440for processing of DDC requests from the target device300.

The video receiver418can automatically detect the active input interface. For example, when DVI-I is used, video receiver418can automatically detect if RGB or digital DVI is being received. When receiving VGA, video receiver418needs to be adjusted by software440to align the clocking “phase.” Software440does this by monitoring data made available by the video engine428and configuring the video receiver418, then monitoring the data available by the video engine428. This cycle is repeated until the optimum setting is reached.

Software440determines when a video source is detected by the video receiver418, identifies the source, and then informs the video processing engine428. Video processing engine428receives video from the video receiver418and prepares the video for the network communications hardware430by encoding the digital video.

Video processing engine428is configured by software440. When the video processing engine428receives digitized video it makes available a number of observed/measured video characterizing data in its registers. Software440reads this data and based on pre-configured ranges deduces the VESA Video operating mode (table lookup for resolution, settings, etc.) after which it programs the video processing engine428for the chosen video mode.

In the exemplary embodiment, video processing engine428compresses the video by using a scheme based on the directional algorithm concepts previously developed with some newly added enhancements to compress frames of video. Its specific application is to reduce the bandwidth used in transmitting a video frame buffer across an Ethernet LAN. Video processing engine428uses the compression algorithm to create video packets.

The key to the algorithm is that each side of the link has a version of the previous frame to use as a reference. This allows each pixel in subsequent frames to be defined in one of the following several ways:1. No change from pixel in previous frame (NO_CHANGE)2. Same as pixel in line above (COPY_ABOVE)3. Same as pixel as immediately to the left (COPY_LEFT)4. Series of pixels from a preceding known subset (MAKE_SERIES)5. Make new pixel (NEW_PIXEL or MAKE_PIXEL)6. Delta from the same pixel in the previous frame (DELTA_NC)7. Delta from the pixel immediately above (DELTA_CA)8. Delta from the pixel immediately preceding to the left (DELTA_CL)9. Short Delta from the same pixel in the previous frame (Short_Delta_NC)10. Short Delta from the pixel immediately preceding (Short_Delta_CL)11. Make Pixel using fewer bits (Short_Make_Pixel)

The compression algorithm is described in greater detail in co-pending U.S. application Ser. No. 11/707,879, entitled “Video Compression Algorithm” filed Feb. 20, 2007, which is incorporated herein by reference.

In the exemplary embodiment when video processing engine428compresses video, software440configures video processing engine428by specifying the following parameters: video mode (resolution) when VGA is used, the # of Intermediate frames used for color depth (range 0 to 7), the color depth on the reference and intermediate frames. (9, 12, 15, 18, 21, 24 bit color), the watermark for latency between DIP400and DUS500for frame dropping in units of 256 lines, the max frame rate expressed as a ratio with respect to the frame rate being received from the video receiver418. Frame dropping can be set to drop all frames, drop every second frame (take a 60 fps video stream to a 30 fps video stream), or 1 in every 3 frames (takes a 60 fps stream to a 40 fps stream).

The video processing engine428compresses the video based on parameters set by software440and feedback from DUS500. The compressed video is forwarded to network communication hardware430where network communication hardware430prepares compressed video for transmission over the IP network100to DUS500.

The DUS500presents a video connector for connection to a monitor600. The DUS500can present any appropriate video connector (e.g. component, S-video, DVI, VGA, etc.). In the exemplary embodiment, DUS500presents a DVI-I connector to a monitor600. When the DUS500presents a DVI-I connector to a monitor600, a VGA-to-DVI-I adaptor may be required depending on the cable available with the monitor600. Further, when DUS500presents a DVI-I connection to a monitor600. The I2C DDC channel on the DVI connector is presented to software540so it can query the monitor600for its EDID table. The DUS500on power-up reads the monitor600EDID table. If monitor600cannot support the full capabilities as advertised by the DIP400to the target device300, a warning message is presented on the monitor600.

Video engine528processes video data received over the IP network100and forwards it to the appropriate video transmitters518and/or519(depending on the video connector). Software540will need to configure the video engine528with the video mode (resolution) being used when VGA is used. The DIP400will inform the DUS500using an in-band command. On notification of the mode, software540looks up a table to find the setting associated with the mode. In the exemplary embodiment, software540will also specify the # of intermediate frames used for color depth (range 0 to 7), and the color depth on the reference and intermediate frames (9, 12, 15, 18, 21, 24 bit color) to video processing engine528.

The DVI transmitter519receives digital video from video processing engine528and encodes and transmits the data on the DVI link to a monitor600without requiring involvement or any explicit configuration from software540.

The RBG/VGA transmitter518receives digital video from video processing engine528and converts to RGB format for transmission to an attached monitor600without requiring involvement or any explicit configuration from software540. DVI-I interface used in the exemplary embodiment supports H and V sync. In alternative embodiments sync on green is can also be supported.

Video packets created by video processing engine428are encapsulated in SSL/TCP packets of a fixed size for transmission across the network100. Using fixed size packets enables the hardware based TCP transport engine434to be optimized to retransmit on a TCP packet basis rather than on a pure byte stream basis, thereby by maintaining a consistent bounded video frame received by video processing engine528without the need to manage byte stream re-transmit and associated packet reassembly in hardware.

TCP will guarantee the delivery of video packets in the correct order to video processing engine528. Video processing engine528generates an explicit acknowledgement for each video packet received. The acknowledgement contains the current Frame# and Line# received by the video processing engine528for processing.

The acknowledgement is sent across network100to video processing engine428. The DIP400is the controlling source for video flow control. An important aspect is to monitor the video latency across the system and adjust the video frame rate to the DUS500to maintain an acceptable latency. The DUS500displays all video frames sent to it. Video processing engine428uses the acknowledgment to understand how many lines the video receiver418is behind the video transmitter518or519, this is a measure of the latency of the video between the video processing engine528and the video processing engine428, and is a measure of the latency caused by each ends'encryption/TCP transport stack and network link. The DIP400monitors the Frame# and line# within the frame being processed by the DUS500with respect to the current Frame# and line# being processed by it. If these exceed specific thresholds the DIP400reduces the Frame rate to the DUS500.

When the latency difference between the line being processed by the DUS video processing engine528and the line being processed by the DIP video processing engine428is greater than a configurable watermark then a frame will be dropped by DIP video processing engine428, effectively reducing the frame rate until the latency is below the defined watermark. This mechanism minimizes the latency on the DIP-DUS video path which is important for mouse performance.

Software440can configure the color depth to be used by video processing engine428to manage the video experience and associated network bandwidth. The management of bandwidth is maximized by the use of reference and intermediate frames from the perspective of color depth, where software440can specify a number of frames (intermediate frames) between core references frames that carry a specified lower color depth.

When receiving VGA video, software440needs to monitor specific network engine data434, and use it to deduce the video VESA mode settings from a table and then program video processing engine428.

On setting the VESA mode in video processing engine428, the video processing engine428sends an in-band video mode command to the DUS500informing it of the mode change. The DUS software540will read the mode change event, lookup the mode data, and program the DUS video processing engine528accordingly.

The system components and behavior with respect to an audio media stream are illustrated inFIG. 9.

DIP400presents audio connectors to a target device300. In the exemplary embodiment the audio connector comprises a stereo connector (e.g. a 3.5 mm mini jack, a pair of RCA jacks, etc.) for audio output from target device300and a mono connector (e.g. a 3.5 mm mini jack or an RCA jack) for audio input to target device300. The audio connectors present audio signals output from target device300to audio codec416and receives audio signals to be input to the target device300. Audio codec416uses a basic linear quantization scheme to convert analog audio out signals into digital signals and forwards the digital signals to audio engine426. Audio codec416also receives digital signals from audio engine426. Audio codec416stores received digital signals in a playout buffer and converts the digital signals to analog signals which are presented to target device300. In the exemplary embodiment, audio engine426and audio codec416are FPGA based.

DUS500presents audio ports to an audio peripheral700. In the exemplary embodiment the audio ports comprises a stereo port (e.g. a 3.5 mm mini port, a pair of RCA ports, etc.) for a pair of speakers702and a mono port (e.g. a 3.5 mm mini jack or an RCA jack) for a microphone704or an audio input device. Audio codec516uses the audio ports to present audio signals to speakers702and receive audio signals from microphone704. Audio codec516uses a basic linear quantization scheme to convert analog audio signals from microphone704into digital signals and forwards the digital signals to audio engine526. Audio codec516also receives digital signals from audio engine526. Audio codec stores the received digital signals in a playout buffer and convert the digital signals to analog signals which are presented to speakers702. In the exemplary embodiment audio engine526and audio codec516are FPGA based.

In the exemplary embodiment, the system provides two 44.1 KHz sampled 16-bit channels (stereo) from a target device300to an audio peripheral700and a single 44.1 KHz sampled 16-bit channel from the audio peripheral700to the target device300(i.e. mono only transport). A 44.1 KHz sampling rate will provide transport of audio signals with frequency components up to 22 KHz. Because the audio peripheral700to target device300transport is a single channel, stereo microphones plugged into DUS500will result in transmission of a single channel to DIP400, where the same signal is sent on each of the two channels to target device300.

Audio engine426and526transmit and receive digitized audio over the IP network100using a system specific audio packet format. A transmit side takes the digitized audio stream from its codec, constructs an audio packet and forwards it to its network engine for transmission over the TCP link. The receiver side obtains the audio packets from its network engine and adds the data to the playout (jitter) buffer. The playout buffer makes a stream of samples available to its audio codec which converts the sample to analog form and plays out the audio. The audio packets transmitted on network100will have a structure that enables the transport of specified number of samples for up to two channels in a single packet.

It should be noted that although exemplary DIP400is described as receiving two audio channels from target device300, such a description is for exemplary purposes only and is not intended to limit the number of audio channels DIP400can receive. DIP400can receive any number of audio channels (e.g. 5, 6, or 7 as used in surround sound systems). Likewise, DIP400can have multiple audio input channels. Further, although DIP400is described as sending/receiving analog audio signals to/from a target device300, DIP400can be configured to receive digital audio signals and combinations of digital and analog audio signals.

Management of the playout buffer to accommodate variances in the received packet interval (network jitter) is vital to quality playout. The algorithm shall work in principle as follows.

After the connection is established, the receiver will commence playout of audio samples when it has the playout buffer 50% full. This enables the receiver to tolerate variances in the received audio samples of plus or minus the number of samples that are contained in 50% of the playout buffer.

If the playout buffer depletes, the previous sample is replayed until the next sample arrives. If the playout buffer is full, the playout recommences at the center of the playout buffer (the playout is reset to center of the playout buffer again).

To mitigate against having to take such abrupt actions that may result in an audible interference, the playout buffer has low and high water marks, set at 20% and 80% respectively. If the playout buffer goes below the 20% mark each sample is played out twice, until the buffer fills above 20% after which normal sample playout recommences. If the playout buffer goes above 80% every second sample is discarded until the buffer goes below 80% after which normal playout re-commences.

DVD playout requires the audio playout to be within plus or minus 80 ms of the video to maintain lip sync. The exemplary system takes no explicit action to attempt to keep audio and video in sync as the processing of network latencies are such that lip sync should be maintained. However, a mechanism can be added in alternative embodiments.

The performance of the audio streaming connection is primarily a function of the audio packet size, variance in network latency and playout buffer (jitter) size. The playout buffer size can be configured on an audio engine to accommodate 10 ms to 250 ms network jitter. An audio engine shall send 5 ms of audio for two streams in each packet. This is the minimum packet size to effectively operate with a 10 ms jitter buffer. The size of the audio packet determines the granularity of the jitter experienced by the receiver, the larger the packet the greater the efficiency of the transmission. However, there is also an increase in the impact to the audio quality due to changes in packet latency due to packet loss or congestion in the network100.

To contain 5 ms of audio, an audio packet will need to be capable of carrying 882 bytes of raw sample data (excluding audio packet header, TCP, IP, and Ethernet headers). The playout buffer (jitter) size can be modified under software control. The buffer sizes required at the receiver to accommodate network jitter in the 10 to 250 ms range will be between 1764 bytes and 88200 bytes.

Playout buffer sizes for the exemplary embodiment can be calculated as follows:

44100×2 bytes=88200 bytes per second (88.2 bytes per ms) digitized audio stream. Stereo has two streams providing a total of 176.4 bytes per ms of audio.

5 ms of Audio=5×176.4=882 bytes

To accommodate+−10 ms network jitter, will require a buffer of 1764 bytes×2=3528 bytes.

To accommodate+−250 ms network jitter, will require a buffer of 44100 bytes×2=88200 bytes.

The system components and behavior with respect to a keyboard & mouse media stream and a mass storage media stream are illustrated inFIGS. 10-13.

FIG. 10illustrates the behavior on powerup of DUS500and DIP400before a connection between DUS500and DIP400is established.

DIP400presents a low/full speed capable USB port412aand a full/high speed capable USB port to a target device300. In the exemplary embodiment, the DIP/target device interface412is comprised of two peripheral controllers: low/full speed capable USB port412aand a full/high speed capable USB port412b.

The DIP400on power-up enumerates to a target device300. Port412aenumerates itself to a target device300as a composite USB device containing the following: a keyboard device with one interrupt endpoint for traffic from the keyboard that will provide a report descriptor describing a standard keyboard report format and a mouse device with one interrupt endpoint for traffic from the mouse that will provide a report descriptor describing a standard mouse report format. Port412benumerates itself to a target device as a mass storage device, using bulk only transport class with a subclass of “Transparent SCSI” command blocks. Target device300will load its standard keyboard, mouse and mass storage device drivers in response to this enumeration. The DIP400must be able power up and be in a state to let the target device300know that a keyboard is present before the target device300determines that it cannot see a keyboard.

The DUS500presents the following peripheral ports: a PS/2 mouse, PS/2 keyboard, and four USB ports where each USB port will accept low, full, or high speed USB peripherals.

The USB ports of DUS500interface a USB controller512. USB controller512can be a commercial off-the-shelf host controller capable of low, full, and high speeds. In the exemplary embodiment, the USB ports will be four USB type-A connectors. However, another number or other types of USB connectors can also be used. In the exemplary embodiment, controller512is required at a minimum to be capable of simultaneously supporting three low/full speed devices and one high speed device. The USB driver implementation can be simplify by not supporting split USB transfers. This will mean that a high speed hub cannot be used to connect low/full speed devices to the DUS500. The PS/2 ports of DUS500interface PS/2 interface514. The PS/2 interface514can be implemented using a standard FPGA PS/2 implementation.

The DUS500on powerup will enumerate attached USB devices1000and initialize attached PS/2 keyboard800and mouse900peripherals. Devices1000are also enumerated on insertion post powerup. DUS500assigns an address to the device1000and reads its descriptors to identify and configure itself for the specific device.

Standard “keyboard/mouse” report descriptors are stored (hard coded) on the DUS500to be made available to the DIP400when a connection is established. PS/2 keyboard/mouse commands are translated to USB format on DUS500.

Keyboard and Mouse data is made available to the OSD550when activated. When an OSD550is not activated all received keyboard and mouse data is discarded until a connection is established.

From the target device300perspective there is no mass storage device attached until a connection is made between DIP400and DUS500. This way DIP400does not have to dummy SCSI responses to the target device300until a connection is established. Because on making a connection the target device300is forced to re-enumerate, it will re-issue the keyboard status on enumeration. LED and Caps lock settings or data packets for a keyboard or mouse sent from the target device300will not need to be stored for sending to the DUS500when the connection is made.

USB peripherals1000can be inserted into any one of the USB ports including low/full speed HUBs412a. When DUS USB driver is not capable of handling split speed transactions (i.e. a mix of low/full and High speed devices can be inserted into the Hub, which will need to share one High speed link to the DUS), the insertion of high speed USB HUBs will not be available. For simplicity, support for HUBs on the DUS500can be excluded.

The exemplary system behavior with respect to USB Keyboard and Mouse connection control and data flow is illustrated inFIG. 11.

Before establishing a USB keyboard and mouse session the following conditions must be satisfied: DUS500and DIP400have been powered up, a USB keyboard and mouse have been enumerated by DUS500, DUS500has established the AVSP control session with DIP400, the network engines434and534have been enabled, and the keyboard and mouse session has been enabled on the session cert.

The DUS500sends the native report descriptors904for the enumerated keyboard and mouse to the DIP400over SSL/TCPIP using an extended AVSP command. Software550manages the transfer of keyboard and mouse data from USB controller512to network communication hardware530. Implementations may choose to transfer the actual keyboard and mouse reports with software550.

The DIP400receives the report descriptor904and forces re-enumeration of the composite keyboard & mouse USB port308, reporting the new report descriptor to target device300in the enumeration. This will ensure that the native peripheral keyboard or mouse reports can be transported from a peripheral to target devices300without translation. This minimizes the processing of keyboard and mouse data, and can facilitate hardware to assist in the implementation. Other than control of the session software440is not involved in keyboard and mouse report transfer. Power is not removed from the peripheral900when the DIP400forces the target device300to re-enumerate after a connection is established. This means that it is feasible to transfer device or report904from the DUS500and report to the target device300by the DIP400by the time of connection establishment. It also means that the second port (first port has keyboard, mouse, mass storage) can be used for generic HID devices that use the keyboard and mouse protocol, enabling the correct driver to be loaded on the target device300.

The DIP400having completed enumeration enables the keyboard and mouse data path. The DIP400will need to send the keyboard settings sent by target device300to DUS500, so that the DUS500can initialize the keyboard LEDs, Cap Lock, Num Lock, etc. settings to be consistent with target device300. The DUS500on receiving acknowledgement from DIP400of enumeration completion enables its keyboard and mouse path. Keyboard and mouse data now to flows between peripherals and the target device300.

If a keyboard or mouse was not inserted on the DIP400prior to a DUS-DIP connection being made, the peripherals are enumerated on insertion and the sequence described above is followed once the report descriptors904are known to DUS500. Removal of a keyboard or mouse does not result in an interaction with the DIP400, it simply sees no keyboard or mouse data. The DIP400only re-enumerates as described when a device is inserted. The DUS500monitors each key stroke for the OSD activation key sequence. When the OSD activation key sequence is detected, the OSD550is activated and all keyboard and mouse traffic received from peripherals on the DUS500is directed to the OSD550. When OSD550is dismissed, keyboard and mouse data flow on the DUS500is re-enabled.

DUS USB controller512must poll the device at the advertised rate. Typically max rate of every 8 ms for mouse/keyboard=125 transactions per second. Target device USB controller308must poll mouse and keyboard endpoints at the advertised rate. For example, 8 ms for mouse and 10 ms for keyboard may be advertised so as to never get a build up of packets. Alternatively a rate reported by the peripheral900to the DUS500, can be advertised.

The USB keyboard report modifier byte can be scanned by hardware to detect active modifier key presses. An implementation may choose to implement this functionality in hardware for efficiency. OSD hot keys on the system can be one or more of the available modifier keys (shift, ctrl, etc.) pressed two times in sequence.

It is critical that the latency in sending mouse reports904from the peripheral900to the target device300is minimized to optimize the mouse latency as observed by the DUS500user. Reducing software involved in the mouse900to target device300path will greatly reduce the latency in this direction.

However, there is a natural latency introduced by virtue of the USB interrupt transfer device polling mechanism. A typical USB mouse attached to a PC could see the worst case up to 8 to 10 ms (assuming 8 or 10 ms polling of mouse by PC) latency from peripheral to PC. When the present system is added to the path the following latency is added to the peripheral to PC path: latency in DUS500(<1 ms worst case), latency in network100(assume <0.5 ms worst case), latency in DIP400(<1 ms worst case), and target device300to DIP400polling interval (worst case 8 to 10 ms).

Thus, the overall worst case latency of 12.5 ms is added, assuming DIP400is advertising a 10 ms poll interval. The latency in the video path from target device300to monitor600needs to be added to get the overall round trip mouse latency.

The number of video frames per second affects mouse performance. As described in accordance withFIG. 8, video engine428contains an algorithm to automatically adjust the frames per second if the latency between the DUS500and DIP400exceed a specific value. A monitor600connected directly to a target device300using 60 frames per second, means a frame is updated every 16.66 ms. There could be a worst case latency of 16.66 ms from the time a target device300updating the mouse on the video and the video being displayed on the monitor.

When the present system is added to the path the following add example latency to the target device300to monitor300path: latency in DIP400(<1 ms worst case), latency in network100(assuming 60 fps through network <0.5 ms worst case), latency in DUS500(<1 ms worst case), DIP400to target device video based on 60 Hz monitor display, (worst case<=16.66 ms). Thus, the overall worst case additional latency on target device300to DUS500path is approximately 19 ms assuming video at 60 fps.

An overall worst case round trip delay added by the present system is 31.5 ms. This is: peripheral900to target device300path (worst case 12.5 ms)+target device300to peripheral900path (worst case 19 ms).

An overall average round trip delay added by the present system is 18 ms. That is: peripheral900to target device300path (average 7 ms)+target device300to peripheral900path (average 11 ms). 18 ms is comparable to the mouse latency added by prior art products.

The DIP400will have reported a standard system keyboard and mouse report format and polling rate to the target device300. Therefore, when an AVSP control channel is created between the DUS500and DIP400, no transfer of descriptor information is required.

The keyboard and mouse flows will need to be translated to the USB format on the DUS500, however hardware assist to be employed in the USB keyboard and Mouse flows through DIP400as for native USB peripherals.

Establishing the connection with the DUS500and DIP400can force the target device300to re-enumerate the USB port and will report the full set of descriptor information gleaned from the generic device on the DUS. This will cause the target device300to load the device specific driver and enable the device to operate. Such devices include joysticks, tablet based controllers, drawing tools, etc.

The exemplary system behavior with respect to PS/2 Keyboard and Mouse connection control and data flow is illustrated inFIG. 12.

Before establishing a PS/2 keyboard and mouse session the following conditions can be satisfied: DUS500and DIP400have been powered up, a PS/2 keyboard and mouse have been initialized by DUS500, DUS500has established the AVSP control session with DIP400, the network engines434and534have been enabled, and the keyboard and mouse session has been enabled on the session cert. The DUS500can configure the mouse report rate to a value that enables it time to process each key make and break code and translate to USB (i.e. every 20-25 ms or more).

The DUS500sends report descriptors to the DIP400over SSL/TCPIP using an extended AVSP command. The report describes the USB format into which the PS/2 keyboard/mouse data is translated to by the software540.

The DIP400receives the report descriptor and forces re-enumeration on the composite keyboard and mouse USB port, reporting the new report descriptor to the target device300in the enumeration. This will ensure that the keyboard and mouse report formats can be interpreted by the target device300. The DIP400having completed enumeration enables the keyboard and mouse data path. The DIP400will need to send the keyboard settings sent by the target device300to the DUS500so that the DUS500can initialize the keyboard LEDs, Cap Lock, Num Lock, etc. settings can be consistent with target device300. The DUS500on receiving acknowledgement from the DIP400of enumeration completion enables its keyboard and mouse path. Keyboard and mouse data now flows between peripherals and the target device300. Other than control of the session software440is not involved in keyboard and mouse report transfer.

Power is not removed from the peripheral when the DIP400forces the target device300to re-enumerate after a connection is established. This means that it is feasible to transfer device or report descriptors from the DUS and report to the target device300by the DIP400at the time of connection establishment. In the case of PS/2 this can be used to setup the DIP400correctly for PS/2 mouse report rate, report format.

PS USB HC must poll mouse and keyboard endpoints at an advertised rate. For example, 8 ms for mouse and 10 ms for keyboard may be advertised so as to never get buildup of packets. Alternatively a rate reported by the peripheral to the DUS500can be advertised, in this case every 25 ms. The keyboard/mouse USB port308(low/full) is re-enumerated using the report descriptors obtained from the DUS500.

If a keyboard or mouse was inserted on the DUS500prior to the DUS-DIP connection being made, the PS/2 peripheral is initialized on insertion and the sequences as described are followed once the report descriptors are known to the DUS500. Removal of a keyboard or mouse does not result in an interaction with the DIP400, it simply sees no keyboard or mouse data. The DIP400only re-enumerates as described when a device is inserted.

The exemplary system behavior with respect to Mass Storage (Virtual Media) connection control and data flow is illustrated inFIG. 13.

On power up the DIP400will have reported a standard Mass Storage bulk transfer only device to target device300. However, when an AVSP control channel is created between the DUS500and DIP400on establishment of a DUS-DIP connection, the target device300SCSI requests can be transported across the system to/from the peripheral.

Similar to the Keyboard and Mouse connection establishment, the opportunity exists to modify the descriptor reported by the DIP400by obtaining the report descriptor from the DUS500and forcing the target device300to re-enumerate the full/high speed port.

The system transfers the SCSI transactions without copying or translating the transactions. The standard Mass Storage PC device driver will use a common set of SCSI commands that all Mass Storage compliant devices support. This level of support can handle all the required functionality provided by memory sticks, however with CD/DVDs special functions like drawer open and CD selection functions would not be available.

However, reporting the actual peripherals report descriptor will mean that the actual real device driver will be loaded on the PC and the full functionality of the driver could be utilized, provided the system can handle the SCSI command set. This should be possible provided the implementation can “transport” SCSI commands with interpreting them (other than to extract length and direction indicators).

Before establishing a USB mass storage session the following conditions should be satisfied: DUS500and DIP400have been powered up, a mass storage device1000has been enumerated by DUS500, DUS500has established the AVSP control session with DIP400, mass storage session has been enabled on the session cert. The implementation simply transports SCSI commands, and does not need to interpret or interact/spoof with the target device300driver.

The DUS500sends the device and interface descriptors for the enumerated mass storage device1000to the DIP400using an extended AVSP command. The DIP400receives the report descriptors and forces re-enumeration of the full/high speed port310, reporting the new report descriptor to the target device300in the enumeration. The DIP400having completed enumeration enables the mass storage data path. The DUS500on receiving acknowledgement from the DIP400of enumeration completion enables its data path. Mass storage data now flows between peripherals and the target device300. All data is transferred by hardware under software control. If no mass storage device was inserted on the DIP400prior to a DUS-DIP connection being made, the mass storage device1000is enumerated on insertion and the sequence described above is followed once the report descriptors are known to the DUS500. Removal of a device results in the removal of the data path on the DIP400. The DIP400only re-enumerates as described when a device is inserted, as it must keep a device there to guarantee maintenance of power.

Software550does not copy or translate data SCSI media transfer packets. Other than initial connection establishment, software is not involved in the data transfer. Other than initial connection establishment, software450is not involved in the SCSI packet transfer.

The DUS500will require software involved in each 512 byte block of the SCSI transaction to facilitate transfer from the host controller512to the network engine534and from the network engine534to the host controller512, the data buffer shared by each side.

For the system to provide a good throughput 512 USB transactions of a SCSI transaction are to be grouped by hardware into a larger data block for each software interaction. The grouping should be such that software interactions are required not more than every 5 ms to 10 ms. 5 ms requires a shared buffer of 13.6 Kbytes for each side (HC, Network engine buffers are required to remain receiving at one side and transmitting at other side at the same time), i.e. 27.2 Kbytes {5/0.189=26.5 intervals (512 packets)=13.568 K transaction sizes required}. 8 ms requires a shared buffer of 21.5 Kbytes for each side (HC, Network engine), i.e. 43 Kbytes {8/0.189=42 intervals (512 packets)=21.504 K transaction sizes required}.

The buffering, as described, assumes a single mass storage device, adding another mass storage device will require twice the buffer space to maintain a similar throughput.

In this scheme the DIP400is not aware or SCSI level transactions. It is simply transferring data blocks from USB to network100and from network100to USB. The DUS500needs to have minimal awareness of the SCSI transactions. It needs to know the SCSI request type (CMD, Data, Status), its length, and if its an IN or OUT request so it can manage the HC USB scheduling correctly. The memory HC and network engine FPGA should be dual port shared memory.

The transfer of media between a DIP400and DUS500can occur within any of the following three modes of operation: extender, desktop and matrix. Regardless of the mode of operation, the DIP400is a slave device and has no awareness of the configuration it is operating in.

FIGS. 14-18illustrate the extender configuration. In the extender configuration a single DUS500and DIP400are present and a DUS500connects to a specific DIP400based on a direct physical connection—no login is required. The DUS500and DIP400establish media sessions between one another without the use of a trusted third party. Thus, a central authentication, authorization and administration engine, i.e. MgmtApp200, is not required in the extender configuration. Extender mode does not have the concept of users. In a point-to-point extender configuration all administration and configuration shall be achieved through the DUS OSD550or serial interface560. The OSD550on the DUS500shall enable user and administrator level configuration. The serial interface on the DUS500shall provide commissioning, general administration and debug capabilities.

The extender configuration is employed when a DUS500and DIP400are connected directly using a single cable, as shown inFIG. 14, or when the DUS500and DIP400are connected via a physical or virtual Ethernet LAN segment defined by MAC broadcast scope (e.g. an IP subnet), as shown inFIG. 15.

Once powered on, a DUS500automatically connects to a partner DIP400, where partner DIP400is a single DIP400directly connected to DUS500or a single DIP400on the IP subnet of the DUS500. DUS500-DIP400connections are over SSL authenticated connections. When used in a direct cable configuration the DUS500-DIP400association is defined and protected by the physical wire connection.

When used on a physical subnet, the implementation utilizes broadcast auto-discovery. The DUS500will only connect and remain connected to a DIP400provided one DUS500and one DIP400are present on the subnet. Otherwise, a DUS500will not connect to a DIP400(it enters desktop mode). If the implementation does not utilize auto-discovery, the DUS500will only connect to a DIP400configured with the same IP address to which the DUS500is configured to connect to. The DUS500can be configured to connect to any DIP400IP address. This IP address can be changed via the DUS500serial port.

Minimal administration is required in extender mode. The appliances will operate “straight-out-of-the-box.” That is, the administrator simply needs to install a DUS500and DIP400with default settings in an extender configuration as illustrated inFIGS. 14 and 15. If the appliances do not have default settings the settings can be reset via the DUS500serial port560.

An administrator installs the appliances by connecting peripherals to the DUS500, connecting DUS500to an Ethernet 10/100/1000 cable, connecting DIP400to the cable, connecting DIP400to target device300, powering on the target device300which powers up the DIP400via USB ports, and powering up the DUS500.

Once the DIP400is powered on, it initializes itself by using its factory default IP addressing mechanism (e.g. DHCP) to search for an IP address. In extender mode no DHCP server is available. Once a DUS500is powered on, it displays an initialization screen and initializes itself using its static IP address data.

Once DUS500and DIP400are installed, the DUS500begins the process of establishing a connection with the DIP400. This process is illustrated inFIG. 16. In the exemplary embodiment, the DUS500and DIP400utilize AIDP (Avocent Install Discovery Protocol) broadcast discovery. An IP configuration protocol (e.g., ASMP (Avocent Secure Management Protocol)) is used by the DUS500to obtain DIP400information. The DUS500looks for a partner DIP400by broadcasting AIDP discovery packets. The DUS500will remain searching until it finds a DIP400or DUS500, sending a broadcast every second. When an initialized DIP400receives an AIDP discovery protocol request it responds by identifying itself. This allows the DUS500to know of its existence. When a single DIP400is detected, the DUS500uses the AIDP protocol to program it with the IP address it has for the DIP400and continues to search for other DUSs and DIPs but at a slower rate (e.g. every 10 seconds). The DIP400stores the IP data persistently and configures itself to use static IP addressing. When DIP400is inserted into a network100it stays quiet and only responds to requests. The exception being SNMP traps if they are enabled. In this case, the DIP400will send an SNMP trap on startup to the configured destination address.

Once a single DIP400is detected and programmed with an IP address, the DUS500verifies that the DIP400is compatible and configures the DIP400by establishing an ASMP session with the DIP400. The persistent session certs on the DUS500and DIP400are used to authenticate an ASMP SSL session. The DIP400receives and responds to ASMP requests for version and other information from DUS500. The DUS500will obtain the DIP400revision information using the ASMP protocol and verify it is compatible. The DUS500may choose to configure specific parameters on the DIP400using ASMP. The DIP400may receive and respond to ASMP requests to configure data. Once the configuration procedure is complete, the DUS500closes the ASMP session with the DIP400and establishes an AVSP control SSL connection with the DIP400using the AVSP protocol as described in accordance withFIG. 6.

The DUS500activates AVSP keep alive transmission/reply checking on the session control connection between DUS500and DIP400. The DUS500configures and enables its media sessions as indicated in the session cert. The DIP400configures and activates its media streams on detection of an AVSP keep alive response from the DIP.400. The DIP400responds to requests to establish an AVSP session control SSL connection, exchanging the session cert maintained persistently on the DIP400. When the DIP400receives an AVSP keep alive on the session control connection, it configures its media sessions as indicated in the session cert received from the DUS500(the DUS500will enable its media streams on reception of the keep alive response). The DUS500sends a keep alive response to the DUS500. The DUS500user is now connected to the target device300.

If more than one DIP400or another DUS500is detected at anytime the DUS500resets open associations it has and enters desktop mode. Until the DUS500has been explicitly provisioned to be in desktop mode, it will always attempt to start in extender mode on power cycle.

If the AIDP discovery procedure after finding a DIP400, misses three consecutive discovery replies from the DIP400it assumes the DIP400is missing and enters the fast discovery mode to either detect the DIP400returning or detect a new DIP400having been inserted. If AIDP discovery procedure having found a DIP400, finds a subsequent discovery response indicates a DIP400with a different MAC address (i.e. DIP400has been switched), the discovery procedure initiates reset of all open associations the DUS500and restarts this procedure at the point when a single DIP400is discovered. An implementation may choose not to use the AIDP discovery protocol. An implementation may choose not to use ASMP for retrieval of DIP400information by the DUS500. Extensions to AVSP can be used to retrieve the required information on connection establishment.

FIG. 17illustrates the architectural behavior and broad protocol usage for a DUS500and a DIP400in an extender mode configuration. When a connection is established, the DUS500needs to be able to accommodate the possibility of a target device300being reset or the DIP400being replaced. When this occurs, the connection needs to be re-established rapidly.

The DUS500shall complete the powerup sequence within 20 seconds, including any self test diagnostics it may run. The DUS500shall during powerup keep the DUS500user informed of its activities via an OSD message display on monitor600. The DUS500and DIP400each shall have a persistently stored session cert (factory default shipped with product) for use in extender mode. The certs are used in SSL link establishment. DUS500session cert shall have media session property extensions that are used to determine the media session and session properties to connect. At power up DUS500always responds to AIDP discovery requests irrespective of operating mode.

When the target device300is powered down, resulting in abrupt power down of the DIP400. The DUS500stops receiving AVSP keep alive replies (every 500 ms). After missing three consecutive replies the DUS500declares the connection with the DIP400broken. The DUS500resets its open SSL/TCPIP associations with the DIP400(media streams and AVSP control session), and displays a message indicating that the connection is broken with the DIP400. The DUS500places the AIDP discovery procedure into fast detection mode (1 second interval instead of 10 seconds). When the DUS500AIDP procedure detects a DIP400again, it indicates if it is the same DIP400that was lost (based on MAC address). If the same DIP400is discovered the DUS500follows the DUS500powerup usecase sequence after the point where the ASMP session is closed. That is, the DUS500re-establishes the AVSP control session with the DIP400and the media session are re-established.

If AIDP discovery procedure having found a DIP400, finds that a subsequent discovery response indicates a DIP400with a different MAC address (i.e. DIP400has been switched), the discovery procedure initiates reset of all open associations with the DUS500(as in the case they may already be reset if another application detects problems communicating with a DIP400more quickly) and restarts at the DUS500at the point after a single DIP400is discovered for the first time. The implementation may choose not to use the AIDP auto discovery and IP configuration protocol.

On restart of the target device300and the subsequent DIP400powerup the time it takes to establish a connection on the DUS500and DIP400again needs to be fast enough to be able to have a DUS500user access the target device bios. This concern is mitigated by the fact that if the same DIP400is re-discovered (having been lost) the connection re-establishment is immediate.

When a connection is established, it is possible that the DUS500is powered down. On power down of a DUS500a connection present is effectively removed totally and must be re-established in full. The DIP400on detecting loss of a connection with a DUS500, clears any open associations with the DUS500and readies itself for another connection attempt. This process is as follows:

The target DUS500is powered down, resulting in abrupt power down of the DUS500. The DIP400stops receiving AVSP keep alive requests (every 500 ms). After missing three consecutive requests the DIP400declares the connection with the DUS500broken. The DIP400resets its open SSL/TCPIP associations with the DUS500(media streams and AVSP control session). The DIP400is now ready to accept another AVSP control session establishment request (re-establish the connection again). An implementation may choose not to use ASMP between DUS500and DIP400instead using an AVSP extension to retrieve data from the DIP400.

Once a connection is established a cable fault breaking communication in both directions is also possible. A network or cable fault will result in the DUS500missing keep alive responses and the DIP400missing keep alive requests. The DUS500behaves as indicated when the DIP400is powered down. The DIP400behaves as indicated when the DUS500is powered down. On resumption of the network/cable connectivity, the DUS500will re-discover the DIP400within one second (the fast discovery interval). The AIDP discovery procedure recognizes that it's the same DIP400(MAC address returned in discovery response) and re-establishes the AVSP control connection and associated media sessions with the DIP400.

A network/cable fault that breaks communication in the DUS500to DIP400direction will cause the DIP400to behave as indicated in when the DUS500is powered down. A network/cable fault that breaks communications in the DIP400to DUS500direction will cause the DUS500to behave as indicated when the DIP400is powered down.

Another aspect of the extender configuration is the capabilities of a DUS500with respect to items presented on the OSD (on-screen display)550. All items presented on the OSD550are assumed to be modifiable by the DUS500. Administration items are carried out through the password protected serial port560. The DUS500user hits the OSD550hotkey sequence, after which the OSD550is displayed and all keyboard and mouse transactions are directed to the OSD550. The user can navigate the OSD550as required. When a connection is made, OSD550shall, as well as providing the capability to view and perform actions locally on the DUS500, provide the ability to view and perform actions that require data retrieval or modification on the DIP400. The ASMP protocol is used to view version and addressing data on the DIP400. The AVSP protocol is used to modify the characteristics of the media sessions. Implementations may choose not to use ASMP, instead using an extension to AVSP.

When the serial interface560is password protected, the administrator inserts a password and is presented with a serial menu from which configuration can be achieved. Menu options include setting IP address, download of image file via xmodem, firmware upgrade of DUS500, firmware upgrade of DIP400, etc. Hidden menu options will exist for manufacturing development and tech support debugging purposes.

FIG. 18is an exemplary functional model of extender configuration.FIG. 18summarizes operation of DUS500and DIP400in extender mode as previously described. As shown inFIG. 18for the exemplary embodiment, DUS500discovers DIP400using AIDP, DUS500administers a DIP400using ASMP, DUS500and DIP400establish a control channel using AVSP, and media sessions are created. Further, as shown inFIG. 18DUS500can use WOL (wake on LAN) to power on a target device300when an appropriate network connection is established. This process is described in greater detail in accordance with the desktop/matrix configuration.

FIG. 19illustrates an exemplary desktop/matrix configuration. As shown inFIG. 19, any number of DUSs500and any number of DIPs400are connected to a local area network100a. As such, a particular DUS500and DIP400establish media sessions between one another with the use of a trusted third party, the Management Application (MgmtApp)200connected to local area network100a. Further, target devices300and local area network100aare connected a wide area network100b. A management client1100and network1200are also connected to wide area network100b.

Desktop and matrix modes both involve a user logging into a DUS500and connecting to a DIP400. The difference between the two modes is that in desktop mode the user will automatically be connected to a predetermined DIP400. Matrix mode is similar to desktop mode except that when a user logins to the DUS500, a user obtains a list of target devices300(or DIPs400). The user then selects a target device300and a specific connection is established. Login and Auto login modes are available in both modes.

The MgmtApp200provides the following functions: administration, authentication, and authorization. The core MgmtApp200administration functions include the ability to: administer a database of target device users, system administrators, and administer appliances (e.g. DUSs500and DIPs400). The MgmtApp200can configure appliances by: upgrading/downgrading appliance firmware versions, configuring appliance addressing information, configuring the login mode used by the DUSs500(i.e. Login or Auto login). MgmtApp200can also administer user access session data. This includes the DUSs a specific user is allowed to login from, the operating mode to be used when accessing from a specific DUS (e.g. desktop or matrix), the media sessions allowed for a specific user (e.g. video, vMedia, keyboard/mouse, or audio). Further, an administrator can use MgmtApp.200to enable and disable media sessions on a per user basis. An administrator can set the maximum media session properties allowed for a specific user. An administrator can set the maximum quality and performance experience required by a user. This provides a mechanism to manage the network bandwidth utilized by users. The following are configurable by an administrator to manage network bandwidth: video properties (e.g. frames per second and color depth), vMedia properties (the ratio of bandwidth usage between the high bandwidth users video and vMedia) and audio properties (e.g. The jitter/playback buffer size). The ratio of bandwidth usage setting effectively guarantees a specific minimum percentage of the bandwidth for Video. The jitter/playback buffer size is a characteristic of network latency rather than bandwidth.

The MgmtApp200provides authentication of: administrators access to the MgmtApp200through a web browser (i.e. Mgmt Client1100access) and target device user access to a DUS500. Target device user access to a DUS500authentication includes: Internal Authentication (MgmtApp200authenticates users/passwords) and External Authentication through the use of a third party authentication servers (LDAP, Active Directory, etc).

The MgmtApp200authorization functions include: authorizing system administrators and DUS-Target PC user system access rights, and media sessions definition and connectivity.

The general system level principles of desktop/matrix configurations are described by describing the following usecases: (1) Installation/Administration which includes: Appliance Installation/Administration (described in accordance withFIG. 20), target device administration (described in accordance withFIG. 21), and user administration, (2) Power-up of installed DUS and DIP (described in accordance withFIGS. 23 and 24, respectively) (3) Connection Establishment and Removal (FIG. 22), (4) Power-down of installed DUS and DIP, and (5) Cable or network fault.

The initial installation and commissioning of appliances to bring the system to a state ready to establish media sessions between a DUS500and DIP400is described in accordance withFIG. 20.

For the initial installation it is assumed that DUSs500and DIPs400have default factory settings and DUSs500by default have login enabled and are connected according toFIG. 19. It is also assumed that explorer software that includes MgmtApp200is installed on a dedicated server. The MgmtApp200is used to setup a standard configuration that needs to be performed on each newly discovered appliance (set SNMP trap address, set management IP address, etc.).

The MgmtApp200discovers DIPs400and DUSs500on the network100using one or more of the following methods: automatically using AIDP broadcasts to the local subnet, automatically using directed AIDP broadcasts to specified remote subnets, or manually by entering the specific a DIP400or DUS500hostname or IP address. If static IP addressing is required, MgmtApp200is used to configure static IP address data on discovered DIPs400and DUSs500. The AIDP protocol is used to achieve this.

The MgmtApp200establishes an ASMP session with each of the discovered DIPs400and DUSs500to “commission” the devices and add them to its database. MgmtApp200verifies that the DIPs400, DUSs500, and explorer software versions are compatible (flagging incompatible appliances) and auto configures DIPs400and DUSs500, if auto configuration items are setup. The MgmtApp200closes the ASMP session with the DIPs400and DUSs500on completion of “commissioning.”

The MgmtApp200polls DIPs400and DUSs500in its database to determine availability (SNMP/UDP poll of MIB variable). The MgmtApp200listens for SNMP traps to augment its polling of DIPs400and DUSs500to determine availability. The DIPs400and DUSs500can now be configured, administered, and upgraded using ASMP. The appliances are ready for connections to be established.

DUSs500can alternatively be locally configured via their serial ports560to be in desktop mode. DUS500appliances can be configured for autologin access. When a DUS500is configured for autologin, the MgmtApp200adds the DUS500specific “autologin” user to its database. The user is administered on the MgmtApp200just like other users. The DUS500resets itself on setting to autologin mode, and continually attempts to login on power up.

The DUS500and DIP400hostnames can be modified via the ASMP MIB interface. The factory default host names are “DUS xxxxxx” and “DIP xxxxxx,” where xxxxxx is the appliance MAC address. The default IP addressing mechanism used by the DUS500in Desktop/Matrix mode is DHCP, unless configured to operate using static IP addressing. The default access mode of a DUS500is login. A DUS500login access mode (login/auto login) can only be configured via its ASMP interface. The username and password to be used in autologin mode can be configured on the DUS500via its ASMP interface only.

After DIPs400and DUSs500have been installed and commissioned as described in accordance withFIG. 20, the MgmtApp200must associate each DIP400with a target device300. Target devices300are the primary reference used by administrators. At a user interface level the DUS500user is associated with one or more target devices300. Target device300host names can be automatically or manually associated with DIPs400by the administrator. To automatically associate a DIP400and target device300, the MgmtApp200can automatically discover target devices300that have DIPs400attached and by using a unique identifier of the DIP400and target device300(MAC addresses) and auto associated the two. The auto association procedure can be run on installation and run at defined intervals subsequently. The administration of the target device300is described in accordance withFIG. 21.

It should be noted that it is assumed that auto discovery using WMI will detect target devices300using a Microsoft Windows OS. On starting the target device300discovery GUI, the administrator tells the GUI where and how to discover target devices300by entering: the subnets to search and the Microsoft Domain to search. For each subnet specified, MgmtApp200polls each IP address using WMI query to detect target devices300. For each Microsoft Domain specified, MgmtApp200queries the Domain controller for the list of target devices300. The MgmtApp200searches for target devices300and queries target devices300using WMI for the following matches: a USB device with an appropriate Vendor ID is attached and for devices with an appropriate Vendor ID, a device ID that identifies it as a DIP400. Target devices300with DIPs400attached, have the following information retrieved and stored in the target device list on the Mgt. App.200: PC hostname and MAC address, PC MAC address (required for remote wakeup), PC OS type (can be used for auto logout marco selection etc.), and DIP MAC address. The MgmtApp200looks up the DIPs in its database and matches it with the MAC address reported from the target device300. The DIP400is then automatically associated with the target device300and stored in the MgmtApp200database. The OS type can be used to select the login and logout macros and have them provisioned on the DIP400using ASMP. The MgmtApp can at a preconfigured time or specified intervals repeat the auto discovery procedure to maintain an updated view of target device300and DIP400associations. Changes in the DIP-Target device associations are flagged to the administrator.

FIG. 21also shows that the administrator can manually associate a DIP400with a target device300by using the manual target device add GUI. This takes place entirely on the MgmtApp200. The administrator enters the target device host name, and selects the DIP400from the list of DIPs in MgmtApp200database.

Once target device administration occurs, user administration can occur. For user administration in the exemplary embodiment the database of users and their associated configuration data is maintained in MgmtApp200database and DUS500and DIP400do not store user information. Further, the following description describes a user is being configured for internal authentication in desktop mode.

The first step of user configuration is the administrator configuring the password policy. This includes configuring: required password fields (numeric, capital, etc), minimum password length, and password expiry. The administrator then creates a user account. The administrator then configures the user as being internally authenticated. The administrator configures the following connection associated information: DUSs500from which the user is allowed to login from, the mode used when logging in from specific DUS500is set as desktop. The desktop mode data is configured for the user. Desktop mode data includes: a primary target device selected from list of known target devices, and secondary target device selected from list of known target devices. After the administrator completes these steps, a user is ready to participate in connections.

It should be noted that in alternative embodiments a user can be configured as using an external authentication service. It should be noted that althoughFIG. 21describes configuration for desktop mode matrix mode can also be configured. Further, in the exemplary embodiment the database can contain up to 200 user accounts.

The desktop mode login and associated connection is described in accordance withFIG. 22.

Prior to login it is assumed that necessary Installation/Administration steps have occurred. In addition the following description assumes that DUS500is powered down and DUS500used DHCP. As shown inFIG. 22, the DUS500user powers up the DUS500. The DUS500displays an initializing OSD message or message sequence to inform the user of powerup progress. The DUS500configures its IP address using DHCP. The DUS500can now respond to the MgmtApp200SNMP polls (to determine availability). The DUS500sends an SNMP “cold boot” trap to the configured trap destination IP addresses (at least one being the MgmtApp200). The DUS500displays OSD login screen. The DUS500user submits username and password. The DUS500establishes an ASMP session with the MgmtApp200and submits the username and password for authentication using the ASMP login messages. The MgmtApp200authenticates the username and password. The MgmtApp200verifies if the user is allowed login from the DUS500. The MgmtApp200looks up the mode the user is configured to use for this DUS500(assumed desktop in this usecase). The MgmtApp200reads the primary target device300and looks up the associated DIP400. If the DIP400is marked as unavailable an attempt to wake up the target device300using WOL protocol is attempted threes times. The MgmtApp200opens an ADSAP2 session with the DUS500and DIP400. The MgmtApp200looks up the media session properties configured for the user. The MgmtApp200builds an X.509 Cert for the Session. One for the DUS500and one for the DIP400. The cert contains amongst other items: DUS/DIP IP address, Session Retry Timeout value, Session Retry Count, Media Session Properties. The MgmtApp200writes the cert to the DIP400and DUS500using ADSAP2 protocol. The MgmtApp200closes the ADSAP2 session. The DIP400stores the Session Cert persistently, so that it can accommodate fast re-connection on power cycling/reset on the target device300. The MgmtApp200sends an ASMP Login reply with successful status to the DUS500. The DUS500on receiving the successful login response, informs the user and establishes an AVSP control channel with the DIP400exchanging X.509 session certs. On establishment of the AVSP control channel, the DUS500and DIP400configure their respective media transfer hardware sessions, but do not enable them yet. The DUS500starts a fast keepalive request on the channel. The DIP400on receiving the keepalive from the DUS500enables its media streams. The DUS500on receiving the initial keepalive reply enables its media streams. The media session is now established.

It should be noted that DUS500may have been powered up already and have been used for previous connections. In this case, to establish a connection the DUS500user brings up the DUS500OSD and logins from there. If DUS500is configured for Auto login access, then the login OSD is not displayed, the configured autologin username and password is submitted by the DUS500to the MgmtApp.200.

If there was an error the MgmtApp200returns a login response indicating the error to the DUS500. The DUS500displays the appropriate message to the user. In the event of a connection failure, the DUS500will attempt to re-establish the AVSP control channel with the DIP400for the “Timeout Period” indicated in the cert. If the attempt fails it will retry the connection attempt (after a back off for the same time period), the number of times it retries is indicated in the “Session Retry Count.” An unavailable DIP400, or failure to open an ADSAP2 session with the DIP400associated with the target device300, the MgmtApp200will check if the target device300responds to a poll, if not it sends a WOL packet to the target device300and will make three attempts to wake up the target device300, if the DIP400is unresponsive after three attempts the secondary target device is attempted. The DUS500is informed of progress with an ASMP login reply (indicating status and that login is still in progress). AVSP keepalive are sent by the DUS500every 500 ms.

The procedures involved with power up of a DUS500are illustrated inFIG. 23. InFIG. 23, it is assumed that DUS500has been installed and commissioned as previously described. It is also assumed that the DUS500has been configured for desktop mode with login access and the DUS500uses the default DHCP IP addressing mode.

As shown inFIG. 23, the process begins by a DUS500user powering up the DUS500. The DUS500displays an initializing OSD message sequence to inform the user of powerup progress. The DUS500initialization completes and searches for an IP address through DHCP. The DUS500configures its IP address data. The DUS500can now respond to MgmtApp200SNMP polls (to determine availability). The DUS500sends an SNMP “cold boot” trap to the configured trap destination IP addresses (at least one being the MgmtApp200). The MgmtApp200can update its availability status for the DUS500. The DUS500displays the OSD login screen. The DUS500may receive and respond to ASMP requests to configure or retrieve data. The DUS500is ready to initiate login and subsequent connection establishment. It should be noted that if DUS500is configured for autologin, then the login OSD is not displayed.

The behavior on power up of the DIP400is described in accordance withFIG. 24. InFIG. 24, it is assumes that the DIP400has been installed and commissioned as described previously described. The DIP400is a slave device and has no awareness to the configuration it is operating in (e.g. extender or desktop/matrix). The DIP400uses the default DHCP IP addressing mode.

As shown inFIG. 24the process begins by target device300being powered up. The DIP400powers up as soon as it has obtained power from the target device300via its USB ports. The DIP400initialization completes and searches for an IP address through DHCP. The DIP400configures its IP address data. The DIP400can now respond to MgmtApp200SNMP polls (to determine availability). The DIP400sends an SNMP “cold boot” trap to the configured trap destination IP addresses (at least one being the MgmtApp200). The MgmtApp200updates its availability status for the DIP400. The DIP400may receive and respond to ASMP requests to configure or retrieve data. The DIP400is ready to accept connection requests.

It should be noted that the DIP400must be able to powerup and be in a state to let the target device300know that a keyboard is present before the target device300determines that it cannot see a keyboard. Further it should be noted that the DIP400may need to have an independent power supply as it may draw too much current to be supplied through two USB port connections.

The system needs to be able to accommodate the possibility of a target device300being reset, requiring rapid connection re-establishment or the possibility of the DIP400being replaced. The following description describes what happens when a target device300is power cycled when a DUS500and DIP400are connected a desktop or extender configuration.

The target device300is powered down (or reset), resulting in abrupt power down of the DIP400. The DUS500stops receiving AVSP keepalive replies (every 500 ms). After missing three consecutive replies the DUS500declares the connection with the DIP400broken. The DUS500resets its open SSL/TCPIP associations with the DIP400(media streams and AVSP control session), and displays a message indicating that the connection is broken with the DIP400. If the session cert “Session retry Count” is non zero, the DUS500will backoff for a time indicated by the Session Cert “Session retry timeout” value and then attempts to re-establish the AVSP control session with the DIP400. The DUS500will keep retrying the establishment of the AVSP control session for the “Session Retry Timeout” period, after which it will terminate the retry attempt. On terminating each retry attempt the DUS500sends an SNMP trap informing of the failed connection attempt. The DUS500, after waiting for a period equal to the “session retry timeout” commences retrying the establishment of a connection with the DIP400. The cycle repeats until the “Session Retry Count” have been reached or the AVSP control session has been established. When the AVSP control session has been established, the connection setup completes as described in the DUS login/connection usecase after the AVSP session is established.

It should be noted the if a DIP400had been replaced, the SSL authentication with the DIP400would fail and any attempt to re-establish the connection by the DUS500terminates and a message is displayed to inform the DUS500user. An SNMP trap is sent to the MgmtApp200informing it of the connection termination.

On restart of target device300and subsequent DIP400powerup the time it takes to re-establish a connection on the DUS500and DIP400should be fast enough to enable a DUS500user to access the target device BIOS.

The following description describes system behavior when a DUS500is power cycled when a connection is established. The DUS500is powered down, resulting in abrupt power down of the DUS500. The DIP400stops receiving AVSP keep alive requests (every 500 ms). After missing three consecutive requests the DIP400declares the connection with the DUS500broken. The DIP400resets its open SSL/TCPIP associations with the DUS500(media streams and AVSP control session). The DIP400sends an SNMP trap to the configured trap destination addresses (at least one being the MgmtApp200) indicating that the connection has terminated abruptly. The DIP400is now ready to accept another AVSP control session establishment request (re-establish the connection again).

The following description describes the behavior of DUS500and DIP400in the event of a cable or network failure when a connection exists. For this description, it is assumed that the network or cable fault breaks communication in both directions.

A network or cable fault occurs. The DUS500misses keepalive responses. After missing three consecutive responses, it will clear the open association with the DIP400, and commences retrying the establishment of the AVSP control session. The DIP400misses keepalive requests, after missing three consecutive requests the DIP400will clear open associations so it can facilitate the re-establishment of the connection with the DUS500. On resumption of the network/cable connectivity the DUS500will re-establish the AVSP control channel with the DIP400provided the network/cable fault was repaired before the DUS500“session retry” count was reached.

It should be noted that a network fault that has been recovered before the DUS500detects three consecutive missing keepalive replies (approx 1.5 seconds) will be tolerated without any interruption, other then any network level TCPIP retries that may occur. A network or cable failure that results in the DUS500having exhausted its session retry count will result in DUS500terminating the connection attempts. The DUS500will inform the DUS500user by displaying a message and send an SNMP trap to inform the MgmtApp200. Similarly, the DIP400will have closed its open associations with the DUS500and informed the MgmtApp200of a failed connection. A network/cable fault break in communication in the DUS500to DIP400direction will cause the DIP400to behave as indicated in the DUS500power-down use case. A network/cable fault break in communication in the DIP400to DUS500direction will cause the DUS500to behave as indicated in the DIP400power-down usecase.

FIG. 26is a diagram illustrating the interoperability between prior art KVM switching products and DUS500and DIP400. An example of prior art KVM switching products are Avocent DSR products. These products are described in submitted document entitled “DSR Switch Installer/User Guide” published by Avocent Corporation in 2005, Document No. 590-419-501B, which is incorporated by reference in its entirety. In this system, software allows digital user1400to access the switching system1300over network100. The communication protocols used for DUS500and DIP500are different from those used the prior art system. However, a digital user1400will be able to access a DIP400by modifying software on the digital user station or by inserting a proxy server (not shown) between digital user1400and DIP400. Likewise DUS500will be able to access the switching system1300by modifying the software installed on the switch in the switching system1300or by inserting a proxy server (not shown) between DUS500and switching system1300. Further, DUS500and DIP400will be able to access any prior KVM system when a prior art communication protocol is able to be converted into a format acceptable to DUS500and DIP400. For switch products with “local port” switching access DIP400can be modified to support seamless operation. This seamless operation allows a DIP400to “present” as a list of devices rather than a single device—thus a user can select which device to connect to and the DIP400plays out the local switch operation for the attached KVM switch. It should be noted that DUS500and DIP400can be coexist on a network100with prior art switching systems with out interference.