Flexible cross-connect with data plane

A flexible cross-connect with a data plane is presented which allows the establishment of connections between network interfaces at any network interface card to another network interface on any other network interface card. The system can cross-connect connections at an STS-1 and VT 1.5 granularity, and allows the switching and routing of information in a data plane without the use of the cross connect fabric. This permits routing, bridging, and concentration of data services to be performed without burdening of the cross connect. For reliability, a range of protection configurations can be employed including 1:1, 1:5 and mixed 1:N protection. A backplane is used which provides point-to-point traces between each card and the cross connect unit, between each card and a timing, communications and control unit, and between the network interface cards themselves.

BACKGROUND OF THE INVENTION

Telecommunications systems are carrying increasing amounts of information, both in the long distance network as well as in metropolitan and local area networks. At present data traffic is growing much faster than voice traffic, and will include high bandwidth video signals. In addition to the requirement for equipment to carry increasing amounts of telecommunications traffic there is a need to bring this information from the long distance network to businesses and to locations where it can be distributed to residences over access networks.

The equipment which has been developed to carry large amounts of telecommunications traffic includes fiber optic transport equipment which can carry high speed telecommunications traffic. The data rates on fiber optic systems can range from millions of bits per second (Mb/s) to billions of bits per second (Gb/s). In addition, multiple wavelengths of light can be carried on an optical fiber using Wavelength Division Multiplexing (WDM) techniques.

The ability to carry large amounts of telecommunications traffic on an optical fiber solves the long-distance point-to-point transport problem, but does not address the issue of how to add and remove traffic from the high-speed data stream. Equipment for adding and removing traffic has been developed and is referred to as “add-drop” multiplexing equipment.

Traditional designs for add-drop multiplexers are based on the use of multiple interface cards which receive high speed data streams, create a time division multiplex signal containing the multiple data streams, and route the time division multiplex signal to a cross-connect unit which can disassemble the data streams, remove or insert particular data streams, and send the signal to another interface card for transmission back into the networks. By aggregating the multiple data streams into a time division multiplexed data signal, the data rate of the time division multiplexed signal is by definition several times the rate of the maximum data rate supported by the interface cards. Such solutions have proven adequate for interface data rates in the range of 155 Mb/s to 622 Mb/s, but for data rates over 1 Gb/s there are a number of problems which arise due to the transport of and timing of the multiplexing and transmission of the high speed signals between cards in the cross-connect. Optical signals of 2.4 Gb/s have become a standard and there is a need for cross-connect equipment which can support multiple 2.4 Gb/s data streams.

Standardized interfaces and transmission hierarchies for telecommunications signals have been developed and include the pleisochronous digital hierarchy (PDH) standards, the Synchronous Digital Hierarchy (SDH) standards, and the Synchronous Optical Network (SONET) standards. In addition to these telecommunications transport standards and systems data standards and systems have been developed for interconnecting businesses and computers within businesses. These metropolitan and local area network (MAN/LAN) standards include Ethernet, Gigabit Ethernet, Frame Relay, and Fiber Distributed Data Interface (FDDI). Other standards, such as Integrated Services Digital Network (ISDN) and Asynchronous Transfer Mode (ATM) have been developed for use at both levels. Although individual pieces of equipment can be purchased to support telecommunications or MAN/LAN standards, these devices generally either connect data streams using a signal protocol or convert entire data streams from one protocol to another. There is a need for a device which can establish interconnectivity between interfaces at the MAN/LAN level, while providing cross-connection to the telecommunications PDH/SDH/SONET network.

Multiple interfaces are presently supported in cross-connect equipment by the use of different interface cards. For high-speed signals, these cards must be inserted into particular slots in order to insure that the signal can be transported between the interface card and the cross-connect unit and to another interface card. It would be desirable to have a cross-connect system in which cards can support high-speed optical signals of at least 2.4 Gb/s in any card slot.

It would also be useful to have a system which would support routing, bridging, and concentration functions within MANs/LANs, as well as permitting access to the telecommunications PDH/SDH/SONET network.

Because of the high data rates and amount of traffic carried in the telecommunications signals, it is necessary to insure that there are redundant interface units in the cross-connect, and that a protect interface card can be used if a working interface card fails.

For the foregoing reasons, there is a need for flexible cross-connect with a data plane that can support multiple high speed optical interfaces in any card slot, can establish connectivity between data cards and the transport network and which provides adequate protection against failed units.

SUMMARY OF THE INVENTION

The present invention provides a flexible cross-connect architecture with a data plane based on use of interface cards which are inserted into card slots connecting to a backplane which provides point-to-point connectivity between each card and centralized cross-connect and timing, communications, and control units. The cross-connect unit can establish connections between any interface card and any other interface card, or between an interface card and itself.

A star backplane is utilized in which point-to-point connections are established between network interface cards and common cards including a cross-connect card and redundant cross-connect card, and a timing, communications and control card and redundant timing, communications and control card. In addition, the star backplane supports point-to-point connections between the network interface cards, allowing the creation of a data plane which does not require use of the cross-connect to route data.

In a preferred embodiment the interface cards support a variety of data and telecommunications interfaces including SONET OC-192 interfaces operating at 9.95 Gb/s. The point-to-point connections between the interface cards and the cross-connect operate over a parallel 32 bit data bus, operating at 311 MHz and supporting transport of STS-192 payloads. In an alternate embodiment a limited number of card slots support STS-192 data rate connections to the cross-connect, while all of the card slots support STS-48 connections to the cross-connect.

The present invention utilizes a backplane which supports direct connections between interface cards, allowing for the creation of a data plane in the form of a fully or partially connected mesh. One advantage of the data plane is that signals can be routed between interface cards without use of the cross-connect in order to realize bridging, routing, and other MAN/LAN functions without encumbering the cross-connect.

Another advantage of the data plane is that traffic signals can be aggregated in the data plane and routed to the telecommunications network. As an example, Ethernet data can be aggregated on one or more interface cards which form part of the data plane. The aggregated traffic can be used to fill a DS-3, STS-1 or other signal which forms part of a SONET channel. The cross-connect can insert the aggregated signal into a higher level SONET signal for transport on the telecommunications network. This feature allows for the cost effective use of the equipment and alleviates the need for a customer to lease an expensive high speed optical signal for a limited amount of data traffic.

The present invention supports a cross-connect unit, a control unit, a plurality of interface cards, and has a plurality of interface card slots which are connected to a backplane. The backplane establishes point-to-point connections between the interface cards and the control unit and between the interface cards and the cross-connect such that any signal from an interface unit can be cross-connect with a signal from another interface unit, independent of the slots in which the interface units are located.

In a preferred embodiment a variety of interface cards are used to support electrical connections including Ethernet, ATM, PDH and SDH rates, as well as optical connections at rates up to STS-192. In a preferred embodiment any interface card can be located in any interface card slot and signals from a card can be cross-connected with any other signal including a signal from that card itself.

In an alternate embodiment optical connections of up to OC-48 are supported in any interface card slot, and optical connections of OC-192 are supported in particular slots.

An advantage of the present invention is that multiple SONET rings can be supported from one piece of equipment, since cross-connections can be established between separate rings at the cross-connect.

In a preferred embodiment cross-connection is performed at high speed by pre-aligning signals on the interface cards to create a frame aligned signal which arrives at the cross-connect. Pre-aligning the signal can be accomplished through the use of a programmable offset located on each interface card and controlled by a central timing, communications, and control unit.

A feature of the present invention is the ability of the cross-connect to break the signal down to its lowest common denominator to subsequently perform the cross connection. In a preferred embodiment the cross-connection is done at a VT 1.5 level while in an alternate embodiment the cross connection is performed at the STS-1 level.

A feature of the present invention is the ability to protect against failed interface cards (electrical protection). In a preferred embodiment this is accomplished by establishing connections on the backplane which connect each card with an outwardly adjacent card, as well as providing connectivity to a designated protect card. Traces are established on the backplane which permit the system to be configured for 1:1, 1:5 or simultaneous 1:2, 1:1, 1:N, and unprotected protection schemes on each side of the cross-connect. The timing, communications, and control unit can be utilized to monitor for failed devices and control use of a protect card.

An advantage of the present invention is that protect cards can be used to carry traffic when not being used by the working card. Another advantage of the present invention is the ability to change the working:protect ratio without modification of cards or the backplane.

These and other features and objects of the invention will be more fully understood from the following detailed description of the preferred embodiments which should be read in light of the accompanying drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In describing a preferred embodiment of the invention illustrated in the drawings, specific terminology will be used for the sake of clarity. However, the invention is not intended to be limited to the specific terms so selected, and it is to be understood that each specific term includes all technical equivalents which operate in a similar manner to accomplish a similar purpose.

With reference to the drawings, in general, andFIGS. 1 through 14in particular, the apparatus of the present invention is disclosed.

The present invention supports a number of telecommunications and networking standards including those described and defined in the following references: Bellcore Standard GR-253 CORE, Synchronous Optical Network (SONET) Transport Systems: Common Generic Criteria, Issue 2, December 1995; GR-1230 CORE, SONET Bidirectional Line-Switched Ring Equipment Generic Criteria, Issue 3A, December 1996, Bellcore TR-NWT-000496, SONET Add-Drop Multiplex Equipment (SONET ADM) Generic Criteria, Issue 3, May 1992; Bellcore Transport System Generic Requirements FR-440, Issue No. 98, September 1998; IEEE/ANSI 802.3 Ethernet LAN specification; and Networking Standards, by William Stallings, published by Addison-Wesley Publishing Company (New York, 1993), all of which are incorporated herein by reference.

Outline of the Detailed Description

I. System Overview

A. System architecture

D. Timing communications and control subsystem

E. System Communications Link (SCL)

F. Data timing and alignment

G. Software architecture

II. Redundancy and protection

III. System transport configurations

I. System Overview

FIG. 1illustrates a block diagram of the flexible cross-connect and data plane system capable of routing traffic across two high-bandwidth planes. The system includes a SONET plane100and a data plane110. A centralized fully non-blocking cross-connect (XC)120located in the SONET plane100interfaces with SONET plane network interface subsystems130and data plane network interface subsystems140. Standardized SONET, SDH and PDH telecommunications traffic enters the system through SONET plane network electrical and optical interface subsystems130and, through point-to-point connections150to the XC120, is processed by the XC120before being returned to the network through a SONET plane network interface subsystem130or a data plane network interface subsystem140.

As in the SONET plane100, data plane network interface subsystems140interconnect150to the XC120for processing by the XC120before being returned to the network. The data plane110also allows for processing and interconnection160between data plane network interface subsystems140before being returned to the network or before being interconnected through network interface subsystem to cross-connect connections150to the XC120for additional processing and before being returned to the network.

The interconnected mesh formed by interconnecting data plane network interface subsystems140using data plane network subsystem interface connections160defines data plane110. Data plane network interface signals include packet data transport formats such as Frame Relay and ATM, MAN/LAN formats including Ethernet, FDDI, or Token Ring. The interfaces supported by data plane network interface subsystems include electrical Ethernet interfaces such as 10BaseT and 100BaseT, as well as optical interfaces for 1000BaseT and Gigabit Ethernet. Other data-centric interfaces can be used and are understood by those skilled in the art.

In a preferred embodiment, network interface subsystem to cross-connect connections150between the XC120and the SONET plane network interface subsystems130or between the XC120and a data plane network interface subsystems140are in a single specified format. In a preferred embodiment the network interface subsystem to cross-connect connections150are realized as STS-192 formatted signals transported on a multi-trace electrical bus formed on a high-speed backplane.

An alternate embodiment of the flexible cross-connect is realized inFIG. 2. As shown inFIG. 2, specific network interface subsystems are designated as high-speed interface subsystems200and others are designated as low-speed interface subsystems220having corresponding high-speed connections230and low-speed connections240to the XC120. In this embodiment the low-speed interconnections operate at the STS-48 rate of 2.488 Gb/s, while the high speed interconnections230operate at the STS-192 rate of 9.953 Gb/s.

In a preferred embodiment high speed network interface subsystems200are realized as printed circuit boards containing active and passive electrical and optical components, and can contain multiple network interfaces202operating at the same or different speeds. Low speed network interface subsystems220are also realized as printed circuit boards with active and passive electrical and optical components, and can contain multiple network interfaces202operating at the same or different speeds. As an example, a low speed network interface subsystem220can be realized as a DS-1 interface board supporting 14 DS-1 interfaces. Alternatively, a low speed network interface subsystem can be realized as an Ethernet board supporting multiple Ethernet interfaces.

FIG. 3illustrates a block diagram of a preferred embodiment of the present invention. As shown inFIG. 3, the XC120has direct point-to-point network interface subsystem to cross-connect connections150to interface subsystems301,302,303,304,309,311,312,313,314,319. Each of the interface subsystems301–304,309,311–314,315represents an interface card which is either of the class of cards which are SONET plane network interface subsystems130or which are data plane network interface subsystems140. The designation L and R in network interface subsystems301–304,309,311–314,315are used to designate the left-hand side and right hand-side of a mechanical configuration, but are not intended to be architectural limitations.

Referring toFIG. 3, there are multiple point-to-point system communication links352between a centralized timing, control, and communications subsystem (TCC)300to each of the interface subsystems301–304,309,311–314,315. The TCC300is also directly connected to XC120via the TCC to XC communication bus360. In a preferred embodiment the system has a redundant XC325and a redundant TCC375.

In a preferred embodiment the cross-connect is formed by utilizing a backplane in a mechanical rack with card slots, each card slot permitting the insertion of one or more types of cards. Card slots terminate at connectors installed in the backplane. Connections in the backplane form the connections illustrated as network interface subsystem to cross-connect connections150and data plane network subsystem connections160.FIG. 4Aillustrates a card configuration for the system.

In a preferred embodiment data plane110has 160 Gb/s aggregate bandwidth to support communications between data plane network interface subsystems140. In a preferred embodiment this capacity is realized by 1 and 3 Gb/s point-to-point links between network interface cards which form the data plane network interface subsystems140. In a preferred embodiment data plane100has an extremely low latency, fully meshed, point-to-point fabric between each network interface card to provide a fully non-blocking data switch backplane.

The SONET plane100is formed by point-to-point connections between the network interface card slots and a card slot supporting a cross-connect unit. In a preferred embodiment a fully non-blocking cross-connect module located in the XC slot can groom traffic in STS-1 or VT1.5 payload increments to any port on any card. This maximizes bandwidth efficiency by making it possible to groom SONET traffic in STS-1 or VT1.5 increments. In a preferred embodiment XC unit120supports all VT 1.5 through STS-192 applications. Using the redundant cross-connect slots, the system can also be configured as a fully protected stand-alone bandwidth manager.

The system handles all traffic transparently and can consolidate a combination of TDM, ATM, and Ethernet/IP traffic over SONET protocol. Because every interface card from DS-1 to OC-192 can be installed in a single shelf, each terminal can provide access and transport interfaces. The system can be deployed in a number of network configurations including Terminal Mode (TM), Add-Drop Multiplexor (ADM), Regenerator, and SONET Ring.

A. System Architecture

In a preferred embodiment the system is realized as a rack with card slots, as illustrated inFIG. 4A. The rack consists of a card cage, a backplane, and set of plug-in cards.FIG. 4Aillustrates arrangement of cards in a preferred embodiment using a backplane and card cage hosting17cards. Mechanical card guides and backplane connectors801on backplane800form card slots. The card slots in the card cage are numbered from 1 to 17, left to right. The plug-in cards are grouped into two general groups. The first group is the common equipment cards, which include XC card440, redundant XC card442, TCC card430, redundant TCC card432, and the miscellaneous interface card (MIC card)450. The second group comprises the network interface cards810and includes low speed cards420and high speed cards400, which form SONET plane network interface subsystems130and data plane network interface subsystems140

XC120is realized as a XC card440located in a slot towards the center, as shown inFIG. 4A. A redundant XC325is realized as a redundant XC card442located in a card slot towards the center. A timing, communications, and control (TCC) unit300and a redundant TCC unit375are realized as TCC card430and redundant TCC card432, also located in card slots towards the center of the system.

FIG. 4Billustrates another view of a preferred embodiment of the mechanical configuration of the system, and includes the backplane800and backplane connectors801.

As shown inFIG. 4A, high speed network interface cards400and low speed network interface cards420are supported. In one embodiment high speed network interface cards400support one or more electrical and optical interfaces up to OC-192 data rates, while low speed network interface cards420support data rates of up to OC-48 rates. Traffic carried through these network interfaces is routed over the backplane to a central cross-connect point located on the XC card440or on redundant XC card442.

In one embodiment, the XC card440and the redundant XC card442can switch any STS-1 payload to any low speed network interface card420or high speed network interface card400. In a preferred embodiment, the XC card440and the redundant XC card442can switch any SONET VT1.5 virtual tributary located within a SONET STS-1 payload to any low speed interface card420or high speed interface card400.

All cards are powered through local on-card switching power supplies. Redundant −48V power is provided through the backplane connectors, and is diode-ORed on each card. Staged pre-charge pins are provided to allow for well-controlled power initialization at card insertion.

Below the card cage proper is a plenum460, which is used to provide uniform cooling air distribution to the cards above. The area in front of the plenum is used for fiber routing. A fan tray470is used to support cooling fans which circulate air above the plug-in cards and to air ramp480.

In a preferred embodiment, network interface subsystem to cross-connect connections150are realized as parallel data bus on backplane800comprised of 32 signals representing 32 bit streams, supporting STS-192 (high speed) payloads. In this embodiment single-ended signals connected via end-terminated controlled impedance traces. Gunning Transistor Logic (GTL) is used on cross-connect unit120and on network interfaces202to accommodate a data rate of 311.04 MHz. In this embodiment, no clock is carried over the backplane. Clock recovery is performed at the receiving end by monitoring data transitions using clock recovery techniques known by those skilled in the art.

In an alternate embodiment only a subset of network interface subsystem to cross-connect connections150support STS-192 payloads associated with high speed network interface subsystems200, with the remainder supporting STS-48 (low speed) payloads associated with low speed network interface subsystems220. Connections to low speed network interface subsystems220are created using a parallel data bus of 8 bits representing 8 signals carried on single-ended, connected via end-terminated, controlled-impedance traces. GTL logic is used at both the receiving and transmitting devices. Clock recovery is performed by monitoring data transitions.

In another embodiment high speed network interface subsystems200support STS-48 payloads using a parallel data bus comprising 16 bits plus sync (17 signals) operating at 155.5 MHz over single-ended, connected via end-terminated, controlled-impedance traces. GTL logic is used on the transmitting and receiving devices. Clock signals are transmitted using a differential clock using (Low Voltage Differential Signals) signals at 155.52 MHz, connected via end-terminated, controlled-impedance traces.

In this embodiment low speed network interface subsystems220support STS-12 payloads using a parallel data bus comprising 4 bits plus sync (5 signals) operating at 155.5 MHz transmitted over single-ended, connected via end-terminated, controlled-impedance traces. GTL logic is used on the transmitting and receiving devices. Clock signals are transmitted using a differential clock based on LVDS signals at 155.52 MHz, connected via end-terminated, controlled-impedance traces.

In a preferred embodiment the backplane is comprised of a 32 layer board which supports connections between inserted cards with electrical traces within the board. Groups of controlled impedance traces have matched lengths to create equal propagation delays. In a preferred embodiment a 75Ω trace impedance is used, although other controlled impedances including 50Ω, 100Ω, or higher impedance traces can be used. Connector pins are interspersed with ground signals to reduce coupling and crosstalk between signal lines.

In a preferred embodiment traces used to carry high speed signals are a controlled impedance and have a termination resistor which is equal to the transmission line (trace) impedance, and which serves as a pull-up resistor for the output transistor of the transmit device which is connected to the trace.

In addition to traces used to realize payload connections over high speed connections230and low speed connections240, which form the network interface subsystem to cross-connect connections150, traces are in place to realize the data plane network subsystem interface connections160and the system communication links352.

In addition to the traces used to support card to card connections, traces exist in the backplane to support protection as illustrated inFIGS. 12A–12C.

One advantage of the present invention is that by using controlled impedance traces on the backplane it is possible to utilize backplane traces for the transport of electrical signals from the network either directly or with an impedance conversion. In the present invention DS-3 signals which are unbalanced and require approximately a 75Ω impedance can be transmitted on the backplane, while DS-1 signals which are received on a 100Ω balanced transmission line are converted to a 75Ω unbalanced signal using a balun. The 75Ω unbalanced signal can be propagated directly on the backplane and converted back to a balanced signal for transmission on the network. This feature allows use of the backplane traces for a wide variety of purposes including protection switching and the interconnection of cards.

The backplane also supports the use of Backplane Interface Cards (BICs) which provide space for connectors including SMB, CHAMP connectors manufactured by Amphenol, or other coaxial or twisted wire pair connectors. The BIC interfaces to the backplane through pins and receiving holes on the backplane.

In a preferred embodiment, protection switching relay matrices are mounted on the interface cards which support electrical interfaces. In an alternate embodiment, protection switching relay matrices can be mounted on BICs.

The XC120provides the switching fabric for the system. As the central switching element for the system, the XC120features low latency and fast switching to establish connections and perform time division switching at an STS-1 level between the XC120and the SONET network interface subsystem130and between the XC120and the data plane network interface subsystem140.

FIG. 5is a block diagram of XC card440. As a plug-in card, connections to the XC card440occur through an XC backplane connector520. On the board, the XC switch matrix500connects to XC backplane connector520. Low speed line connections501–504and505–508connect a cross-connect matrix500to XC backplane connector520. High speed line connections511–514connect cross-connect matrix500to backplane connector520.

The XC switch matrix500also connects to the XC microprocessor540, XC flash memory560and local random access memory (RAM)550over the microprocessor to XC matrix control bus590. The XC microprocessor540connects to a XC card flash programmable gate array (FPGA)530over the microprocessor to XC card FPGA interface532.

The XC card FPGA530connects to the XC switch matrix over the XC matrix to FPGA interface596and connects to the loop filter580providing a loop filter up control signal582and a loop filter down control signal584. The loop filter580provides a frequency control signal586to the voltage controlled crystal oscillator (VCXO)570. The XC card FPGA530receives reference clock A592and reference clock B594from reference clock sources through XC backplane connector520.

XC card FPGA530supports control of cross-connect matrix500via signals received from TCC card430over the main system communication interface534or over the protect system communication interface536. Functions performed by XC card FPGA include management of the SCL, and filtering of protection and notification information which is subsequently sent to and received from XC matrix500. In a preferred embodiment the functions of XC card FPGA530are realized in an application specific integrated circuit (ASIC).

The XC card FPGA530also connects to the TCC card430over the main system communication interface534, through the XC backplane connector520and the backplane800, and to the redundant TCC card432over the redundant system communication interface536and through the XC backplane connector520and the backplane800.

In a preferred embodiment the XC switch matrix500resident on the XC440is a full crosspoint, non-blocking, switch and supports broadcast switching. Designs for such switching matrices are well known to those skilled in the art. XC switch matrix500allows network operators to concentrate, groom, or hairpin network traffic from one interface card to another without limit on card location within the equipment. An STS-1 on any of the input ports may be mapped to an STS-1 slot on any of the output ports.

In one embodiment, the XC switch matrix500can switch any STS-1 payload coming over a low speed line connection501–508or over a high speed line connection511–514to any low speed interface card420or high speed interface card400.

In a preferred embodiment, the XC switch matrix500can switch any SONET VT1.5 virtual tributary located within a SONET STS-1 payload coming over a low speed line connection501–508or over a high speed line connection511–514to any low speed interface card420or high speed interface card400.

Provisioning of the switch matrix500is accomplished via information which is relayed from the TCC card430through the main serial communication interface534or which is relayed from the redundant TCC card432through the protect serial communication interface536. This information is monitored by the XC card440and used to establish and tear down connections. Main serial communication interface534and protect serial communication interface536represent the board level connections which correspond to system communication link352.

D. Timing Communications and Control Subsystem

The timing communications and control card (TCC)430performs system initialization, provisioning, alarm reporting, maintenance, diagnostics, IP address detection/resolution, SONET DCC termination, and system fault detection for the system. The TCC also ensures the system maintains Bellcore timing requirements.

FIG. 6is a block diagram of the TCC card430. Connections to other system cards400420432440442450are made through a TCC backplane connector620and via backplane800. The serial communication link (SCL) termination link640brings SCL352from each low speed network interface card420and each high speed network interface card400to the SCL time slot interchanger (TSI)634. The SCL TSI634also provides outgoing communications, timing, and control signals to the XC card440and the redundant XC card442and to the time division multiplexer/serial communications controller (TDM/SCC)630.

The TDM/SCC630is a multiplexer and cell bus processor providing a TDM signal to the SCL TSI634and to the message router632. The message router632also receives cells from the SCL TSI634and is a cell switch for all the interface cards. The message router632, TCC flash memory602, random access memory603, timing controller604, framer line interface unit (LIU)605, SCL TSI634and TDM/SCC630all receive control signals from the TCC control processor600over the microprocessor bus601. The TCC control processor600also has cell based connection to the TDM/SCC630, a connection to a 10BaseT Ethernet/craft interface module670which connects to the TCC backplane connector620, and a control and communication connection to the DCC processor611. A redundant TCC connection653is provided and connects the TCC control processor600on the TCC card430to a serial port on the TCC control processor600on the redundant TCC card432. The DCC processor611has a bi-directional connection to the TDM/SCC630, connection to local DCC processor flash memory612and random access memory613, and a connection to a 10BaseT Ethernet/modem interface module652which connects to the TCC backplane connector620.

An Ethernet hub660connects to the backplane connector620, the 10BaseT and craft interface module670, and has an interface on the front panel675.

The TCC card430supports multichannel high-level data-link control (HDLC) processing for Data Communication Channels (DCC). Up to 48 DCCs may be routed over the serial communication link (SCL)352and terminated at the TCC card430. Ten DCCs are selected and processed on TCC card430. This facilitates remote system management interfaces.

The TCC card430also originates and terminates a cell bus carried in SCL352. The cell bus supports links between any two cards in the system for peer-to-peer communication. Peer-to-peer communication speeds protection switching for redundant cards. The system database, IP address, and system software are stored in TCC non-volatile flash memory602and612, allowing quick recovery in the event of a power or card failure.

The TCC card430performs system timing functions for the system. The TCC card430monitors the recovered clocks from each low speed interface card420and from each high speed interface card400, and two DS1 (BITS) interfaces for frequency accuracy. One of the recovered clocks, one of the BITS, or an internal Stratum 3 reference is selected as the system timing reference. Any of the clock inputs may be provisioned as a primary or secondary timing source. A slow reference tracking loop allows the TCC card430to synchronize to the recovered clock, providing holdover if the reference is lost.

E. System Communications Link (SCL)

In a preferred embodiment there are several types of internal communications paths used to transport timing, communications, and control signals. The combined signals are referred to as the system communications link (SCL)352. SCLs352connect-interface subsystems301–304,309,311–314,315with TCC unit300, as shown inFIG. 3. In a preferred embodiment SCLs352are carried on point-to-point connections with low latency and guaranteed bandwidth in a byte oriented manner.

Referring toFIG. 7, the SCL includes SONET overhead bytes located within the SONET overhead channel704, such as the data communication channel (DCC) bytes D1711, D2712, and D3713. SONET overhead bytes can also include orderwire, and K1 and K2 automatic protection switch (APS) bytes. Format for SONET overhead is described in GR-253 and is understood by those skilled in the art.

SCL352supports intercard communications for configuration, performance monitoring, and other general purposes. In a preferred embodiment SCL352combines both byte-oriented time division multiplexed (TDM) and cell-switched capability within one physical link.

In a preferred embodiment, the physical link consists of a 19.44 MHz differential LVDS clock, a frame synchronization signal, an enable signal, a transmit data signal, and a receive data signal. Although the clock frequency is 19.44 MHz, through the use of the enable signal, the data rate is reduced to 16.384 Mb/s.

Referring toFIG. 7, the SCL is subdivided into four bit-interleaved 4.096 Mb/s channels; a TDM TCP/IP channel700, a TDM SONET overhead channel704, a bit-interleaved fast cell bus channel720, and a TDM spare channel730. Each channel is further divided into 64 eight-bit timeslots, with a 125 μs frame rate. The fast cell bus720is used to carry internal cell-switched intercard communications.

In a preferred embodiment SCLs352are realized as physical point-to-point connections on backplane800between each high-speed network interface card400and each main TCC card430and redundant TCC card432, and between each low speed network interface card420and each main TCC card430and redundant TCC card432. Every card connects to both TCC card430and redundant TCC card432. In a preferred embodiment the SCL352connected to the currently working TCC is used to carry active data.

The SCL352from each high-speed network interface card400or from each low speed network interface card420is terminated on the TCC card430, where each SCL352is split into the three 64-byte TDM channels700,704,730, and the 64 byte fast cell bus channel720. The TDM channels700,704, and730are connected to the SCL TSI634, where, working in conjunction with the TDM/SCC630, the individual bytes within each channel are reassembled onto the SCL output channel651. The SCL TSI634is capable of arbitrary timeslot rearrangement, and is also able to place programmed byte values in a given output timeslot. In this manner, bytes (such as SONET overhead bytes) are collected from the various high speed network interface cards400and from the various low speed network interface cards420and are sent to their destination cards.

The cell switched channel720of the SCL352is sent to the cell-switch message router632, which routes each received cell to the destination encoded in the cell header.

The SCL also provides a system watchdog or tripwire function through dedicated timeslots and pattern generation and detection hardware at each end of the links.

F. Data Timing and Alignment

In order to permit the transport of high speed STS-48 and STS-192 payloads from low speed interface cards420and high speed interface cards400to cross connect card440and redundant cross connect card442, synchronization signals are sent from the cross connect card442, or in the case of a failed cross connect card440from the redundant cross connect card442, to low speed interface cards420and high speed interface cards400.

In a preferred embodiment pre-alignment of the signals is performed on the low speed interface cards420and high speed interface cards400. The pre-alignment of the signals is accomplished through the use of a programmable offset generator which is used to account for the delay between the interface cards and the cross connect. Referring toFIG. 1, this is an offset implemented in SONET plane network interface subsystems130and data plane network interface subsystems140to facilitate cross connection of high data rate streams at cross connect120.

FIG. 8illustrates the system connections between a interface card810which represents any card used to realize a SONET plane network interface subsystem130or a data plane network interface subsystem150and XC card440. Interface card810contains a Bridging Transmission Convergence Application Specific Integrated Circuit (BTC ASIC)840, and XC card440contains a SONET cross connect Application Specific Integrated Circuit (SXC ASIC). TCC card430is represented inFIG. 8and contains a PLL880and a voltage controlled oscillator884.

In a preferred embodiment SXC ASIC850supports STS-192/48/12 interfaces using ports operating at 311 MHz using single-ended Gunning Transistor Logic (GTL) signals, and connects across backplane800to BTC ASIC840. In a preferred embodiment, SXC ASCI850can be controlled by a microprocessor, an example of which is the Motorola860class of processors. Alternate processors can be used and are known to those skilled in the art.

BTC ASIC840interfaces to backplane800at 311 MHz as well as at 155 MHz, to support interface cards810which are of the classes of high speed network interface subsystems200or low speed network interface subsystems220. In a preferred embodiment the BTC ASCI840contains an interface port to a microprocessor, examples of which are the Motorola850and860classes of processors.

As illustrated inFIG. 8, a timing reference clock860originating as part of a received signal820is extracted in BTC ASIC840- and flows to the TCC card430. At the TCC card430the timing reference clock passed through a TCC timing tracking loop885formed by PLL880and voltage controlled oscillator884resulting in a system master clock890.

The system master clock890is transmitted to SXC ASIC850on the cross-connect card440, to the interface card810over a backplane800, and out on the transmitted signal830. The format of the signals flowing between the BTC ASIC840and the SXC ASIC850includes parallel data lines, a sync signal, and a clock signal.

FIG. 9illustrates the structure of SXC ASIC850. Inputs from BTC #1960through BTC #N962on several interface cards810are received at SXC ASIC850. Each input flows through a FIFO950, the depth of which can be controlled by a subsequent frame aligner940. The frame aligner940/FIFO950combination is used to delay each of the incoming SONET frames (from the BTC ASICs840) such that upon arrival at a STS switch matrix900all STS signals are frame-aligned. This allows the STS selectors910and frame generators920to assemble a SONET frame at each of the outputs easily and with no extra buffering. The assembled SONET frame is constructed of the STS-1 signals which have been demultiplexed by STS-1 demux930and selected by STS-1 selectors910. FIFO950is needed to accommodate the delays between the XC card440and interface card810(including propagation delays arising from backplane900, connector propagation delays, and ASIC I/O and internal logic delays). In an alternate embodiment, STS switch matrix900is replaced with a VT 1.5 switch matrix to allow cross-connection at the VT1.5 level of the SONET STS-1 frame. Other granularity cross connect fabrics can be used to permit cross-connection at other standard or non-standard data rates.

FIG. 10illustrates the structure of BTC ASIC840, which is resident on high speed network interface card400, low speed interface card420, or any other interface card which comprises the SONET plane network interface subsystem130or data plane network interface subsystem140. The SONET line input1090is received by line input framer1000and has an arbitrary input frame alignment, and may have small frequency errors as well, which will create a shifting frame alignment. A receive pointer processor1010is used to identify the alignment of the individual STS1's within the received frame, and a pointer generator1020and frame generator1030are used to create a new SONET frame that is aligned to meet the requirements of the SXC ASIC850. This alignment is controlled by the frame alignment of the SONET frame received via the backplane800from SXC ASIC850, and by a programmable offset generator1040that will cause the frame generator1030to send the frame, advanced by a controlled amount from the alignment of the frame from the SXC ASIC850. The amount of offset is selected based on the predetermined system delays, the depth of the FIFO's within the SXC ASIC850, and the additional data path delays within the SXC ASIC. In addition, the controllable offset is used to allow for different cross-connect designs that have different delay characteristics. Output from frame generator1030appears at an output to SXC1094and an output to redundant SXC1095.

Referring toFIG. 10BTC ASIC840has an input from SXC1096and an input from redundant SXC1097. SXC input framers1004are used to frame the received signal, and selector1080is used to select one of the signals. A frame sync indicator1050is sent to the programmable offset generator1040, which in conjunction with control signals1041from a microprocessor resident on high speed network interface card400or low speed network interface card420.

The signal being generated for transmission in BTC ASIC840is sent from the selector1080to an overhead insertion unit1060and line output frame generator1032to form a SONET line output1092.

Monitoring units170are used to monitor framed signals and to determine erroneous states which should be reported to TCC card430.

G. Software Architecture

FIG. 11illustrates the software architecture of the TCC unit300in a preferred embodiment. In a preferred embodiment the software is realized using the JAVA and C programming languages running on the operating system sold under the trademark VXWORKS by the Wind River Systems Corporation. In a preferred embodiment the low-level software which communicates between boards in the system is written in C, but management software performs as a HTML server, and is written in C and Java. In a preferred embodiment the software runs on an MPC860 processor.

As illustrated inFIG. 11, a network management interface1100is present and serves as the interface to the rest of the software. In a preferred embodiment network management interface1100is realized in the JAVA programming language, and allows the use of any browser in a network element running a TCP/IP stack to address the system.

A provisioning manager1110is present and is responsible for managing the provisioning database for the system. The provisioning manager1110interfaces with subordinate cards via an equipment & link state manager1120and to the management software via network management interface1100.

The equipment & link state manager1120addresses network interface cards via an inter-card communications module1130, and is the central point of communications between the TCC card and the subordinate cards, which can be considered to be cards forming SONET plane network interface subsystems130, data plane interface subsystems140, and cross connect120. The equipment & link state manager1120notifies other components on the TCC when a slave card needs service, and blocks information being sent to a slave card that is in the process of reading its software image from a shelf controller. In addition, the equipment & link state manager1120maintains information about the state of each slot, card and communications link, and in a preferred embodiment acts as the single authority on the state of each component in the system. The equipment & link state manager1120on each non facility protected card is responsible for initiating an equipment protection switch when a partial or full failure is detected on a card. In a preferred embodiment, a card presence/alive message is transmitted over SCL352and it transported to peer cards, subordinate cards, TCC, and the cross connect. The equipment & link state manager1120is responsible for monitoring this link and initiating the proper action when a failure is detected.

Provisioning manager1110also talks with database manager1140which maintains a database1142of equipment and service related information.

The software also supports alarm filtering and reporting through an alarm filtering and reporting module1150. The alarm filtering and reporting module1150confirms that a failure condition exists for a pre-programmed amount of time and can report alarms which have been filtered. Provisioning manager1110is responsible for programming of times and filters in the alarm filtering and reporting module1150.

A BLSR connection map manager1160maintains information related to ring configurations, and in particular maintains a record of the K1/K2 bytes of the SONET line overhead using in ring networks.

A synchronization manager1170provides for the provisioning and monitoring of an internal stratum 3 clock reference, the provisioning and monitoring of a building integrated timing supply (BITS) input; provisioning and control of the DSX-1 formatted BITS output, and selection of the timing reference for the system. In addition, synchronization manager1170selects the timing reference for the BITS output, processes and acts upon synchronization status messages, and controls synchronization switching on synchronization reference changes.

An embedded debugger1175provides the ability to detect and repair software errors as determined in conjunction with equipment & link state manager1120.

A software program manager1180supports downloading and upgrading to new system software which is stored in software storage1182. Specifically software program manager1180supports the receiving of new software loads, access to a flash memory file system, upgrading of the boot image, and access to software images when subordinate cards boot.

An inter-node communications module1190supports communications between the system and other nodes, using both TCP/IP and open shorted path first (OSPF) protocols.

II. Redundancy and Protection

In a preferred embodiment the system employs an individual card protection architecture, where a protected card that fails is replaced by the protection card. This contrasts with a bank-switching protection architecture, where an entire “bank” of cards switches should any member of the bank fail. The individual card protection method offers a significant advantage in flexibility, in particular where a mixture of protected and non-protected services are provided from the same shelf.

In a preferred embodiment the protected common equipment cards include XC card440, and TCC card430.

Backplane800supports connections such that each of the 12 network interface cards810has fully duplicated connections to XC card430and redundant XC card432and to the TCC card430and redundant TCC card432. A card that is at the receiving end of one of these connections uses hardware detection to monitor SCL352for activity and valid data patterns. Should the active link fail, the processor on the receiving board is notified by the monitoring hardware. Link selection is performed by either local board firmware or by automatic hardware switching.

The electrical interfaces on the network interface cards810allow both 1:1 and 1:N equipment protection by way of interconnections between neighboring cards of a protection group. Relays on the network interface cards810are used to connect the active card to the appropriate electrical interface, thus supporting protection without requiring manual intervention and rearrangement of external cabled connections.

Referring toFIG. 12A, 1:5 protection for a left group1200and right group1210is illustrated. Network interface connections1205are illustrated as well as backplane protection traces1208. The backplane protection traces1208in backplane800provide connectivity between each card and an outer card, via pin connections on the card backplane connectors to pins on the backplane connectors. Backplane pins are connected to other backplane pins via traces in the multilayer backplane800. Connections are also provided between each working card and the protect card. Right group1210illustrates the mirror image backplane protection traces1208which support right group1210.

In the configuration illustrated inFIG. 12A, failure on a working card is accommodated by the routing of traffic to an adjacent (away from the center) card, or to the protect card.

In a preferred embodiment the protect card can be used to carry traffic, but that traffic may be abandoned in the event of a failure on a working card.

FIG. 12Billustrates the case for 1:1 protection in left group1200in which each working card has an adjacent protect card. When used in the 1:1 configuration there are unused backplane traces1220, represented as dotted lines inFIG. 12B. Although there are unused backplane traces1220, these traces do not need to be removed from backplane800. TCC unit300which is realized as TCC card430can be programmed to configure the system for 1:1 protection, leaving certain backplane traces unused for that particular configuration.

Right group protection using the 1:1 configuration illustrated inFIG. 12Bcan be accomplished by creating the mirror image of the traces shown inFIG. 12Bon the right side of backplane800.

FIG. 12Cillustrates a combined 1:2, 1:1 and unprotected card configuration for left group1200. This configuration can be accomplished using the same backplane traces that are used for 1:5 and 1:1 protection schemes. As with the other protect schemes, TCC card430controls the configuration.

An advantage of the present invention is that cards can be configured via software and the protection schemes can be varied without the use of jumper cables on the backplane or rearrangement of cards in the shelf.

III. System Transport Configurations

The present system can be utilized in a variety of configurations supporting transport of data on MAN/LAN, interLATA and interexchange networks. Because the system can establish cross connections between any interface cards and can use a data plane for local switching, a wide variety of transport configurations can be supported, including configurations in which a virtual LAN is created in the data plane, and access to the SONET plane is granted for transport to other nodes.

FIG. 13illustrates prior art for an optical ring architecture having four SONET network elements A1300, B1310, C1320, and D1330with optical fiber spans1350creating the ring structure. In this configuration the SONET network elements A1300, B1310, C1320, and D1330can be SONET line termination equipment (LTE) or add-drop multiplexers (ADMs). The fiber spans may be composed of one or more unidirectional or bi-directional fibers or optical cables. Input signals1340are typically electrical signals which are then routed by SONET network elements over the optical fiber ring1350to the appropriate destination SONET network element.

FIG. 14Aillustrates a multi-ring network commonly found in the telecommunications industry. InFIG. 14A, a first ring is composed of four SONET network elements A1300, B1310, C1320, and D1400interconnected by optical fiber links1350. A second ring composed of another four SONET network elements E1410, F1420, G1430and H1440interconnected by optical fiber links1450. For signals originating on one ring but having a destination on the other ring, a back-to-back interconnection of SONET network elements is required. This back-to-back configuration is illustrated by node-to-node connection1453between SONET network element D1400and SONET network element H1440. Present equipment supports a single ring in an individual add-drop multiplexer such as those represented by SONET network elements D1400and H1440.

FIG. 14Billustrates a multi-ring network that utilizes a SONET network element K1460having the capability of a flexible cross-connect to support multiple optical rings, as can be realized using the present invention. As shown inFIG. 14B, one ring is composed of SONET network elements A1300, B1310, C1320, and K1460that are interconnected with optical fiber segments1350. A second ring that is connected to the first ring by the sharing of SONET network element K1460, is composed of SONET network elements E1410, F1420, G1430, and K1460interconnected by optical fiber segments1450. This configuration eliminates the need for the back-to-back configuration of SONET network elements D1400and H1440while maintaining the full capability and functionality of the configuration.

FIG. 15illustrates a mesh network created by connections between low speed network interface subsystems220. In the present invention the mesh network can be realized in data plane110without the use of cross connect120, while simultaneously accessing cross connect120for connectivity to SONET plane100.

As illustrated inFIG. 15, the group of low speed network interface subsystems220are interconnected with point-to-point connections160between each of the low speed network interface subsystems220. In a preferred embodiment, these connections160are 1 and 3 Gb/s point-to-point links between network interface cards in the data plane110.

FIG. 15also illustrates how centralized XC120connects to high speed network interface subsystems over high speed point-to-point interconnections230, and connects to low speed network interface subsystems220over low speed point-to-point interconnections240.

FIG. 16Aillustrates an embodiment of a mesh network created by point-to-point connections between all low speed network interface subsystems220.FIG. 16Billustrates another embodiment of a mesh network created by point-to-point connections between all high speed network interface subsystems200and all low speed network interface subsystems220. The partial mesh and full mesh networks represented inFIGS. 16A and 16Brespectively can be used to realize data planes110which encompass low speed network interface subsystems220and both low speed network interface subsystems220and high speed network interface subsystems200respectively.

Although this invention has been illustrated by reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made which clearly fall within the scope of the invention. The invention is intended to be protected broadly within the spirit and scope of the appended claims.