Messaging with flexible transmit ordering

In one embodiment, a system includes a packet reception unit. The packet reception unit is configured to receive a packet, create a header indicating scheduling of the packet in a plurality of cores and concatenate the header and the packet. The header is based on the content of the packet. In one embodiment, a system includes a transmit silo configured to store a multiple fragments of a packet, the fragments having been sent to a destination and the transmit silo having not received an acknowledgement of receipt of the fragments from the destination. The system further includes a restriction verifier coupled with the transmit silo. The restriction verifier is configured to receive the fragments and determine whether the fragments can be sent and stored in the transmit silo.

BACKGROUND

Typical network devices can implement packet transfers via various packet interfaces, such as Ethernet interfaces SGMII and XAUI. Packet transfers can also be completed over PCI Express (PCIe), a read/write based interface.

Messaging systems such as Serial Rapid Input Output (S-RIO) divide packets (or messages) into fragments. A source node transmits the fragments over a physical layer of network systems to a destination node. Intermediate nodes may receive and forward the packet to the destination as well. Upon receiving the fragments, the destination reassembles the fragments into the original packet.

S-RIO is a serial, packet-based interconnect protocol. S-RIO is optimized for providing communications among devices in an embedded system. S-RIO is commonly implemented in networking and signal processing applications.

SUMMARY

In one embodiment, a system includes a packet reception unit. The packet reception unit is configured to receive a packet, create a header indicating scheduling of the packet in multiple cores and concatenate the header and the packet. The header is based on the content of the packet.

The system may further include a memory internal to the packet reception unit. The packet reception unit may further be configured to receive multiple packets. The packets may originate from more than one source.

The system may further include multiple reassembly stores. Receiving the packet may further include receiving fragments of the packet, storing the fragments in a particular reassembly store corresponding with the packet, and when the particular reassembly store contains fragments of the packet such that the stored fragments represents the packet as a whole, forwarding the packet to the cores. The particular reassembly store may be allocated to the packet upon receiving a first fragment of the packet. The particular reassembly store may accept other fragments of the packet after receiving the first fragment of the packet. The packet reception unit may further be configured to send a retry message when each of the reassembly stores are unavailable to receive a first of the at least one fragments of the packet. The packet reassembly stores may be stored in a memory internal to the packet reception unit. The packet reception unit may be further configured to, store the at least one fragments in one of a plurality of memories within the reassembly stores, and, when the one of the plurality of memories is filled, copy the at least one fragments to an external memory.

The system may further include a packet allocation unit configured to direct each fragment associated with the packet to the particular reassembly store corresponding with the packet. The system may further include at least one port within the packet reception unit. Each reassembly store may be assigned to one of the ports. The reassembly store assignments to the ports may be configurable to affect quality of service of each of the ports.

The system may further include multiple packet reception units. Each packet reception unit may share the reassembly stores. Each reassembly store may be assigned to one of the packet reception units. The reassembly store assignments to the packet reception units may be configurable to affect quality of service of each of the packet reception units.

In one embodiment, a method includes receiving a packet and creating a header that indicates scheduling of the packet in multiple cores, the header based on the content of the packet, and concatenating the header and the packet. The method may further include receiving at least one fragment of the packet, storing the at least one fragment in a particular one of multiple reassembly stores corresponding with the packet, and forwarding the packet to multiple cores when the particular reassembly store contains at least one fragment of the packet such that the stored at least one fragment represents the packet as a whole.

In one embodiment, a system includes a transmit silo configured to store a multiple fragments of a packet, the fragments having been sent to a destination and the transmit silo having not received an acknowledgement of receipt of the fragments from the destination. The system further includes a restriction verifier coupled with the transmit silo. The restriction verifier is configured to receive the fragments and determine whether the fragments can be sent and stored in the transmit silo.

Determining whether the fragments can be sent and stored in the transmit silo in the restriction verifier may be programmable. The restriction verifier may be programmed based on the restrictions of the destination for receiving out of order fragments or packets. The restrictions may limit a number of outstanding fragments, outstanding packets, outstanding fragments per destination, outstanding packets per destination, outstanding fragments per mailbox of the destination, and outstanding packets per mailbox of the destination.

The system may further include multiple controllers. Each controller may be assigned to at least one transmit silo and at least one restriction verifier. The restrictions of each restriction verifier may limit a number of outstanding packets per controller, outstanding fragments per controller, outstanding packets per destination per controller, outstanding fragments per destination per controller, outstanding packets per mailbox of the destination per controller, and outstanding fragments per mailbox of the destination per controller. The assignments of the transmit silos and the restriction verifiers may be configurable to affect quality of service. The controllers may be configured to divide the packet into the multiple fragments, and attach a header to each fragment. Each header may indicate an order of the fragment and a total number of fragments in the packet.

The transmit silo may be further configured to delete a particular fragment of the packet when the transmit silo has received an acknowledgement of receipt of the particular fragment from the destination.

In one embodiment, a method includes storing multiple fragments of a packet in a transmit silo, the fragments sent to a destination, wherein the transmit silo has not received an acknowledgement of receipt of the fragments from the destination, receiving the fragments, and determining whether the fragments can be sent and stored in the transmit silo. In another embodiment, the method includes dividing the packet into the plurality of fragments and attaching a header to each fragment. Each header indicates an order of the fragment and a total number of fragments in the packet.

DETAILED DESCRIPTION

FIG. 1is a block diagram100of a system incorporating a packet reception unit102illustrating reception of a packet. The packet reception unit102is used in a messaging system. In one embodiment, the messaging system is Serial Rapid Input/Output (“S-RIO”), but other embodiments could use other messaging or network protocols. In another embodiment, the messaging system is based on a multi-source/multi-destination retry protocol. In yet another embodiment, the packet reception unit102is a component of a semiconductor chip.

The packet reception unit receives a packet104as input and transmits a concatenated header and packet112as output. The packet reception unit directs the packet104to a header creation module106. The header creation module creates and outputs a header108for the packet104which is based on the content of the packet104itself. The header108includes detailed data about the packet104, such as its order relative to other packets. The header108can further include a standard packet instruction header with critical fields generated from a table lookup. The critical fields can assist scheduling and prioritizing the received packet104in a plurality of cores or processors.

In one embodiment, the header108can be 16-bytes. The first eight bytes of the header108include data specific to the messaging protocol of the packet104, such as S-RIO. The data in the first eight bytes includes: priority of the packet; an indication of whether a source and destination of the packet are eight or 16 bits; an indication of whether the system is the primary or secondary destination of the packet104; a size of the packet104; the source of the packet; a recipient mailbox extension of the packet; a letter field; and a sequence number used to determine the relative order of the packets or doorbells. The second eight bytes of the header108include data that allows for generation of quality of service data and scheduling parameters. The system can then schedule processing of the packet based on the data.

A concatenator110concatenates the header108and the packet104to create the concatenated header and packet112. In one embodiment, the concatenation is a pre-pending of the header108to the packet104. A person of ordinary skill in the art can appreciate that the order of the header108and packet104and any other intervening data can be adjusted, e.g. the header108can be appended to the packet104as well.

FIG. 2is a block diagram200of a system incorporating the packet reception unit102illustrating reception of multiple packets from multiple sources. A first source206A and a second source206B are configured to transmit packets to the packet reception unit102. The first source206A transmits Packet A204A and Packet B204B to the packet reception unit102, and the second source206B transmits Packet C204C and Packet N204D to the packet reception unit102. A person of ordinary skill in the art can appreciate that the first source206A and second source206B are examples of sources, and that any number of sources could send packets to the packet reception unit102. A person of ordinary skill in the art can also appreciate that a particular source can send multiple packets to the packet reception unit102.

The packet reception unit102then concatenates a header to each of the packets204A-D, as described in reference toFIG. 1. The packet reception unit102then outputs i) concatenated header and packet A212A corresponding to packet A204A, ii) concatenated header and packet B212B corresponding to packet B204B, iii) concatenated header and packet C212C corresponding to packet C204C, and iv) concatenated header and packet N212D corresponding to packet N. A person of ordinary skill in the art can appreciate that the packet reception unit102inFIG. 2employs the same process for concatenating a header and packet described of the packet reception unit102in reference toFIG. 1.

Many messaging systems break packets into smaller fragments. A person of ordinary skill in the art can appreciate that, while the block diagrams ofFIGS. 1 and 2describe the packet reception unit102as receiving packets as one unit, the packet reception unit102can also receive packets as multiple fragments to be reassembled into a whole packet.FIG. 3AthroughFIG. 6illustrate how the fragments can be reassembled into the whole packet.

FIG. 3Ais a block diagram300of a system for receiving packets and directing the packets to a processor. A first quad-lane module (“QLM”)302A is coupled to a first reception interface310A. A second QLM is coupled to a second reception interface310B. Both the first and second reception interfaces310A-B include similar elements. A person of ordinary skill in the art can apply the description of the first reception interface310A and the elements incorporated therein recited below to implement analogous elements of the second reception interface310B.

The first reception interface310A includes a messaging core316A. The messaging core316A receives network data312A from the first QLM302A and transmits retries and acknowledgments (ACKs)314to the first QLM302A. The messaging core316A receives network data312A including messages, packets, or fragments of a packet. Upon receiving fragments of a packet within the network data312A from the first QLM302A, the messaging core316A determines whether to forward the fragments to the first inbound message port318A or the second inbound message port320A. The first or second inbound message port318A or320A forwards the fragments322A to the reassembly stores324A.

The reassembly stores324are a collection of individual reassembly stores. The individual reassembly stores are configured to hold fragments322A where each reassembly store holds fragments from a particular packet. A fragment, as described above, holds a segment of data from a packet. Likewise, a person of ordinary skill in the art can recognize that a packet is divided into a finite number of fragments. The reassembly stores324enable the reception interface310A to receive fragments from multiple sources and reassemble the fragments into packets.

Upon receiving a fragment, the reception interface directs the fragment to an appropriate reassembly store324. A packet allocation unit, as discussed later in reference toFIG. 3CandFIGS. 4-6, directs the fragment to the appropriate reassembly store324. Referring toFIG. 3A, upon a reassembly store324receiving all of the fragments that make up the particular packet, the reassembly store324outputs packets326, which includes the particular packet, to a packet input unit304. In one embodiment, the particular packet is concatenated with the header108as described in reference toFIGS. 1 and 2. Referring toFIG. 3A, the packet input unit304receives the packets326, reads the header108of the packet326, and directs the packet326to a packet input port306A-D. The packet input unit304directs the packet326to a particular packet input port by analyzing the data in the header108and determining which port is most desirable for scheduling the work in the packet and managing quality of service of multiple cores in regards to the work within the packet.

FIG. 3Bis a block diagram350of a system for receiving packets and directing the packets to a processor.FIG. 3Bis similar to the system described in reference toFIG. 3A, however, the reassembly stores324A-D, in reference toFIG. 3B, are shared between the reception interfaces310A-B and the inbound message ports318A-B and320A-B. In reference toFIG. 3B, the reassembly stores324are partitioned to individual inbound message ports318A-B and320A-B across both of the first reception interface310A and the second reception interface310B. The first inbound message port318A of the first reception interface310A is coupled to output fragments322A to a first set of reassembly stores326A. The second inbound message port320A of the first reception interface310A is coupled to output fragments322A to a second set of reassembly stores326B. The first inbound message port318B of the second reception interface310B is coupled to output fragments322B to a third set of reassembly stores326C. The second inbound message port320B of the second reception interface310B is coupled to output fragments322A to a fourth set of reassembly stores326D. The allocation of the sets of reassembly stores326A-B to the inbound message ports318A-B and320A-B are configurable to affect quality of service. Further,FIG. 3Billustrates that the reassembly stores324A-D are shared across the first and second reception interfaces310A-B. In one embodiment, the reassembly stores324A-D are in a memory shared between the first and second reception interfaces310A-B.

A person of ordinary skill in the art can recognize the system can include any number of reassembly stores326. In one embodiment, the system includes 46 reassembly stores326. Each reassembly store326has a particular amount of memory. In one embodiment, each reassembly store326has 128 bytes of memory. A person of ordinary skill in the art construct reassembly stores326with more or less memory.

FIG. 3Cis a block diagram360of a system for receiving fragments322A to be reassembled into a packet.FIG. 3Cillustrates how a fragment is directed to a particular reassembly store. As described in reference toFIG. 3A, the inbound message ports318A and320A output fragments322A to a packet allocation unit362. The packet allocation unit362receives reassembly store availability information364and determines whether to store the fragment in one of the reassembly stores324. If the packet allocation unit362determines a reassembly store is available, it sends an allocated fragment366to the reassembly stores324. If the packet allocation unit362determines a reassembly store is not available, it generates a retry message368which is sent to the source of the fragment322A, so that the source can resend the fragment322A.

FIG. 4is a flow diagram illustrating a process400for receiving a particular fragment of a packet. In one embodiment, in reference toFIG. 3C, the packet allocation unit362receives the particular fragment and executes the process400. In reference toFIG. 4, first the packet allocation unit receives a fragment (402). The packet allocation unit determines whether a reassembly store has been allocated to the packet corresponding to the received fragment (404) by determining whether it has previously assigned a “reassembly ID” to the packet. If the packet allocation unit362has assigned a reassembly ID to the packet, then it has allocated a reassembly store for the packet. If the packet allocation unit362has not assigned a reassembly ID to the packet, then it has not allocated a reassembly store for the packet.

If a reassembly store is not allocated to the packet, the packet allocation unit362then determines whether a reassembly store is available to be allocated (428) by determining whether it can allocate a reassembly ID to the packet. In one embodiment, the packet allocation unit determines whether it can allocate a new reassembly ID to the packet by comparing the number of assigned reassembly IDs to a number of total available reassembly stores. If the number of assigned reassembly IDs is equal to the number of total available reassembly stores, the packet allocation unit determines it cannot allocate a reassembly ID. If the number of assigned reassembly IDs is less than the number of total available reassembly stores, the packet allocation unit determines it can allocate a reassembly ID.

If the packet allocation unit cannot allocate a reassembly ID to the packet, it sends a retry message (434). If the packet allocation unit can allocate a reassembly ID to the packet, it then determines whether the received fragment is the first fragment of the packet (430). If it is not the first fragment of the packet, the packet allocation unit sends a retry message (434) because the fragment is received out of order. If it is the first fragment of the packet, then the packet allocation unit allocates a reassembly store by assigning a reassembly ID to the packet (432). Then, the system stores a piece of the fragment in the newly allocated reassembly store (408).

If a reassembly store has already been allocated to the packet (404), the packet allocation unit determines if the fragment is received in order (406). If the fragment is not received in order, the packet allocation unit sends a retry message (434). If the packet is received in order, packet allocation unit stores a piece of the fragment in the allocated reassembly store (408). The piece of the fragment is stored in a memory chunk associated with the reassembly store on an on-chip memory. The memory chunks of the reassembly stores are each dynamically allocated to a reassembly ID representing a packet. In one embodiment, the piece of the fragment is 8-bytes and each reassembly store is a memory chunk of 128-bytes. Fragments are stored in pieces because fragments can vary in size. Fragments can be larger than the size of the memory chunk, in some instances.

After storing a piece of the fragment in an allocated reassembly store (408), the packet allocation unit determines whether the on-chip memory of the reassembly store is full by checking whether the memory chunk assigned to the reassembly store is full (410). The packet allocation unit determines if the 128-byte chunk of the on-chip memory allocated to the packet is full after storing the piece of the fragment. If the memory chunk is full, the packet allocation unit copies fragments and pieces of fragments from the on-chip memory to an external memory (412). Then, the packet allocation unit clears the memory chunk allocated to the reassembly store on the on-chip memory to store additional fragments and pieces of fragments (414). However, the packet allocation unit362does not deallocate the reassembly ID from the packet because the system continues to receive fragments from the packet. The packet continues to have a reassembly store allocated to it, even if the memory chunk allocated to it has been cleared, because the packet continues to have an assigned reassembly ID. The packet retains the reassembly ID until the entire packet is received.

Then, the packet allocation unit checks whether the entire fragment has been stored in the reassembly store, either in the on-chip memory, the external memory, or a combination of the on-chip memory and the external memory (416). If the entire fragment has not been stored, the packet allocation unit stores the next piece of the fragment in the allocated reassembly store (408). If the entire fragment has been stored, the packet allocation unit sends an acknowledgement of the fragment (418).

After sending an acknowledgment of the fragment (418), the packet allocation unit determines whether the fragment is the last fragment of the packet (420). When the fragment is the last fragment of the packet, the packet allocation unit copies the fragments from the on-chip memory to an external memory (422). Then, the packet allocation unit clears the on-chip memory (424). Last, the packet allocation unit de-allocates the reassembly store by freeing the reassembly ID, so it can be used by another packet (426). At this point, the packet is received by the semiconductor chip and processed accordingly in a plurality of cores, in one embodiment. The packet allocation unit then waits to receive a next fragment from a new packet (436). If the fragment is not the last fragment, the packet allocation unit waits to receive the next fragment from the same packet (436).

FIG. 5is a block diagram500of a system employing a packet reception unit102including a packet allocation unit362, illustrating an example of fragment reception. The packet allocation unit362is coupled to a memory512that stores fragments in reassembly stores514within the packet reception unit102. The reassembly stores514are a collection individual reassembly stores. Each reassembly store includes a dynamically allocated memory chunk. In one embodiment, the memory chunk is 128-bytes. The packet reception unit102is also coupled with a messaging core526. The messaging core includes a FIFO524. The FIFO524receives and queues fragments from at least one source. Each source sends at least one packet to the packet reception unit102.

In this example illustrating the function of the packet allocation unit362, the FIFO524stores i) Fragment1of Packet A502; ii) Fragment2of Packet A504; iii) Fragment1of Packet B506; and iv) Fragment6of Packet C508in the above recited order and directs them, in order, to the packet reception unit102, which then directs them to the packet allocation unit362.

The packet allocation unit362first processes Fragment1of Packet A502. The packet allocation unit362first dynamically allocates a new reassembly ID for the new packet (Packet A). The packet allocation unit362then assigns the reassembly ID to the empty memory chunk516, such that the memory chunk516stores fragments of Packet A. The packet allocation unit362then directs Fragment1of Packet A502to the empty memory store516over fragment storage line524. A person of ordinary skill in the art can recognize that the number of memory chunks in the reassembly stores can be larger than the number of reassembly IDs.

A person of ordinary skill in the art can recognize that the empty memory chunk516can be allocated for fragments from any packet with a reassembly ID, not just packet A. In this example, the packet allocation unit362allocates the empty memory chunk516to Packet A because the packet has reassembly ID and the fragments of Packet A are the first to need the memory chunk516. In other words, the empty memory chunk516is not dedicated only to Packet A, but rather is dynamically allocated to any packet that requires a memory chunk to store incoming packets, as long as the empty memory chunk516is empty at the time the packet is received and the packet is assigned a reassembly ID.

The packet allocation unit362then processes Fragment2of Packet A504. The packet allocation unit362determines that the previously empty memory chunk516stores fragments of Packet A. The packet allocation unit362then stores Fragment2of Packet A504in memory chunk516over fragment storage line524.

The packet allocation unit362then processes Fragment1of Packet B506. In this example, no reassembly IDs are available for Packet B. The packet allocation unit362determines that no available reassembly stores514are available to receive a fragment from a new packet because a previously available reassembly ID is now allocated for Packet A and other reassembly IDs are available. The packet allocation unit362sends a retry message520, optionally stating that the reassembly stores are full.

The packet allocation unit362then processes Fragment6of Packet C508. The packet allocation unit362determines that allocated reassembly store528is allocated to Packet C, and that it contains Fragment1and Fragment2of a message of Packet C. The packet allocation unit362also determines that allocated reassembly store528does not contain Fragments3-5of Packet C, indicating that Fragment6of Packet C is out of order. The packet allocation unit362generates a retry message, optionally including that Fragment6of Packet C is received out of order.

FIG. 6is a block diagram600illustrating a memory chunk612of a reassembly store receiving fragments604A-E over time. Reassembly store memory chunk612A-E illustrates the same memory chunk612in different states at different points of time. Packet A602is divided into five fragments604A-E at a source (not shown). The packet allocation unit362first receives Fragment1604A. The packet allocation unit362allocates reassembly ID to Packet A602, allocates empty memory chunk612A to the reassembly ID, and then directs Fragment1604A to be stored in the memory chunk612A of the reassembly store.

Next, the packet allocation unit362receives Fragment2604B and then Fragment3604C. Packet allocation unit362determines both Fragment2604B and Fragment3604C are part of Packet A from fragment header data and directs them to be stored in memory chunk612B with Fragment1604A because Packet A has a reassembly ID associated with the memory chunk612B.

Next, packet allocation unit362receives Fragment4604D and then Fragment5604E. Packet allocation unit362determines both Fragment4604D and Fragment5604E are part of Packet A from fragment header data and directs them to be stored in memory chunk612C with Fragment1604A, Fragment2604B and Fragment3604C. The packet allocation unit362receives Fragments2-5of Packet A604B-E in order after it receives Fragment1of Packet A604A because Packet A has a reassembly ID associated with the memory chunk612C.

At this point, memory chunk612D is full. The memory chunk612D then copies the five fragments604A-E into external memory614. External memory614is a memory external to the packet reception unit102, and can be included on a semiconductor chip or an external memory to the semiconductor chip. After the memory chunk612D copies the fragments604A-E into memory, memory chunk612E is cleared. The memory chunk612E may then be used by any packet assigned a reassembly ID. In one embodiment, after the memory chunk612E is cleared, it stores the same packet, i.e. packet A602.

A multi-source/multi-destination retry protocol, in addition to receiving fragments and reassembling them as packets, can also transmit a packet as fragments. The packet has to be broken into multiple fragments and transmitted over a network.

FIG. 7Ais a block diagram700of a system employed to send fragments over a network. In one embodiment, the system is embodied in a semiconductor chip. A restriction verifier704receives and stores fragments702before sending eligible fragments708to the transmit silo710. The transmit silo710sends a transmit silo status706to the restriction verifier704. The transmit silo710receives a fragment and sends the fragment712to its destination. The transmit silo holds the sent fragment in its memory until it receives an acknowledgement714that the destination received the fragment. In this manner, the transmit silo determine whether a fragment has not been acknowledged and resend the fragment, and also prevent fragments712from being sent out of order, in combination with the restriction verifier704. When the transmit silo710receives an acknowledgment714of a fragment, the transmit silo710deletes the fragment and updates the transmit silo status706.

The restriction verifier704determines whether each particular fragment can be transmitted to and stored in the transmit silo710. The restriction verifier704can be configured to limit i) a number of outstanding fragments stored in the transmit silo710; ii) a number of outstanding packets stored in the transmit silo710; iii) a number of outstanding fragments per destination stored in the transmit silo710; iv) a number of outstanding packets per destination stored in the transmit silo710; v) a number of outstanding fragments per mailbox of the destination stored in the transmit silo710; vi) a number of outstanding packets per mailbox of the destination stored in the transmit silo710. In an embodiment where the transmit silo710is also coupled with a controller, the restriction verifier can further be configured to limit: vii) a number of outstanding packets per controller stored in the transmit silo710, viii) a number of outstanding fragments per controller stored in the transmit silo710; ix) a number of outstanding packets per destination per controller stored in the transmit silo710; x) a number of outstanding fragments per destination per controller stored in the transmit silo710; xi) a number of outstanding packets per mailbox of the destination per controller stored in the transmit silo710; and xii) a number of outstanding fragments per mailbox of the destination per controller stored in the transmit silo710. Based on one or more of these restrictions, the restriction verifier prevents the transmit silo from sending packets out of order.

FIG. 7Bis a block diagram720of a system employed to send fragments over a network including a controller722. The controller722receives packets724and is configured to divide, or break-up, the packets724into multiple smaller fragments702. The controller722sends the fragments702to the restriction verifier704, which forwards the fragments702to the transmit silo when the fragments and transmit silo meet certain conditions, as described in reference toFIG. 7A.

FIG. 7Cis a block diagram740of a system employed to send fragments over a network including a settings file744. The settings file744receives and records user configuration742. The settings file includes settings for any of the categories the restriction verifier is configured to limit, as described in reference toFIG. 7A. In reference toFIG. 7C, the settings file sends restriction verifier configuration746based on the settings file744to the restriction verifier704. The restriction verifier704, upon receiving the restriction verifier configuration746, reconfigures itself to accommodate the updated settings file744.

FIG. 8is a block diagram800of a system employed to transmit fragments after receiving packets from a packet output unit802at a first and second transmit interface808A-B. In one embodiment, the packet output unit802is embodied in a semiconductor chip. The packet output unit802pushes packets to packet output ports806A-D through a first and second packet engine804A-B. A person of ordinary skill in the art can employ any number of packet engines804and packet output ports806. The packet output ports806A-D are coupled with the first transmit interface808A and the second transmit interface808B.FIG. 8shows a first packet output port806A and third packet output port806C coupled to the first transmit interface808A via first and third controllers722A and722C, respectively.FIG. 8also shows a second packet output port806B and fourth packet output port806D coupled to the second transmit interface808B via the second and fourth controller722B and722D, respectively. However, a person of ordinary skill in the art can recognize any configuration of the coupling of the packet output ports806A-D to the transmit interfaces808A-B is employable.

Focusing on the operation of the first transmit interface, the first and second controllers722A and722B receive packets from the packet output unit802. A person of ordinary skill in the art can employ any number of controllers regardless of the number of restriction verifiers and transmit silos710A. The controllers722A-B break the received packets into fragments and forward them to the restriction verifier704. The restriction verifier704determines, as described in reference toFIG. 7A, whether the fragments can be sent to the transmit silo710A. If so, the restriction verifier704forwards the packet to the transmit silo710A. The transmit silo issues a request to the transmit messaging core818A to send the fragment. The transmit core transmits the fragment as data to network820A to QLM824A.

When the QLM receives an acknowledgement from the destination that the fragment has been received, the QLM transmits the acknowledgment to the transmit messaging core818A as data from network822A. The transmit messaging core818A then sends the acknowledgment to the transmit silo710A as responses816A. The transmit silo710A then deletes the fragment from its memory, freeing up a silo spot for another fragment.

A person of ordinary skill in the art can recognize that the second transmit interface808B is similar in operation to the first transmit interface808A.

A person of ordinary skill in the art can recognize that a transmit interface and a receiving interface can be useful if combined in the same system.FIG. 9is a block diagram900of a system configured to embody both a transmit interface and receiving interface as described above. In one embodiment, the system is embodied in a semiconductor chip. The system includes the packet input unit304and the packet input ports306A-D as described in reference toFIG. 3. The packet input ports306A-D are coupled to receive data from inbound message ports318A-B and320A-B, respectively.

Referring toFIG. 9, the system further includes the packet output unit802and the packet output ports806A-D, as described in reference toFIG. 8. The packet output ports806A-D are coupled to output packets to outbound message controllers722A-D, respectively, as described in reference toFIGS. 7 and 8.

Referring toFIG. 9, the system further includes multi-source/multi-destination retry interfaces902A and902B. The multi-source/multi-destination retry interfaces902A and902B are coupled with the inbound message ports318A-B and320A-B and packet output ports722A-D. A person of ordinary skill in the art can couple the inbound message ports318A-B and320A-B and packet output ports722A-D with the different multi-source/multi-destination retry interfaces902A and902B in any combination.