A HIGH PERFORMANCE INTEROPERABLE NETWORK COMMUNICATIONS ARCHITECTURE (INCA)

Abstract: An interoperable, software only network communications architecture (INCA) is presented that improves the internal throughput of network communicated data of workstation and PC class computers at the user level, application program level, by 260 % to 760 %. The architecture is unique because it is interoperable with all existing programs, computers and networks requiring minimal effort to set up and use. INCA operates by mapping network data between the application and operating address space without copying the data, integrating all protocol execution into a single processing loop in the application address space, performing protocol checksumming on a machine word size of data within the protocol execution loop, and providing an application program interface very similar to existing application program interfaces. The network interface driver functions are altered to set up network data transfers to and from the application address space without copying of the data to the OS address space, while buffer management, application to message multiplexing/demultiplexing and security functions are also being performed by the modified network interface driver software. Protocols are executed in the application address space in a single integrated protocol processing loop that interfaces directly to the INCA NI driver on one end and to the application on the other end in order to minimize the amount of times that network communicated data must travel across the internal memory bus. A familiar looking application program interface is provided that differs only slightly from existing application program interfaces which allows existing applications to use the new software with a minimum of effort and cost.

Description: A HIGH PERFORMANCE INTEROPERABLE NETWORK COMMUNICATIONS ARCHITECTURE (INCA) FIELD OF THE INVENTION This invention relates generally to computer network communications. More particularly, the present invention relates to a method to improve the internal computer throughput rate of network communicated data. BACKGROUND OF THE INVENTION Network technology has advanced in the last few years from transmitting data at 10 million bit per second (Mbps) to near 1 Gigabit per second (Gbps). At the same time, Central Processing Unit (CPU) technology inside computers has advanced from a clock rate of 10 Million cycles (Hertz) per second (MHz) to 500 MHz. Despite the 500% to 1000% increase in network and CPU capabilities, the execution rate of programs that receive data over a network has only increased by a mere 100%, to a rate of approximately 2 Mbps. In addition, the internal computer delays associated with processing network communicated data have decreased only marginally despite orders of magnitude increase in network and CPU capabilities. Somewhere between the network interface (NI) and the CPU, the internal hardware and software architecture of computers is severely restricting data rates at the application program level and thereby negating network and CPU technology advances for network communication. As a result, very few network communication benefits have resulted from the faster network and CPU technologies. Present research and prototype systems aimed at increasing internal computer throughput and reducing internal processing delay of network communicated data have all done so without increasing application level data throughput, or at the expense of interoperability, or both. For -l- purposes of this specification, network communicated data includes all matter that is communicated over a network. Present research solutions and custom system implementations increase data throughput between the NI of the computer and the network. However, the data throughput of the application programs is either not increased, or is only increased by requiring new or highly modified versions of several or all of the following: application programs, computer operating systems (OSs), internal machine architectures, communications protocols and NIs. In short, interoperability with all existing computer systems, programs and networks is lost. A range of problems associated with network operations is still present. The present state of the art of increasing the performance of internal computer architectures : 1. Prevents computers from utilizing the tremendous advances in network and CPU technologies; 2. Fails to solve the problem by focusing mainly on NI to network transfer rate increases rather than on increasing network communicated data throughput at the application program level; 3. Severely restricts computer network communicated data throughput at the application program level to a fraction of existing low speed network and CPU capabilities; 4. Prevents the use of available and the implementation of new computer applications that require high network communicated data throughput at the application program interface or low internal computer processing delays; 5. Requires a massive reinvestment by the computer user community in new machines, software programs and network technologies because of a lack of interoperability with existing computer systems and components. A need therefore exists for higher network communicated data throughput inside computers to allow high data rate and low processing delay network communication while maintaining interoperability with existing application programs, computers, OSs, NIs, networks and communication protocols. SUMMARY OF THE INVENTION It is an object of the present invention to provide high network communicated data throughput and low network communicated data processing delay inside computers. It is a further object of the present invention to maintain interoperability with all existing computer programs, computers, OSs, NIs, networks and communication protocols. It is a further object of the present invention to be useable on all existing personal computer (PC) and workstation (WS) class computers, as well as on most other, greater capability computer systems with only minor software modifications to the existing application programs and/or OS. It is a further object of the present invention to dramatically increase the network communicated data throughput at the application level (not just NI to network). It is a further object of the present invention to dramatically increase the network communicated data throughput at the application level (not just NI to network level) for small messages (less than or equal 200 bytes) and for large messages. It is a further object of the present invention to speed up communication protocol processing for all levels and types of protocols. It is a further object of the present invention to reduce the amount of times network communicated data is sent across the internal computer memory bus. It is a further object of the present invention to increase the performance of non networking applications on networked computers by processing network management messages and messages not addressed to the machine at least four times faster than presently processed. It is a further object of the present invention to be interoperable with existing computer systems and network components and to not require costly changes to these components to improve performance. It is a further object of the present invention to require only minor software changes to application program interfaces (APIs) or NI drivers. It is a further object of the present invention to be installed, tested and operational in a short amount of time, i.e., minutes to hours. It is a further object of the present invention to provide for application (not just NI to and from the network) throughput of network communicated data at high speed network transmission rates. It is a further object of the present invention to enable the use and implementation of application programs that require high application level network communicated data throughput and low internal computer network communicated data handling delay. It is a further obj ect of the present invention to allow the utilization of high speed network and CPU technologies by enabling applications to process data at high speed network rates. The present invention is a library of programs comprising three main programs integrated into one software library: a computer NI driver, an integrated protocol processing (IPP) loop and an API. The INCA NI driver comprises software that controls the NI hardware and transfers the network messages and data in the messages from the network to the computer's memory. The IPP loop software performs communication protocol processing functions such as error handling, addressing, reassembly and data extraction from the network message. The API software passes the network communicated data sent via the network to the application program that needs the network communicated data. The same process works in reverse for transmitting network communicated data from the application over the network to a remote computer. The existing computer components, e.g., the NI, CPU, main memory, direct memory access (DMA) and OS, are used with control or execution of these resources being altered by the INCA software functions. The advantage of using the present invention is that the existing, inefficient network communicated data handling is greatly enhanced without requiring any additional hardware or major software modifications to the computer system. INCA speeds up internal network communicated data handling to such a degree that data can be transferred to and from the application programs, via the network interface, at speeds approaching that of high speed network communicated data transmission rates. The CPU is required to operate more frequently on more data, utilizing the increased CPU capabilities . The present invention greatly reduces the number of times network communicated data must be transferred across the internal computer memory bus and greatly speeds up the processing of communication protocols, with particular improvement in the checksumming function. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 shows an overview of a typical existing network communication system. Figure 2 shows an overview of the INCA network communication system. Figure 3 shows an overview of the endpoint mechanism. Figure 4a shows examples of the typical non INCA, non IPP for-loops used for protocol processing. Figure 4b shows an example of a single, integrated INCA IPP for-loop used for protocol processing. Figure 5 shows the INCA IPP stages of protocol execution. Figure 6 shows the INCA IPP method of integrating various protocols into a single execution loop. Figure 7 shows the alternative "system calls" comprising INCA's API. Figure 8 shows INCA's performance improvement on WS class computers. Figure 9 shows INCA's small message size performance improvement on PC class computers. Figure 10 shows INCA's performance improvement with all standard message sizes on PC class computers. Figure 11 shows INCA's management and control flow. DETAILED DESCRIPTION OF THE INVENTION Referring to Figure 1, an overview of a typical existing network communication system inside a computer is shown. When a network message 102 is received by the network interface (NI) 104, the NI 104 sends an interrupt signal to the operating system (OS) 106. The network communicated data is then copied from the NI 104 into the OS message buffers 110 which are located in the OS memory address space 108. Once the network communicated data is available in the OS message buffers 110, the OS 106 typically copies the network communicated data into the Internet Protocol (IP) address space 112. Once IP processing is completed, the network communicated data is copied to the User Datagram Protocol (UDP)/Transport Control Protocol (TCP) address space 114. Once the UDP processing is completed, if any additional protocols are used external to the application program, the network communicated data is copied to the Other Protocol address space 116 where the network communicated data is processed further. The network communicated data is then copied to the application program interface (API) address space 118. Finally, the application reads the network communicated data which requires another copy of the network communicated data into the application program memory address space 120. As illustrated in Figure 1, copying of the network communicated data occurs numerous times in the normal course of present operations. Referring to Figure 2, an overview of the INCA network communication system is shown. Contrasting INCA to the typical network communication system in Figure 1, it is evident INCA eliminates several data copying steps and as a result, INCA performs in a more efficient manner for increasing a network's data throughput rate. In addition, INCA implements several other efficiency improving steps which will be discussed later. As shown in Figure 2, the present invention comprises three main software components : an INCA NI driver 202, an INCA IPP (execution loop) 204 and an INCA API 206. These components reside inside computer message buffers 208 together with the current computer software components such as application programs 210, operating systems (OS) 212, communication protocols, and the current computer hardware components such as the NI 214, system memory bus 216 and 218, OS memory address space 220 and application program memory address space 224 and one or more CPUs, disks, etc. The first component, the INCA NI driver 202, is a software set of programming language functions that supports direct access to the NI 214 and performs the following functions: 1. Controls the NI device 214 and other involved devices (e.g., DMA) to set up a transfer of network messages 222 from the NI 214 to OS memory address space 220; 2. Manages the NI 214 to computer memory transfer; 3. Demultiplexes incoming network messages to the proper recipient (i.e., application); 4. Provides protection of network communicated data from different applications; 5. Transfers the network communicated data to the application program memory address space 224; 6. Interfaces to the INCA IPP 204 to control protocol execution; 7. Interfaces to the INCA API 206 to control application program network access; 8. Relinquishes any control over the NI 214 upon completion of message handling. These eight functions are performed by the INCA NI driver 202 component for reception of network messages. In the case of the transmission of network communicated data from one computer to another computer over a network, the eight functions are performed in reverse order. Since the transmitting case is the same as the receiving case, it is not discussed separately. The following description of the INCA NI driver functions is for the receiving case. The INCA NI driver component may include software linked to the INCA software library which is not typically considered NI device driver code. The first two functions, control and management of transferring network communicated data from the NI device to internal computer memory (i.e., random access memory - RAM), or some other type of memory (e.g. cache, hard disk) are initiated when a message arrives at the computer from a network connection. The NI hardware signals the arrival of a message, typically a hardware interrupt. The message arrival notification signal is received by the INCA NI driver 202. Upon receipt of a message arrival notification, the INCA NI driver 202 takes over control of the NI device (e.g., Asynchronous Transfer Mode (ATM) network card), and sets up the registers, firmware, etc., of the device to transfer the message from the device to main memory. Transferring the message or network communicated data is in response to the call functions of either an application program interface, an application program, a network interface device or a network interface driver. The transfer can be accomplished via two main methods, via DMA or programmed input/output (PIO). In the case of DMA transfers of network communicated data between the NI 214 and OS memory address space 220, the INCA NI driver 202 sets up memory and NI device addresses, message buffer sizes/alignments/addresses, and signals the start and completion of every transfer. If an error occurs, the INCA NI driver 202 attempts to resolve the error through such actions as reinitializing a DMA transfer or repeating transfers. At the completion of DMA transfers, the INCA NI driver 202 releases control of any DMA and NI 214, releases any allocated message buffer memory 208, and ceases execution. In the case of PIO, the CPU transfers every byte of network communicated data from the NI to computer memory. The INCA NI driver 202 provides the necessary parameters, memory and NI addresses, transfer sizes and buffer sizes/alignments/addresses for the CPU to transfer the network communicated data to computer memory. In the preferred embodiment, the OS 212 manages the address mapping between the virtual addresses of message buffers specified by an application and the physical addresses required for actual transmission and reception. In an alternative embodiment, the application program manages the address mapping. In yet another embodiment, hardware, such as the NI 214, manages the address mapping. The OS 212 performs virtual memory (VM) management through the use of a memory mapping function such as the UNIX OS mmap() function which maps the message buffers 208 in OS memory address space 220 to the application program memory address space 224. Virtual to physical address translations are therefore handled in the existing OS manner. To enable the OS 212 to perform VM management and address translation, the INCA NI driver 202 must allocate message buffers 208 in the OS memory address space 220 initially, as well as in the application program memory address space 224 to allow the OS 212 to make the required mappings and perform its VM management functions. The INCA NI driver 202 performs these functions as soon as the message arrival signal is received by the INCA NI driver 202. The address locations of the message buffers 208 containing the network communicated data are therefore mapped to the VM locations in the IPP address space 218, with only one physical memory location, hence no copying of the network communicated data is required. Message buffer management and DMA management are performed by the INCA NI driver 202. The INCA NI driver 202 allocates buffer space when an application 210 calls the INCA NI driver 202 with an INCA open() call which opens the INCA NI driver 202 to initialize the DMA transfer. The INCA NI driver 202 receives the NI message interrupt signal and starts the INCA NI driver 202 which causes message buffer allocation to occur in message buffers 208 and in IPP address space 226. The INCA NI driver 202 uses the 4 KB memory page size provided by most OS VM systems, and allocates message buffers in 2 KB increments. Each message is aligned to the beginning of the 2 KB buffer with a single message per buffer for messages smaller than 2 KB. For messages larger than 2 KB, multiple buffers are used beginning with the first and intermediate buffers filled from start to end. The last buffer contains the remnant of the message starting from the 2 KB buffer VM address position. For messages equal to or less than 2 KB, one buffer is allocated and the contents are aligned with the first byte placed at the start of the 2 KB buffer address space. In order to make the mapping from the OS space to user space easier and in order to avoid implementing more memory management functionality into INCA, the message buffers are "pinned" or assigned to fixed physical memory locations in either the application or OS address space. The application specifies message buffers using offsets in the buffer region, which the INCA NI driver 202 can easily bounds-check and translate. By using fixed physical memory locations, the INCA NI driver 202 will not issue illegal DMA access. Since INCA has complete control over the size, location, and alignment of physical buffers, a variety of buffer management schemes are possible. All buffers may be part of a system-wide pool, allocated autonomously by each domain (e.g., applications, OS), located in a shared VM region, or they may reside outside of main memory on a NI device. Physical buffers are of a fixed size to simplify and speed allocation. The INCA NI driver memory management is immutable, it allows the transparent use of page remapping, shared virtual memory, and other VM techniques for the cross-domain transfer of network communicated data. Virtual copying with the mmap() function is used to make domain crossings as efficient as possible, by avoiding physical memory bus transfer copying between the OS 212 and application program memory address space 224. The third function of the INCA NI driver 202 is message demultiplexing (for receiving) and multiplexing (for transmitting). Not all applications on a machine may be using the INCA software to communicate over the network. There may be a mix of INCA and non INCA communicating applications in which case the INCA NI driver 202 must also route messages to the non INCA NI driver or the non INCA protocol processing software, or to some other non INCA software. The INCA NI driver 202 maintains a list of INCA application program addresses known as endpoints. Endpoints provide some of the information required to carry out the INCA NI driver component functions. Referring to Figure 3, an overview of the endpoint mechanism is shown. Endpoints 302 bear some resemblance to conventional sockets or ports. A separate endpoint is established and maintained for each application and each network connection for each application. For applications without INCA endpoint addresses, non INCA networking applications, the INCA NI driver passes the message arrival notification to the non INCA NI driver. Each application that wishes to access the network first requests one or more endpoints 302 through the INCA alternative API "system calls" . The INCA NI driver then associates a set of send 304, receive 306, and free 308 message queues with each endpoint through the use of two INCA "system calls", inca_create_endpoint() and inca_create_chan(). The application program memory address space 300 contains the network communicated data and the endpoint message queues (endpoint send/receive free queues 304, 306, 308) which contain descriptors for network messages that are to be sent or that have been received. In order to send, an application program composes a network message in one or more transmit buffers in its address space and pushes a descriptor onto the send queue 304 using the INCA API "system calls". The descriptor contains pointers to the transmit buffers, their lengths and a destination tag. The INCA NI driver picks up the descriptor, allocates virtual addresses for message buffers in OS memory address space and sets up DMA addresses. The INCA NI driver then transfers the network communicated data directly from the application program memory address space message buffers to the network. If the network is backed up, the INCA NI driver will simply leave the descriptor in the queue and eventually notifies the user application process to slow down or cease transmitting when the queue is near full. The INCA NI driver provides a mechanism to indicate whether a message in the queue has been injected into the network, typically by setting a flag in the descriptor. This indicates that the associated send buffer can be reused. When the INCA NI driver receives network communicated data, it examines the message header and matches it with the message tags to determine the correct destination endpoint. The INCA NI driver then pops free buffer descriptors off the appropriate free queue 308, translates the virtual addresses, transfers the network communicated data into the message buffers in OS memory address space, maps the memory locations to the application program memory address space and transfers a descriptor onto the receive queue 306. Each endpoint contains all states associated with an application's network "port". Preparing an endpoint for use requires initializing handler-table entries, setting an endpoint tag, establishing translation table mappings to destination endpoints, and setting the virtual -memory segment base address and length. The user application program uses the API routine calls "ioctl()" and "mmapO" to pass on any required endpoint data and provide the VM address mapping of the OS message buffers to the application program memory address space locations. Once this has been achieved, the user application is prepared to transmit and receive network communicated data directly into application program memory address space. Each endpoint 302 is associated with a buffer area that is pinned to contiguous physical memory and holds all buffers used with that endpoint. Message descriptors contain offsets in the buffer area (instead of full virtual addresses) which are bounds-checked and added to the physical base address of the buffer area by the INCA NI driver. In summary, endpoints and their associated INCA NI driver "system calls" set up an OS-Bypass channel for routing network communicated data address locations to and from memory to the correct applications. Providing some security is the fourth function performed by the INCA NI driver. To assure that only the correct applications access the message data, application program identifiers to endpoints and endpoints to message data mappings are maintained. An application can only access message data in the endpoint message queues where the identifiers of endpoint(s) of message queues matches the identifiers of endpoints for that application. Any access to network communicated data must come from the intended recipient application or in the case of transmitting network communicated data, access to the network communicated data must come from the transmitting application. Once the network communicated data transfer is set up and demultiplexing of messages is complete, the INC A NI driver performs the function of transferring the network communicated data from the OS memory address space to the receiving application program memory address space. This transfer is required since all present NI devices come under the ownership of the computer OS and any network communicated data transferred via a NI device is allocated to the OS virtual or physical memory address space. INCA makes this transfer without requiring any movement or copying of the network communicated data, thereby avoiding costly data copying. The transfer is made via a mapping of the memory addresses of the network communicated data within the OS memory address space to memory (addresses) within the application program memory address space. For UNIX OS based systems, the UNIX mmap() function is used by the INCA NI driver to perform the transferring of network communicated data to the application address space, mapping the addresses of the network data in the OS address space to the application address space. The sixth function of the INCA NI driver is to interface to INCA's second component, the IPP loop software. Once network communicated data is available in the computer' s memory, the INCA NI driver notifies the IPP software that network communicated data is available for protocol processing. The notification includes passing a number of parameters to provide needed details for the IPP software. The parameters include the addresses of the network communicated data and the endpoints to determine the recipient application program. The IPP component of the invention is an extension of Integrated Layer Processing (ILP) , performing the functions of communications protocol processing. IPP includes protocols above the transport layer, including presentation layer and application layer protocol processing and places the ILP loop into one integrated execution path with the INCA NI driver and API software. In current systems, communication protocol processing is conducted as a part of and under the control of the OS in OS memory address space. Each protocol is a separate process requiring all the overhead of non integrated, individual processes executed in a serial fashion. Existing and research implementations do not integrate ILP with an NI OS-Bypass message handler and driver, do not integrate protocol processing into a single IPP loop, nor do they execute protocols in user application program memory space. Protocol execution by the existing OSs and under the control of the OS are not used by INCA's IPP component. INCA's IPP performs protocol execution using the INCA software library implementations of the protocols linked to the application in the application program memory address space. Referring to Figure 4a, a depiction of a "C" code example of typical protocol processing code is shown. Before the code can be executed, the network communicated data must be copied to each protocol's memory address space. When the code is compiled to run on a reduced instruction set computer (RISC) CPU, the network message data manipulation steps results in the machine instructions noted in the comments. First, the protocol software process, e.g., the Internet Protocol (IP) software, is initialized and the network communicated data is copied from the OS message buffer memory area to the IP process execution memory area. Each time a word of network communicated data is manipulated, the word is loaded and stored into memory. Upon completion of the first protocol, the second protocol process, e.g., the TCP software, is initialized and the network communicated data is copied to this protocol's execution area in memory. Once again, each time a word of network communicated data is manipulated, the word is loaded and stored. This process continues until all protocol processing is complete. Referring to Figure 4b, the INCA system with the IPP method is shown, where each word is loaded and stored only once, even though it is manipulated twice. Each protocol's software execution loop is executed in one larger loop, eliminating one load and one store per word of data. This is possible because the data word remains in a register between the two data manipulations. Integrating the protocol processing for-loops results in the elimination of one load and one store per word of network communicated data. The IPP method of performing all protocol processing as one integrated process, also eliminates the copying of all network communicated data between the initialization and completion of each separate communications protocol used (e.g., copying the data to IP protocol address space, then copying the data to UDP or TCP protocol address space, etc.). In addition, the INCA IPP protocol processing uses an optimized protocol checksum processing routine that calculates checksums on a word (e.g., 4 to 8 bytes depending upon the machine) of network communicated data at a time, rather than the existing method of one byte at a time. INCA's IPP checksum calculation is roughly four times faster than existing checksum calculations. For small message sizes of less than or equal to 200 bytes, which comprise some 90% or more of all network messages, INCA's IPP checksum routine greatly speeds up the processing of small messages since checksum calculation is the maj ority of calculations required for small messages. The IPP component divides protocol processing of network messages into three categories: data manipulation - reading and writing application data, header processing - reading, writing headers and manipulating the headers of protocols that come after this protocol, and external behavior - passing messages to adjacent layers, initiating messages such as acknowledgments, invoking non-message operations on other layers such as passing congestion control information, and updating protocol state such as updating the sequence number associated with a connection to reflect that a message with the previous number has been received. Referring to Figure 5, the INCA IPP component executes the protocols in three stages in a processing loop: an initial stage 502, a data manipulation stage 504 and a final stage 506. The initial stages of a series of lay ers are executed serially, then the integrated data manipulations take place in one shared stage and then the final stages are executed serially. Interoperability with existing protocol combinations such as IP, TCP, UDP and External Data Representation (XDR) combinations requires the IPP software to contain some serial protocol function processing of the network communicated data in order to meet the data processing ordering requirements of these existing protocols. Message processing tasks are executed in the appropriate stages to satisfy the ordering constraints. Header processing is assigned to the initial stage. Data manipulation is assigned to the integrated stage. Header processing for sending and external behavior (e.g., error handling) are assigned to the final stage. Referring to Figures 5 and 6, INCA's IPP method of integrating multiple protocols is shown. The protocols of protocol A 610 and protocol B 620 are combined and INCA's IPP method integrates the combination of protocol A and B 628. The initial stages 612, 622 are executed serially 502 (as shown in Figure 5), then the integrated data manipulation 504 is executed 614, 624, and then the final stages 616, 626 are executed serially 506. Executing the tasks in the appropriate stages ensures that the external constraints protocols impose on each other cannot conflict with their internal constraints. The ILP software starts up directly after reception of network communicated data into the message buffer. It does the integrated checksumming on the network communicated data in the initial stage 502, protocol data manipulations and Host/Network byte order conversions in the middle integrated stage 504, and TCP type flow control and error handling in the final stage 506. The concept of delayed checksumming has been included in the loop. In the case of IP fragments, the checksumming is done only after reassembly. Message fragments are transmitted in the reverse order, i.e., the last fragment is transmitted first, to make the time of checksumming less in the case of UDP. Once the data processing is complete, the packets are multiplexed to the corresponding protocol ports set up by the API. The IPP protocol library software consists of software functions that implement the protocol processing loop and other pieces of protocol control settings such as fragmentation, and in the case of TCP, maintaining the window, setting up time-out and retransmission etc. The TCP library has been implemented with a timer mechanism based on the real-time clock and a Finite State Machine (FSM) implementation. The INCA IPP component integrates protocol processing into one process which executes in the application program's memory address space. Consequently, the number of copies of the network communicated data to and from memory are greatly reduced, as can be seen by comparing Figures 1 and 2. Thus the speed and efficiency with which data can be accessed and used by an application is increased. These repeated transfers across the CPU/memory data path frequently dominate the time required to process a message, and thus the network communications throughput. The IPP component therefore speeds up network communicated data processing through the elimination of time consuming memory bus transfers of the network communicated data. With the use of the IPP loop's high performance protocol checksum software, protocol processing time of network communicated data is greatly reduced, providing a large part of the performance improvement of the invention. Interoperability requires the IPP loops to use and execute existing protocols as well as future protocols. The INCA software libraries accommodate the integration of all existing protocols and future protocols into the IPP component of INCA and provide integration with the INCA NI driver component functions. The seventh function of the INCA NI driver is to interface to INCA's third component, the API. This interface provides the application with network access for sending data and also sets up application address space parameters for transfer of network data to an application. The API component of the invention provides the interface between the existing application programs and the new INCA components. The API allows existing applications to call on the INCA software to perform network communicated data handling in the new, high performance manner. The API limits the changes required to existing application programs to minor name changes to their current API and thereby provides interoperability with existing application programs. The INCA API allows the application to: open a network connection by opening the NI device, specify parameters to the INCA NI driver, specify the protocols to use and their options, set the characteristics of the data transfer between the application and the network using IPP and the INCA NI driver, and detect the arrival of messages by polling the receive queue, by blocking until a message arrives, or by receiving an asynchronous notification on message arrival (e.g., a signal from the INCA NI driver). The API also provides low level communication primitives in which network message reception and transmission can be tested and measured. Referring to Figure 7, the INCA API uses alternative "system call" type programming code structures 701 to 712 in place of the current OS system calls such as socket(), connect(), listen() and bin(). The alternative calls are used to bypass the current OS system calls. In an alternative embodiment, the operating system can include the alternative system calls. The new system calls initiate the INCA IPP and INCA NI driver software library programs to provide the necessary API functionality. The API set of system calls 701 to 712 simplifies the application programming required to use the invention, the INCA software library, to renaming the existing calls by placing "inca " in front of the existing calls. As depicted in Figure 7, the API provides the basic commands like "open()", "close()", "read()", "write()", etc., similar to existing system networking APIs. The "oρen()" call 701, 702 and 709 will perform the following for the user: 1. Open the device for operation; 2. Create the OS Bypass structure and set up a DMA channel for user to network transfer; 3. Map the communication segment from the driver buffer to the user buffer; 4. Open an unique channel for communication between two communicating entities; 5. Fill up this buffer (incabuffer), which will be used in calls to read, write, etc. The "close()" call 703, 704 and 710 will perform the following for the user: 1. Free storage allocated for the buffers; 2. Destroy the communication channel; 3. Unmap the communication segment; 4. Remove the DMA mapping; 5. Close the device used by that particular application. The "read()" call 705 and 706, with a pointer to "incabuffer" as the parameter, will perform the following for the user: 1. Receive any pending packet from the INCA device, which has been transferred via DMA; 2. Pass the received packet (if not fragmented) through the IPP loop; or if fragmented, pass the received packet through a special IPP loop (inlined) which will do IPP, but will keep track of the fragments and pass it on only when the packet is complete and reassembled; 3. Close the read call (the packet is ready for delivery to the application). The "writeO" call 707 and 708, with a pointer to "incabuffer" as the parameter, will perform the following for the user: 1. Create the header for IP/UDP/TCP; 2. Perform IPP processing on the packet; 3. Fragment the packet if it is too large for UDP or IP by passing the received packet through a special IPP loop (inlined), which will keep track of the fragments and pass it on only when the packet fragmentation is complete; 4. Pass on the IP packets for transmission onto the NI. Parameters passed by the application to the IPP and NI driver components of INCA inside the () of the calls include application message identifiers and endpoints. This parameter allows the IPP and INCA NI driver components to multiplex and demultiplex messages to the intended applications or network address. A more enhanced API could include calls or parameters within calls to set up a connection using TCP and also functions that help in the implementation of client/server applications like "listen" etc. Although INCA's API can be located anywhere between the networking application program and any protocol of the protocol stack, the API is typically located between the application and the INCA IPP component. In current OSs, the API typically sits between the session layer (e.g., socket system calls) and the application. Ideally, the API sits between the application and all other communication protocols and functions. In current systems and application programs, many times the application also contains the application layer protocol (i.e., Hypertext Transport Protocol - HTTP), the presentation layer protocol functions (i.e., XDR like data manipulation functions), and only the socket or streams system call is the API. This is not necessarily ideal from a user perspective. By integrating presentation and application protocol functions into the application, any change in these functions necessitates an application program "upgrade" at additional procurement, installation time and maintenance cost. INCA can incorporate all the application, presentation and session (replacement for socket and streams calls) functions into the IPP loop. In the future, this can even be accomplished dynamically, at run time, through application selection of the protocol stack configuration. The API provides the application the link to utilize the INCA high performance network communication subsystem as opposed to using the existing system API and existing network communicated data processing subsystem. The existing system API could also be used to interface to the INCA IPP and INCA NI driver components if the existing system API is modified to interface to the INCA IPP protocol processing and INCA NI driver network interface and instead of the existing system protocol processing and network interface software. The final function is relinquishing control of the NI device. The INCA NI driver uses an alternative "system call", inca_closedev(), in place of the current OS system call to close the NI device and to relinquish control of the NI device. When the NI device has no more network communicated data to be transferred, the INCA NI driver relinquishes control of the NI device to the computer's OS so that other software can use the NI device. Hardware or software interrupts are typically used to signal that the NI device has no more network communicated data to transfer. Upon detection of the no more network communicated data to transfer signal, the INCA NI driver sets the end memory address of the network communicated data buffers. For the mapping of the network communicated data into the application address space, the INCA NI driver performs any required message buffer alignment and passes the network communicated data address range to the IPP software. The NI device is set to a known state and the OS is notified that the device is available to be scheduled for use by other software processes. To illustrate the workings and ease of use of the invention, the following description is provided. The INCA software library is loaded unto the computer's hard disk. If the machine's NI device drivers are implemented as loadable modules, no NI device driver modifications are necessary. If the INCA NI driver is integrated into the OS without being a separate module, the INCA NI driver software is integrated into the OS through a recompilation of the OS. This does not alter the operation of existing programs or the OS , but only adds the INCA NI interface. For those networking applications that will use the INCA software library for network message handling, the API system calls are changed to the INCA API system calls. This procedure can be accomplished via a number of approaches. Once accomplished, all is complete and system operation can resume. These two steps, INCA NI driver insertion and renaming the API calls, provide a system by system approach to using the INCA networking software. System vendors or system administrators are the most likely candidates to use this method. An alternative approach to the above steps is to integrate the entire library into the applications desiring to perform networking with the INCA software. This provides an application by application approach to using INCA. Application program vendors or individual users are the most likely candidates to use this method. Either way, the entire procedure can be accomplished in minutes to hours depending upon the implementor's familiarity with the applications, OS and NI device drivers. For existing applications that do not have their system calls modified, INCA allows the traditional network system interfaces (e.g., sockets) with the applications. Referring to Figure 11, the order of events for receiving network communicated data over the network is shown. Once system operation begins and network messages are received, the order of events for receiving data over the network are as follows: the NI device driver receives a message arrival notification from the NI hardware 1100, typically via a hardware interrupt. The message arrival notification signal is received by the OS and initiates the opening of the INCA enhanced NI driver - the INCA NI driver 1102. The INCA NI driver determines if the network message is for an application that can use the INCA software library to receive messages 1104. If the application is not "INCA aware," control is handed over to the OS for further handling 1108. If the application can use INCA to communicate, the INCA NI driver takes control of the NI device 1106 (e.g., Asynchronous Transfer Mode - ATM network card) and sets up the registers, firmware, etc., of the device to transfer the network communicated data from the NI device to internal computer memory 1110. The INCA NI driver uses an alternative "system call" type programming code structure, inca_opendev(), in place of the current OS system calls to take over the device and set up the data transfer from the NI device to computer memory. The INCA driver then uses the endpoint identifiers to demultiplex incoming messages to the recipient application program 1112. The network message buffers in OS address space are mapped to the recipient application's address space 1114. The INCA IPP software is configured and started for protocol processing 1116. The IPP software performs protocol processing to extract the data from the network message(s) 1118. Once the first IPP loop is completed, the application is notified via the INCA API calls that data is ready for consumption 1120. The application then processes the data 1122. If there are more messages to process, the IPP loop continues processing and the application continues consuming the data 1124. When all messages have been received 1126, the NI driver closes the NI device and relinquishes control of the device to the OS 1128. For transmission of data, the entire process occurs in reverse order and the application uses the API calls to communicate with the IPP software to determine which protocols and protocol options to use, sets up an endpoint by opening the INCA NI driver with the open() API call, establishes an endpoint, sets the endpoint and driver DMA characteristics with INCA API system calls such as ioctl(), and upon transmission completion, uses close() to close the INCA NI driver. The IPP component executes the selected protocols and places the resulting network communicated data into the send queue message buffers. The INCA NI driver ceases control of the NI device and DMA resources with its "system calls" to the OS, maps the send queue in application address space to the OS message buffers in OS address space using the function mmapO, sets up and controls the DMA transfer from the OS message buffers to the NI device, and upon completion, relinquishes control of the NI device and DMA resources. The API calls are the method of communication between the three INCA components and the existing application programs, OS and computer resources such as the NI and DMA devices. RESULTS Tests were conducted on commercially available systems, configured with commercial- off-the shelf (COTS) software, NIs and a FastEthemet network. The INCA testbed consisted of two machines connected via a 100 Mbps FastEthemet. INCA allows applications to process data at rates greater than 10 Mbps, thereby a normal 10 Mbps Ethernet would have caused the network to limit INCA performance. A SUN Microsystems UltraSPARC 1 WS with a 143 MHz UltraSPARCl CPU, 64 MB of RAM, running Solaris 2.5.1 (also known as SUN OS 5.5.1), with a SUN SBus FastEthemet Adapter 2.0 NI was connected via a private (no other machines on the network) 100 Mbps FastEthemet to a Gateway 2000 PC with a 167 MHz Pentium Pro CPU, 32 MB of RAM, running the Linux 2.0 OS with a 3Com FastEtherlink (Parallel Tasking PCI 10/100 Base-T) FastEthemet NI. Messages of varying lengths from 10 bytes to the maximum allowable UDP size of 65K bytes were sent back and forth across the network between the machines using an Internet World Wide Web (WWW) browser as the application program on both machines. This architecture uses the actual application programs, machines, OSs, NIs, message types and networks found in many computing environments. The results should therefore have wide applicability. SUN UltraSPARCl Workstation with and without INCA Referring to Figure 8, the graph illustrates the fact that on a high performance WS class computer, INCA outperforms the current system at application program network message throughput by 260% to 760% depending upon the message size. Since 99% of TCP and 89% of UDP messages are below 200 bytes in size, the region of particular interest is between 20 and 200 byte size messages. Gateway 2000 Pentium Pro PC with and without INCA Referring to Figures 9 and 10, the graphs illustrate that on a PC class computer, INCA outperforms the current system at application program network message throughput by 260% to 590%. Figure 9 shows INCA's 260% to 275% performance improvement for message sizes of 10 to 200 bytes. Figure 10 shows that as message sizes get larger and larger, up to the existing protocol limit of 65K bytes, INCA's performance improvement becomes larger and larger reaching the maximum of 590% at a message size of 65K bytes. Although the method of the present invention has been described in detail for purpose of illustration, it is understood that such detail is solely for that purpose, and variations can be made therein by those skilled in the art without departing from the spirit and scope of the invention. The method of the present invention is defined by the following claims:

Claims: We claim: 1. A method for improving the internal computer throughput rate of network communicated data comprising transferring network communicated data from a network interface device to an application address space with only one physical copying of the data. 2. The method for improving the internal computer throughput rate of network communicated data of claim 1, wherein the copying of the data occurs in response to a call by an application program interface which bypasses the operating system calls. 3. The method for improving the internal computer throughput rate of network communicated data of claim 2, where the call functions of the application program interface are integrated into the existing operating system. 4. The method for improving the internal computer throughput rate of network communicated data of claims 1 or 2, further comprising the transfer of network communicated data from the network interface device directly to the application address space. 5. The method for improving the internal computer throughput rate of network communicated data of claim 4, further comprising the transfer of network communicated data from the network interface device to application address space through an address mapping of the network communicated data between the operating system address space and the application address space. 6. The method for improving the internal computer throughput rate of network communicated data of claim 5 , wherein the transfer of network message data from the network interface device to the application address space is controlled by the operating system. 7. The method for improving the internal computer throughput rate of network communicated data of claim 5, wherein the transfer of network message data from the network interface device to the application address space is controlled by the application program. 8. The method for improving the internal computer throughput rate of network communicated data of claim 5 , wherein the transfer of network message data from the network interface device to the application address space is controlled by a hardware component. 9. The method for improving the internal computer throughput rate of network communicated data of claim 5 , wherein the transfer of network message data from the network interface device to the application address space is controlled by the network interface device. 10. The method for improving the internal computer throughput rate of network communicated data of claim 5, wherein the transfer of network message data from the network interface device to the application address space is a direct memory access transfer. 11. The method for improving the internal computer throughput rate of network communicated data of claim 10, further comprising reinitializing a direct memory access if an error occurs. 12. The method for improving the internal computer throughput rate of network communicated data of claim 10, further comprising repeating a direct memory access transfer if an error occurs. 13. The method for improving the internal computer throughput rate of network communicated data of claims 5, wherein the transfer of the network communicated data is a programmed input/output transfer. 14. The method for improving the internal computer throughput rate of network communicated data of claim 5, wherein the operating system of the computer manages the address mapping between the virtual memory addresses and physical memory addresses of the network communicated data in the operating system and application memory address spaces. 15. The method for improving the internal computer throughput rate of network communicated data of claim 14, further comprising the network interface driver translating the address mapping between the virtual memory addresses and physical memory addresses of the network communicated data in the operating system and application memory address spaces. 16. The method for improving the internal computer throughput rate of network communicated data of claim 14, further comprising the network interface driver demultiplexing network messages and routing the network messages to the proper application. 17. The method for improving the internal computer throughput rate of network communicated data of claim 16, further comprising the network interface driver examining the header of the message to determine the correct destination point of the message. 18. The method for improving the internal computer throughput rate of network communicated data of claim 16, further comprising the network interface driver maintaining a list of the application endpoints. 19. The method for improving the internal computer throughput rate of network communicated data of claim 14, further comprising the network interface driver providing security by permitting only an intended recipient of the network communicated data to access the network communicated data. 20. The method for improving the internal computer throughput rate of network communicated data of claim 14, further comprising the network interface driver notifying and providing parameters to an integrated protocol processing loop to allow an integrated protocol processing loop to perform protocol processing on the network communicated data. 21. The method for improving the internal computer throughput rate of network communicated data of claim 20, wherein the network interface driver sets end memory addresses of the message buffers, aligns the message buffers and passes the range of the message buffers to the integrated protocol processing loop. 22. A method for improving the internal computer throughput rate of network communicated data comprising executing communication protocols in an integrated protocol processing loop. 23. The method for improving the internal computer throughput rate of network communicated data of claim 22, further comprising linking the proper protocols to an application in the application program memory address space. 24. The method for improving the internal computer throughput rate of network communicated data of claim 22, further comprising the integrated protocol processing loop containing iterations of serial and integrated data manipulations. 25. The method for improving the internal computer throughput rate of network communicated data of claim 22, wherein header processing is performed during serial data manipulation, data manipulation is performed during integrated data manipulation and header and external behavior is performed during serial data manipulation. 26. A method for improving the internal computer throughput rate of network communicated data comprising calculating communication protocol checksums one computer word at a time within an integrated protocol processing loop. 27. The method for improving the internal computer throughput rate of network communicated data of claim 26, wherein the size of a computer word is 32 bits. 28. The method for improving the internal computer throughput rate of network communicated data of claim 26, wherein the size of a computer word is 64 bits. 29. A method for improving the internal computer throughput rate of network communicated data comprising: transferring network communicated data from a network interface device to an application address space with only one physical copying of the data; executing communication protocols in an integrated protocol processing loop; calculating communication protocol checksums one computer word size of data at a time within the integrated protocol processing loop; and address mapping of the data occurs in response to call functions, where the operating system's calls are bypassed. 30. The method for improving the internal computer throughput rate of network communicated data of claim 29, wherein the call functions are call functions of an application program interface. 31. The method for improving the internal computer throughput rate of network communicated data of claim 29, wherein the call functions are call functions of an application program. 32. The method for improving the internal computer throughput rate of network communicated data of claim 29, wherein the call functions are call functions of a network interface device. 33. The method for improving the internal computer throughput rate of network communicated data of claim 29, wherein the call functions are call functions of a network interface driver.

Inventors: SCHUG KLAUS H, SCHUG, KLAUS, H.
Applicants: MCMZ TECHNOLOGY INNOVATIONS LL, MCMZ TECHNOLOGY INNOVATIONS LLC
Application year: 1998
Main classification: H04L12
IPC codes: H04L12