Considering Local Bus Traffic in Network Performance Simulations

In a large distributed computing environment based on local area networks, such as Ethernet or FDDI ring, system performance may be significantly degraded by a bottleneck at some segment of a particular network. In this study we examine the effect that local bus traffic has on the overall performance of a network. In particular, we studied server access for networks using two interconnect technologies, VMEbus and RACEway. The results show that local bus traffic on the CSMA/CD or FDDI LAN backbones has a significant impact on overall performance and may decrease access delays by an order of magnitude, or more, depending on traffic load. Figure 1. Generic Configuration of a LAN. INTRODUCTION The proliferation of high-speed local area network (LAN) technologies has left users with a bewildering choice of networks and protocols. In a large-scale distributed computing environment forming a grid based on Ethernet or FDDI ring, system performance may be degraded by a network bottleneck. Figure 1 shows a generic configuration of a LAN with local nodes based on interconnects such as VMEbus or RACEway. Here we have a number of local hosts, workstations, or simply multiple processors housed in a single enclosure, connected via a bus, which use Ethernet to communicate with hosts located on other buses. Hence in addition to regular Ethernet traffic we also have bus traffic that has to be modeled and combined with the LAN traffic. We use the term workstation as equivalent to any local host or processor connected to its peers via a bus or other interconnect. One of the most demanding applications that current LANs support is server access. With the increase in processor speeds and the number of workstations attached to networks, network utilizations have been increasing with detrimental effects on the overall performance. It is difficult for users to determine in advance whether a given network technology will deliver the expected performance benefits, or to determine how to tune their existing systems to take full advantage of a high-speed network. The purpose of this work is to study the migration of a multiple segment LAN, such as CSMA/CD or FDDI, to an extended LAN by introducing bus traffic on a VMEbus or RACEway as a local interconnect. We evaluate the performance of these network configurations under varying load and different topologies and compare the results to the performance of original CSMA/CD network. The main goal is to see how the local bus traffic affects the overall performance of the local area network (LAN). For this purpose, we estimate the server access time in network and bus models built with the SES/workbench simulator [1]. Complex networks involve individual nodes built out of multiple processors connected via some type of a bus or other interconnect, and with the nodes connected by some type of local network such as Ethernet. The processors within a node may or may not have local memory configurations and there may also be memory at the global level. Concentrating on performance of such networks brings us to the analysis of the most common LANs with local node traffic. MODELING BUS SYSTEMS Most of the modern computer systems share a common bus, or other type of interconnect, to exchange information. There is a variety of popular bus standards: ranging from VMEbus to Futurebus to PCI, and modern interconnects, from Myrinet, to SKYchannel and RACEway, to InfiniBand, to name a few [2]. For most computer systems, the critical issue is the temporal determinism of system response. The timing of computations in the system depends not only on the instruction execution time and memory access, but also on the timing of data exchange between various system components. The latter, in addition to the hardware characteristics, is a function of the specific bus architecture and the bus access and arbitration protocols. In most cases, the system has a dedicated bus controller/arbiter – a device designated to manage the bus operation and execute specific bus protocol. Modern interconnects often use distributed arbitration, without a central arbiter node. Queuing models are very adequate for this single-bus tightly coupled multiprocessor architecture [3]. Each processing element is modeled as a finite set of tasks (the task pool), a CPU, a bus interface unit that allows the processing element to access the shared bus, and a set of queues associated with the CPU and the bus interface unit. Each CPU and bus interface unit has a mean service rate and the tasks are assumed to have a mean sleep time in the task pool. Another assumption is that the CPU and the bus interface unit operate independently of each other, and that all the bus interface units in the multiprocessor can be lumped together into a single equivalent Bus Interface Unit. A task is assumed initially asleep in the task pool. When it awakens, it will queue for CPU usage in the ready list, which is a list of tasks that are eligible to use the CPU as a result of either being awakened or as a result of having finished using some resource. Interrupt driven jobs are not placed on the ready list; instead they have their own higher priority queue. After using the CPU, the task may request more CPU service (in which case it will re-queue in the ready list), it may go back to sleep in the task pool, or it may request bus usage. More advanced bus architectures can be modeled as a version of a queuing system with customers, servers, and resources. There are several studies in literature modeling the bus systems in such a way [4]. The models, mostly, focus either on distributed systems and describe networking applications or discuss multiple bus architectures and some real-time issues. When building the bus architecture model, we focus on the bus performance. The three basic metrics, identifying the performance of any bus system, are: • system throughput – how much data can be transmitted per time unit, • bus utilization – what percentage of time the bus is busy, • bus access latency – the time needed by the bus to grant ownership to the requesting agent (a part of bus response time). A high average system throughput is critical for most systems, for example, in various data acquisition/fusion applications. Related high utilization of the bus indicates that the margin for the system upgrade is limited. For realtime systems and distributed computing applications, more critical is the determination of system response, that is, bus access time, used as a performance measure in our study [5]. The above-mentioned criteria of bus performance are affected by the following factors: • number of agents (processors) attached to the bus, • latency of the memory, • bus access protocol and the arbitration method used, • physical characteristics of the bus (multiplexed lines, transmission speed, etc.), • workload characteristics of the processor operation. Selection of the workload, including the frequency of requests and the size of packets sent over the bus (often related to the local processor cache hit/miss ratio) is critical to the performance analysis study. The workload depends on the application and must be clearly identified before simulation model is developed. VMEbus Model If one wants to enhance functionality and performance of the computer networks, by changing the configuration and introducing bus traffic to make them suitable for distributed computing applications, the VMEbus system [6] is a prime candidate because it is both rigorously defined and widely supported. The VMEbus provides support for multiprocessing using shared memory. Modules can be designed to act as masters, slaves or both. Before a master can transfer data it must first acquire the bus using a central arbiter (Fig. 2). A master module can initiate data transfer cycles. A slave module detects bus cycles generated by a master module and participates in the cycle, if it is selected to do so. If certain addresses are selected by the master module, an on-board signal is sent by the local monitor and a message broadcast to all modules. To prevent a system crash, or other problem with data transfer, from locking up the system, a bus timer times all transfers and kills transfers which take longer than some specified time interval. Four of the most important cycles on the VMEbus are: read/write, block transfer, read-modify-write, and address-only. To avoid inconsistency while updating shared memory, read-modify-write bus cycles are used. The read-modifywrite cycle allows updating shared memory as an atomic transaction and prevents race conditions. Arbitration bus Bus In terface