Simulating a large number of users

Simulating a large number of users is described. A method may include receiving a test script including a plurality of commands and invoking a script interpreter. An application thread may be launched to execute the test script. A protocol engine may be invoked for each of the commands in the test script such that each protocol engine has an associated command. Each protocol engine may execute its associated command. A system on which the method may be executed may include one or more chassis or computing devices having one or more network cards. The chassis and/or computing devices may be connected to one or more networks.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to networks and network testing.

2. Description of Related Art

Networks such as the Internet provide a variety of data communicated using a variety of network devices including servers, routers, hubs, switches, and other devices. Before placing a network into use, the network, including the network devices included therein, may be tested to ensure successful operation. Network devices may be tested, for example, to ensure that they function as intended, comply with supported protocols, and can withstand anticipated traffic demands.

To assist with the construction, installation and maintenance of networks and network devices, networks may be augmented with network analyzing devices, network conformance systems, network monitoring devices, and network traffic generators, all of which are referred to herein as network testing systems. The network testing systems may allow for the sending of network communications.

DETAILED DESCRIPTION OF THE INVENTION

Throughout this description, the embodiments and examples shown should be considered as exemplars, rather than limitations on the elements claimed below.

Environment

Referring toFIG. 1, there is shown a block diagram of an environment in accordance with the invention. The environment includes network testing system110coupled to a network140. The network testing system110may include or be one or more of a network traffic generator, a performance analyzer, a conformance validation system, a network analyzer, a network management system, and/or others.

The network testing system110may be in the form of a chassis or card rack, as shown inFIG. 1, or may be an integrated unit. Alternatively, the network testing system may comprise a number of separate units such as two or more chassis cooperating to provide network analysis, network traffic analysis, network conformance testing, and other tasks. The chassis of the network testing system110may include one or more network cards114and a back plane112. The chassis of the network testing system110and/or one or more of the network cards114may be coupled to the network140via one or more connections120. The network cards114may be permanently installed in the network testing system110, may be removable, or may be a combination thereof.

The network testing system110and the network cards114may support one or more well known higher level communications standards or protocols such as, for example, the User Datagram Protocol (UDP), Transmission Control Protocol (TCP), Internet Protocol (IP), Internet Control Message Protocol (ICMP), Hypertext Transfer Protocol (HTTP), address resolution protocol (ARP), reverse address resolution protocol (RARP), file transfer protocol (FTP), Simple Mail Transfer Protocol (SMTP); may support one or more well known lower level communications standards or protocols such as, for example, the 10 Gigabit Ethernet standard, the Fibre Channel standards, and one or more varieties of the IEEE 802 Ethernet standards, Asynchronous Transfer Mode (ATM), X.25, Integrated Services Digital Network (ISDN), token ring, frame relay, Point to Point Protocol (PPP), Fiber Distributed Data Interface (FDDI); may support proprietary protocols; and may support other protocols. Each network card114may support a single communications protocol, may support a number of related protocols, or may support a number of unrelated protocols.

The term “network card” encompasses line cards, test cards, analysis cards, network line cards, load modules, interface cards, network interface cards, data interface cards, packet engine cards, service cards, smart cards, switch cards, relay access cards, CPU cards, port cards, and others. The network cards may be referred to as blades. The network cards114may include one or more computer processors, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), programmable logic devices (PLD), programmable logic arrays (PLA), processors, other kinds of devices, and combinations of these. The network cards114may include memory such as, for example, random access memory (RAM). In addition, the network cards114may include software and/or firmware. One or more of the network cards114may have a resident operating system included thereon, such as, for example, a version of the Linux operating system.

At least one network card114in the network testing system110may include a circuit, chip or chip set, such as network chip118, that allows for communication over a network as one or more network capable devices. A network capable device is any device that may communicate over the network140.

The connections120may be wire lines, optical fiber cables, wireless, others, and combinations of these. Although only one connection120is shown, multiple connections with the network140may exist from the network testing system110and the network cards114to the network140.

The back plane112may serve as a bus or communications medium for the network cards114. The back plane112may also provide power to the network cards114.

The network testing system110, as well as one or more of the network cards114, may include software that executes to achieve the techniques described herein. As used herein, “software” refers to instructions that may be executed by a computer processor. The software may be implemented in a computer language, and may be object code, may be assembly or machine code, a combination of these, and others. The term “application” refers to one or more software modules, software routines or software programs and combinations thereof. The techniques described herein may be implemented as software in the form of one or more applications, plug-ins, lower level drivers, object code, and/or other software.

The software may be stored on and executed from any local or remote machine readable medium such as, for example, without limitation, magnetic media (e.g., hard disks, tape, floppy disks), optical media (e.g., CD, DVD), flash memory products (e.g., memory stick, compact flash and others), and volatile and non-volatile silicon memory products (e.g., random access memory (RAM), programmable read-only memory (PROM), electronically erasable programmable read-only memory (EEPROM), and others). A storage device is a device that allows for the reading from and/or writing to a machine readable medium. Examples of storage devices include CD players, DVD players, flash memory card readers, and others.

The network testing system110may include a CPU card that allows the chassis to also serve as a computer workstation. The network testing system110may have coupled therewith a display and user input devices such as a keyboard, mouse, pen and trackball, and others. A hard disk drive or other storage device may be included in network testing system110to store software that implements the techniques described herein.

The network testing system110may be located physically adjacent to or remote to the devices130coupled with the network140.

The network140may be a local area network (LAN), a wide area network (WAN), a storage area network (SAN), or a combination of these. The network140may be wired, wireless, or a combination of these. The network140may include or be the Internet. The network140may be public or private, may be a segregated test network, and may be a combination of these.

Communications on the network140may take various forms, including frames, cells, datagrams, packets or other units of information, all of which are referred to herein as data units. Those data units that are communicated over a network are referred to herein as network traffic. The network140may be comprised of numerous nodes providing numerous physical and logical paths for data units to travel. There may be plural logical communications links between the network testing system110and a given network capable device130. Examples of logical communications links include, without limitations, channels, pipes, streams, and others.

The network140may be a test network, a production network, other network, or a combination of these. The term “production network” as used herein means a network that is up and running in the regular course of business. As such, a production network includes network traffic from and between end users and other client devices and servers such as web servers and application servers, as well as other network capable devices attached to or otherwise communicating over the production network. The term “test network” means any network that is to be tested, including private segregated networks and publicly accessible networks. The network testing system110may send or otherwise transmit or communicate data units directed to network capable devices such as devices130over the network140.

The network capable devices130may be devices capable of communicating over the network140. The network capable devices130may be computing devices such as workstations, personal computers, servers, portable computers, telephones, personal digital assistants (PDAs), computing tablets, and the like; peripheral devices such as printers, scanners, facsimile machines and the like; network capable storage devices including disk drives such as network attached storage (NAS) and SAN devices; and networking devices such as routers, relays, firewalls, hubs, switches, bridges, traffic accelerators, and multiplexers. In addition, the network capable devices130may include appliances such as refrigerators, washing machines, and the like as well as residential or commercial HVAC systems, alarm systems, set-top boxes, personal video recorders, and other devices or systems capable of communicating over a network. One or more of the network capable devices130may be a device to be tested and may be referred to as a device under test.

The network testing system110may be or include one or more computing devices, particularly network capable workstations and personal computers. The computing devices may be used in place of or to augment a chassis.

Systems

FIG. 2is a first functional block diagram of operating units in accordance with the invention. A network testing system may provide a user interface that allows a user of the network testing system to create test scripts. Test scripts may also be provided by automated testing systems. That is, the test scripts may be computer or system generated. The test scripts may be automatically generated based on analysis of production network traffic or traffic directed to a particular network device, such as a server. A test script may include a sequence of commands or instructions that cause network traffic to be created and transmitted by the network testing system. Each test script may replicate the network traffic generated by a single network user or a group of network users. Each test script may represent or be a simulated virtual user or a group of simulated virtual users. Each test script may be used to stress test a device under test and/or a portion of a network and/or a network. Such a portion of a network or network may include one or more devices under test.

For each test script received, an instance of a script interpreter210is executed.

In one testing scenario, when a script represents the activities of a single user, multiple instances of a single script may be executed to represent multiple users. In traditional network testing systems, an operating system thread may be executed for each script. A thread is a relatively large or heavyweight feature of an operating system that allows multiple operating system tasks to be executed concurrently. The use of threads allows for parallel execution of multiple concurrent tasks. When using a large number of threads, communications between the operating system and the threads are required. The communications between the threads and the operating system that are required to maintain, monitor and execute a large number of threads expend system resources in the form of processor cycles and memory. In this way, the communications between the threads and the operating system required to execute a large number of threads reduces network testing system performance.

Although threads are in some ways well suited to running multiple test scripts or multiple instances of a small number of test scripts concurrently, when used in large quantities, threads may make excessive use of network testing system resources such as memory (actual or virtual), and/or processing power. In some instances, when using a large number of threads to execute test scripts, the amount of addressable memory may be used up before the number of scripts needed to fill a large bandwidth communication line are executed.

In some traditional systems, the overhead (that is, memory requirements) of executing each script as a thread may limit the number of scripts, and thus the number of virtual simulated users. However, to fill up a large bandwidth communication medium, such as, for example, a 1 Mbps, 100 Mbps, or 1 Gbps line, and/or to stress test a device under test with a large number of data units, 10,000 to 30,000 or more test scripts may be required. To test a device, network portion or network to determine how it will behave under heavy network traffic usage and/or heavy network traffic loads, the concurrent execution of many more scripts may be required than is possible in traditional network testing systems. Using the techniques described herein, many more test scripts than have been traditionally executable may be executed by a network testing system to fully stress a device under test and/or to maximize usage of the available bandwidth of a communication channel for testing purposes.

Rather than use traditional operating system threads, application threads may be used. Application threads are a lighter weight construct that require a smaller amount of network testing system resources to execute. That is, application threads require a smaller amount of memory and processor power to execute when compared with traditional threads. In addition, lighter weight threads require less communication between the thread and the operating system. As such, many more application threads may be executed in parallel or concurrently than traditional threads. Application threads may be implemented in many computer languages using various kinds of computer programming constructs. In one embodiment, the application threads are implemented using co-routines using the “C” programming language.

Application threads may invoke extended operations supported by the operating system. The extended operations may be included as part of the operating system kernel, may be plug-ins to the operating system, may be DLL files, may be other higher and lower level software extensions included in or accessible by an operating system. The extended operations may provide specific pre-programmed functionality with lower overhead when executed by application threads rather than traditional threads. The extended operations may allow for relatively simple actions such as opening connections, sending bytes, and closing a connection. In the network testing context, the functionality provided by the extended operations may include verification, fetch, fetch and ignore, monitor, count, and others. More specifically, a “fetch and verify” extended operation may allow for fetching data units via FTP, HTTP or other protocol and verifying the contents of the received data units. Such a command eliminates the transfer of the contents of the data unit from the operating system to an application program for verification, because the data unit has already been verified. Another extended operation may be “fetch and ignore.” This extended operation may allow for receipt of data units when the script or requesting application has no interest in the payload or content of the data units. This reduces system overhead as the operating system will not expend resources either verifying the contents of the data unit or sending the contents of the data unit to a requesting script or application.

For each test script, the script interpreter210invokes an application thread,220. For each instruction in the command lines in the test scripts, the application thread launches a protocol engine230. The protocol engines230prepare the appropriate call to an operating system250to achieve the instruction. The operating system250may be a version of the Linux, Unix, Microsoft Windows, Apple and other operating systems. The operating system may include extended operations such that the command lines in the scripts may be extended operations.

So that the protocol engines230do not have to wait for or block on the operating system250, an input/output (I/O) multiplexor240may be inserted between the protocol engines230and the operating system250.

The I/O multiplexor240may receive calls to the operating system250from the protocol engines230and direct the calls to the operating system250when the operating system250is available. The I/O multiplexor240may also receive any responses from the call placed with the operating system250from the operating system250. The I/O multiplexor240may pass any response to a call to the appropriate protocol engine230.

In one embodiment, the operating system exists in the operating system space260, and the script interpreters210, application threads220and the protocol engines exist in the user or application space204. The I/O multiplexor has an interface directly with the protocol engines230and with the operating system250. As such, the multiplexor240exists in both the user or application space204as well as in the operating system space260.

FIG. 3is a second functional block diagram of operating units in accordance with the invention. Script interpreters310receive and execute test scripts. For each test script, the script interpreter invokes an application thread320. The application threads320execute a protocol engine330for each command in the script. The operating system may include extended operations such that the commands in the scripts may be extended operations. To increase the speed of processing the commands in the test scripts, in this embodiment, the protocol engines330are moved into the operating system350. In this embodiment, the script interpreters310and the application threads320are in user or application space304, and the protocol engines330and the multiplexor are in the operating system space360.

FIG. 4is a third functional block diagram of operating units in accordance with the invention. As above, the script interpreters410receive and execute test scripts. For each test script, the script interpreter410invokes an application thread420. To increase the speed of executing the commands in the test scripts, in this embodiment, the application threads420and the protocol engines430are moved into the operating system space360. The application threads420execute a protocol engine430for each command in the script. The operating system may include extended operations such that the commands in the scripts may be extended operations. In this embodiment, the script interpreters410are in the user or application space404, and the application threads420, the protocol engines430and the multiplexor are in the operating system space460.

With regard to all of the network testing systems descried herein, additional and fewer units, blocks, communication lines, modules or other arrangement of software, hardware, firmware and data structures may be used to achieve the system and techniques described herein.

Methods

FIG. 5is a flow chart of a method in accordance with the invention. A test script is received, as shown in block510. A script interpreter is invoked for each test script received, as shown in block512. For each script interpreter, an application thread is launched to execute the script, as shown in block514. For each command in the script, the application thread invokes a protocol engine, as shown in block516.

Although the protocol engine has been invoked, before the protocol engine executes the command that was provided it, a check is made to determine whether a maximum number of protocol engines has been exceeded, as shown in block520. Alternatively, the check for whether the maximum number of protocol engines has been exceeded may be made before the protocol engine is invoked.

The maximum number of protocol engines may vary depending on the protocol. As such, the maximum number of protocol engines may be based on the particular command's protocol. For example, the number of HTTP protocol engines may be 4,000, while the maximum number of FTP protocol engines may be 1,000. The maximum number of protocol engines may be a system defined constant. The maximum number of protocol engines may vary and may be dependent on available system resources such as one or more of actual and virtual memory availability, memory address space usage, and other factors. In addition, a limit on the number of active protocol engines per simulated virtual user may be imposed. Such a limit may be arbitrary or may be derived from one or more of the size of available actual memory, available virtual memory, memory address space usage, and other factors.

If the maximum number of protocol engines is not exceeded, as shown in block520, the protocol engine executes the command, as shown in block530. The execution of the command causes data units to be sent onto a network. A response to the command, if any, may be received, as shown in block532. A received response may be passed by the protocol engine to the application thread and then to the script interpreter, as shown in block534.

A check is then made to determine whether there are more commands in the script, as shown in block536. If there are more commands, the same or another protocol engine may be invoked to execute the next command, as shown in block538. The flow of execution continues at block520, with the check for whether the maximum number of protocol engines has been exceeded. If there are no further commands in the script, the execution of the script ends, as shown in block550.

If the maximum number of protocol engines is exceeded, as shown in block520, the application thread may sleep for a network testing system defined period of time, as shown in block540. An attempt may then be made to retry executing the script command protocol engine, as shown in block542. The flow of actions then continues at block520, where the check for whether the maximum number of protocol engines has been exceeded is made again. In another embodiment, the action taken at block540is replaced with the application thread sleeping until a protocol engine becomes available. The availability of a protocol engine may be determined based on available network testing system resources.

Additional and fewer steps may be taken, and the steps may be combined or further refined to achieve the methods described herein.

Although exemplary embodiments of the invention have been shown and described, it will be apparent to those having ordinary skill in the art that a number of changes, modifications, or alterations to the invention as described herein may be made, none of which depart from the spirit of the invention. All such changes, modifications and alterations should therefore be seen as within the scope of the invention.