Abstract:
Systems and methods are provided for testing the load state of at least one device or communication network in the loaded state. The system includes at least one programmable control device and at least one session computer connected to, or integrated with, the control device. The control device stores a plurality of session scripts which may be pre-compiled and individually interactive. The session scripts may contain an initialization procedure, a predefined test procedure, and a termination procedure. The session computer includes: a plurality of mutually independent logical or physical connection interfaces; and a plurality of script-processing devices being able to simultaneously establish mutually independent connections to a device to be tested. Each script-processing device executes at least one session script received from the control device through each of the mutually independent connection interfaces to simulate a load of a plurality of users on the device to be tested.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     This application is a continuation-in-part of U.S. application Ser. No. 10/049,867, filed Jul. 2, 2002, which is a national phase application filed under 35 U.S.C. § 371 of International Application No. PCT/EP00/06509, filed Jul. 10, 2000, which claims the benefit of DE 199 37 753.7, filed Aug. 10, 1999, all of which being incorporated herein by reference in their entireties. 
     
    
     BACKGROUND OF THE INVENTION  
       [0002]     1. The Field of the Invention  
         [0003]     The present invention relates to a system and method for testing at least one device in a communications network by one or more users.  
         [0004]     2. The Relevant Technology  
         [0005]     Developers, system providers and administrators of large heterogeneous network configurations based on the IP standard, such as the internet, appear to be facing mounting challenges due to the rapid growth of the Internet and the speedy advancement in transmission and hardware technology. Therefore, testing new network or network expansions prior to actual installation for their load state may be considered an important task. By a load test one may mean that the targeted loading of the network, such as of the routers and servers connected to such a network, in order to determine their performance with respect to the required data throughput and the response time to a user request. There is, therefore, a need for a test system that is able to test network components based on the IP standard, under real load conditions, to be able to ensure that all network components are functioning properly in an error-free manner, within their predefined performance limits.  
         [0006]     The reference U.S. Pat. No. 5,669,000 purportedly concerns a system for remotely testing a computer system, where instructions can be sent from a host computer  100  to so-called target machines. The target machines, in turn, perform actions that a user would otherwise undertake.  
         [0007]     The reference European Patent No. 0 883 271 purportedly concerns a method as well as a system for managing data-service systems. The test method is suited for generating test traffic or test signals to simulate a data transmission when subscribers access services. The test system includes test devices which are set up as decentralized devices, which, via a plurality of so-called measuring routes, may allow a determination of the behavior of mutually influencing modules or the behavior of one module in the entire system.  
       BRIEF SUMMARY OF THE INVENTION  
       [0008]     Exemplary embodiments and/or exemplary methods of the present invention are directed to providing a test system and a test method which will enable the load of a device to be automatically tested by a plurality of network users, it being possible for the test system to be centrally operated by one single operator.  
         [0009]     Further exemplary embodiments and/or exemplary methods of the present invention are directed to providing a semi-automated test system which may be able to establish a plurality of mutually independent connections to a communications network, such as for example a PPP connection carrying IP packets/connections based on the IP standard, in order to run, via these connections, mutually independent test procedures, each corresponding to the operations of a real network user. The test procedures may contain individually interactive procedures that dynamically respond to responses from a device under test. Such a test system may be designed for testing at least one device in the loaded state in a communications network based on the IP standard.  
         [0010]     Further exemplary embodiments and/or exemplary methods of the present invention are directed to providing a test system which includes at least one programmable control device having an assigned memory device in which a plurality of session scripts may be stored, each of which may contain a predefined test procedure or define an individually interactive session. Thus, a session script may be understood to include a scripted simulation in programmed form of a real network user, who for example may performs actions based on the IP standard, or a script to enable an individually interactive session. Additionally, the session scripts may be pre-compiled.  
         [0011]     The IP standard may include establishing a connection to a provider, downloading files to a server, using a web browser, and initiating a connection. A session script may contain, for example, a user identification (user ID), a user password, an IP destination address, for example, of a server connected to the communications network, the user ID and the password of such a server and the service and communications protocol utilized, such as the FTP (file transfer protocol) or the HTTP (hypertext transfer protocol). Other communications protocols that can be utilized include: Telnet, HTTPS, SMTP, POP3, IMAP, RSTP, RTCP, SIP, and the like. Each session script may contain a predefined number of operations that a real user could enter into a personal computer in order to request a specific IP service via the communications network. Each session script may also dynamically respond to responses from the device under test to simulate additional user operation or selected load conditions.  
         [0012]     Further exemplary embodiments and/or exemplary methods of the present invention may involve at least one session computer connected to the control device. Each session computer has a plurality of mutually independent connection interfaces, which may be logical or physical, via which a PPP connection carrying IP packets/connections, or an independent IP connection, to the communications network may be established at any one time. Assigned to each connection interface, in turn, may be a script-processing device, also called a load-generating device in the following, which, in dependence upon a session script assigned by the control device, may establish an IP connection to a device to be tested and start the predefined test procedure. This can allow a running of a plurality of mutually independent test sessions in automated fashion between various simulated users and allow one or more devices to connect to the communications network, such as a router or a server, without an operator having to manually carry out a session at the session computer.  
         [0013]     In exemplary embodiments and/or methods of the present invention, the complexity of the test system may be enhanced by applying the same or different session scripts to a plurality of load-generating devices of a session computer, which, in dependence upon the session script assigned in each instance by the control device, may then be able to establish a separate IP connection to one or a plurality of the devices to be tested and initiate the corresponding test procedure. To this end, implemented in each session computer may be a session-management device which supplies the session script assigned by the control device to each load-generating device.  
         [0014]     In further exemplary embodiments and/or exemplary methods of the present invention, the session computers may be designed to support every existing network-access technology. They can be readily adapted to future network-access technologies. For example, every connection interface of a session computer is connected to an analog and/or digital modem. In further exemplary embodiments and/or exemplary methods of the present invention, one or more interface cards, for example, LAN (local area network) cards, may be inserted into the session computers, which each have a plurality of connection interfaces. Further, each connection interface of a session computer may be assigned to an analog or digital modem or be linked to an available data concentrator to interface to an ATM (asynchronous transfer mode) network. As digital modems, ISDN modems or ADSL (asymmetric digital subscriber line) modems come under consideration. In this manner, a separate IP connection may be established via each connection interface of a session computer.  
         [0015]     The control device and the session computer connected thereto may either be implemented in one single machine or be connected via a backbone network.  
         [0016]     To be able to log, capture or print out and later analyze the various test sequences, each session computer can have a memory for storing status data on each device to be tested, as well as the results and status messages from each initiated test procedure. The status data of a device to be tested may be considered to include data such as the data throughput from and to the device to be loaded, as well as its response time. The response time of a device may be understood, in this context, to be the time that the device requires to react to a specific request from a user. Other status data can include a variety of information, for example specific protocol parameters or other latencies.  
         [0017]     Exemplary embodiments and/or exemplary methods of the present invention may include that the session computers transfer the stored status data on the tested devices, as well as the results and status messages from each active test procedure, to the control device, which is able to display this data on a display device assigned thereto and analyze the same. In addition, the control device may have a keyboard assigned to it, via which one may enter new session scripts, for example, or intervene in active test procedures in order, for example, to abnormally terminate a test procedure or reset parameters. In this manner, the test system may be adapted to any hardware and software change in the communications network, merely by writing a new session script and storing it in the control device.  
         [0018]     The communications network based on an IP standard is, for example, the Internet or any firm-specific Intranet. As devices to be tested, access routers and servers come into consideration, for example, which belong to various service providers. Servers, which are based on an IP standard, are available and, therefore not discussed in detail.  
         [0019]     These and other objects and features of the present invention will become more fully apparent from the following description and appended claims, or may be learned by the practice of the invention as set forth hereinafter.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0020]     To further clarify the above and other advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:  
         [0021]      FIG. 1  illustrates a test system according to an exemplary embodiment of the present invention.  
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0022]      FIG. 1  shows a test system  10  with whose assistance the operability, for example, of the Internet  90 , in particular of its network components, such as access routers  80 , or of servers  100  of various service providers connected thereto, may be tested in the loaded state. Test system  10  may also be referred to as an IP load-test system, to indicate that the test system, as well as the components to be tested with respect to their load, support IP protocols. Test system  10  includes a control and service computer  20 , to which a plurality of test computers, called “session computers” in the following, are connected, in the present example, via a star coupler  30  and a so-called backbone network  35 . For the sake of clarity, merely three session computers  40 ,  50  and  60  are schematically depicted, session computer  40  being shown in some detail. For that reason, the design of the session computers in terms of circuit technology may be explained with respect to session computer  40 .  
         [0023]     Control and service computer  20  has a keyboard, by way of which an operator may generate any permissible session scripts, for example, which are subsequently stored in a memory (not shown) assigned to control and service computer  20 . In addition, the control and service computer may also be accessed remotely in which case an operator creates scripts or load sessions remotely. In this manner, the number of users utilizing computer  20  may be greater than one.  
         [0024]     By session script, one may understand the description of an automated user, who, in conjunction with a session computer, may automatically execute IP-supported operations and activities, which a real Internet user could also undertake manually at a personal computer. In other words, each session script may contain a defined test procedure, which may be used to simulate a typical behavior of a real Internet user at the session computers. Each session script may include an initialization procedure, a test procedure, as well as an end procedure. In addition, the term “session script” as used herein can denote a pre-compiled test script. In other words, a test script can be compiled into a binary form for better performance, smaller CPU and memory footprint, and other advantages. The pre-compiled test script can be executed on a session computer.  
         [0025]     In this context, the initialization and end procedures may be executed only once in each session, while the test procedure may be carried out repeatedly. In addition, the writer of the session script may include certain error and status messages which are generated over the course of a running test procedure. In principle, any operations and actions at all may be utilized in a session script.  
         [0026]     Moreover, by way of control and service computer  20 , variables may also be set within a session script before the test procedure is started. In this manner, generally formulated session scripts may be easily and quickly adapted to special customer requests. Other parameters, such as the number of repetitions of one test procedure and timing intervals, may likewise be defined in a session script.  
         [0027]     In various embodiments of the invention, sessions can be individually interactive. As used herein, the term “individually interactive” means that scripts (or pre-compiled scripts) interact with the devices under test. In other words, the scripts can be dynamically interactive with the devices under test to express modified behavior based upon replies from the devices under test, response times, data throughput, and other variables.  
         [0028]     Moreover, an operator at the control and service computer  20  may determine at which session computer(s), and via which connection interfaces of the selected session computers, a test procedure should be started, how many test procedures should be started at the same time, how long a test procedure lasts, or how often the same test procedure should be repeated.  
         [0029]     Inserted into each session computer  40 ,  50  and  60  may be, for example, four LAN cards  42 ,  52  and  62 , each having, for example, four separate connection interfaces  44   1 - 44   n ,  52   1 - 52   n , and  62   1 - 62   n , respectively, called connection ports. To interface with Internet  90 , in the present example, each connection interface may be connected to a digital ADSL modem  70 , although such an interface connection is shown for connection interface  441 . Each modem  70  may be linked via a transmission line to an access router  80  or to various routers. Test system  10  may also support any other access technology. Thus, instead of ADSL modems, ISDN routers may also be connected to the connection interfaces of the session computers. The connection interfaces of any one session computer may be linked to an available concentrator, which provides an access to an ATM network. In another case, the connection interfaces may be linked via a serial connection to analog or digital modems, with whose assistance a dial-up connection to any router and, thus, to Internet  90  may be established. As described with reference to session computer  40 , a script-processing device, in the following also named load-generating device  45   1 - 45   n , may be assigned to each connection interface  44   1 - 44   n ,  52   1 - 52   n , and  62   1 - 62   n  of each session computer  40 ,  50 , and  60 , respectively, as explained in greater detail further on. It should also be noted here that the load-generating devices may also be implemented as software modules.  
         [0030]     In addition, or alternatively to, physical connection interfaces  44   1 - 44   n ,  52   1 - 52   n , and  62   1 - 62   n , embodiments of the invention can use logical connection interfaces. This is advantageous because for certain systems, like broadband devices, it is not necessary to have 1000 physical interfaces for 1000 simulated users. Rather, a single broad physical band device, like Gigabit Ethernet or ATM, can be used to transport  1000  individual sessions.  
         [0031]     In addition, in each session computer  40 ,  50  and  60 , a session-management device may be provided, whose task is to supply session scripts assigned by control and service computer  20  to selected load-generating devices. With regard to session computer  40 , session-management device is denoted here by  46 . In addition, in each session computer  40 ,  50 , and  60 , a memory may be provided in which the status data from the devices to be tested, as well as the results and status and error messages from the initiated test procedures, are stored. These status data, status and error messages, and results of the test procedures in question may be transferred (transmitted or copied) from each session computer to the control and service computer  20 , and stored there. In addition, the control and service computer may be designed to analyze the messages and results received from the session computers and to graphically display the same via a monitor.  
         [0032]     At this point, it should be noted that test system  10  may be used to test IP networks with respect to their software and hardware components from various manufacturers. The operability of routers and servers within a network based on an IP standard may be tested using one single, central test system.  
         [0033]     Test system  10  may be used to check the handling capacity of server  1100  connected to Internet  90 . Here, the assumption is initially made that router  80  is functioning in an error-free manner, so that error messages occurring during the test procedure may be clearly attributed to server  100  to be tested.  
         [0034]     An assumption may be made, based on the manufacturer&#39;s specifications, that server  100  is able to handle up to 50 users simultaneously, who, for example, want to download data via the FTP (file transfer) protocol. In this case, the operator at control and service computer  20  may select that session script which makes it possible to automatically establish a connection to server  100  to be tested, and download a dedicated file from a dedicated directory of the server. If, in the present test case, 32 queries to server  100  are simultaneously simulated, at the control and service computer the operator selects connection interfaces via which a test procedure should run in each instance. For this, the addresses of the connection interfaces are either entered via the keyboard of control and service computer  20 , or appropriate icons are clicked on at the monitor. For example, the operator may select all 16 connection interfaces  44   i - 44   n  of session computer  40  and the first eight connection interfaces of the two other session computers  50  and  60 , respectively, via which a test procedure should run to server  100 . Control and service computer  20  subsequently may transfer the session script in question and the addresses of the selected connection interfaces to the particular intended session computers.  
         [0035]     At this point, the session-management device in each session computer may assure that the session script is loaded into all load-generating devices  45   i - 45   n  of session computer  40 , as well as into the first eight load-generating devices of session computers  50  and  60 , respectively. Under the control of the session script, each load-generating device may establish an IP connection via the connection interface assigned to it and ADSL modem  70  connected thereto, for example via the PPPoE protocol to router  80 , which may assign each connection interface its own IP address and a user password. After that, an identification may be carried out between server  100  and the particular intended connection interface via protocol PPP. Following this initialization phase, each selected load-generating device may be prompted by the session script to execute the FTP IP service, which may prompt the server to download the files in question to the selected connection interfaces. The test procedure may be subsequently terminated by each selected or dialed load-generating device, and the connection is released.  
         [0036]     During the individual test procedures, predetermined status and error messages may be logged in the session computers to the selected connection interface messages and, at the same time, may be routed to control and service computer  20 , to be able to monitor the running test procedures there. Each session computer  40 ,  50  and  60  may be able to determine the data throughput, as well as the response time of server  100 . Since the average data throughput from and to server  100 , as well as the server&#39;s response time, are preset by the manufacturer, it may be determined from the calculated data throughput and from the ascertained response time, for each selected connection interface, whether server  100  has executed the 32 test procedures with or without errors. In this manner, each Internet component may be automatically tested by test system  10  with respect to its required performance features, given that appropriate session scripts are loaded into selected load-generating devices of the particular intended session computers, and are executed.  
         [0037]     Since the connection interfaces and the load-generating devices of each session computer assigned thereto may be designed independently of one another, in the present example, users, who become active independently of one another, may be simulated using each session computer  40 ,  50 ,  60 . For all purposes, a single operator at control and service computer  20  may suffice in order to be able to operate a test system having any number of automated users.  
         [0038]     Exemplary embodiments and/or exemplary methods of test system  10  may allow automatic testing of a device to be tested to check the load produced by a plurality of network users. To this end, it may be only necessary for an appropriate session script to be written for each permissible user action, and to be stored in control and service computer  20 . Any test situations at all may be simulated by loading appropriate session scripts onto selected load-generating devices of the particular intended session computers  40 ,  50 , and  60 , which then, independently of one another, establish separate IP connections to the devices to be tested, and execute test procedures thereon.  
         [0039]     In addition to the infrastructure testing scenario depicted in  FIG. 1 , wherein an entire infrastructure consisting of multiple devices is being tested, embodiments of the invention may also be used to a test a single device. For example, an access router under test can be connected to a session computer(s) as well as to devices receiving network traffic connected to the other side of the access router. For example, although not depicted in  FIG. 1 , if router  80  is under test, connections could exist directly from session computer  40  to router  80  to server  100 . Alternatively, if server  100  is tested, server  100  could be connected directly to computer  40 .  
         [0040]     The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.