Simulator tool for testing software in development process

A simulator tool for testing software is provided. The simulator tool includes a simulator to test the software, an interface to promote communication between the simulator and the software, a message including a component utilized by the simulator to promote testing of the software, and a test controller operable to communicate the message to the simulator, such that the message is utilized by the simulator to test the software. A method for testing software and applications is also provided.

Not applicable.

REFERENCE TO A MICROFICHE APPENDIX

Not applicable.

FIELD OF THE INVENTION

The field of the present invention includes computer software. More particularly, embodiments of the present invention are concerned with computer software testing.

BACKGROUND OF THE INVENTION

Computer software testing presents a variety of difficulties in the software development cycle. Software testing may need to test the ability of the software to handle multiple inputs at the same time. For example, it is not sufficient to verify that a web server can handle a browser hit from a single user at a time: web servers hosting a popular web site may receive hundreds of browser hits more or less at the same time. This kind of testing may be referred to as concurrent testing.

Software testing may need to test the ability of the software to handle a high volume of actions. For example, it is not sufficient to verify that a long distance telephone switch can route a single phone call at a time: it may be required to handle 100,000 call originations per hour. Testing which verifies the ability of software to handle a volume of actions may be referred to as stress testing or load testing.

As a software product suite matures it is extended and new features and functionality are built onto or added to the previous software. When testing the new extensions the new functionality must be tested, but it is important to retest the old functionality to assure that the quality of the old product has not regressed. Testing directed to assuring the old functionality has not been degraded may be referred to as regression testing. Regression testing may be simply a matter or rerunning the tests used to test the functionality of previous releases.

Software releases or software products may include multiple pieces of software—software components, software modules, or software applications—which cooperate to achieve the goals for the software release or product. The difference between software components, software modules, and software applications is not distinct and is a question of granularity, a question of how big a chunk of software is the subject of discussion. In the description which follows the term software component or component is usually employed, but bear in mind that it is meant that this term comprehend all three of these terms, unless otherwise noted. Sometimes, to insist on this inclusiveness all three terms are employed.

Independent software developers or computer programmers—or separate teams of developers—may be assigned to develop each distinct component or module or application of the software product. Typically, the earlier in the software development cycle that one can find and fix errors in software, the less expensive these errors are to fix. It may be desirable to test a software component before all the other cooperating software components are completely developed and even before the software component to be tested itself is completely developed.

Sometimes code is built into a software component solely to support the testing of other components while the software component is still being developed. This code may later be completely deleted from the software component when full functionality is developed, or it may be left in place but disabled in some manner. Development of this testing code comprises, in some sense, wasted effort since it consumes development time and does not contribute directly to the finished product. Such test code may be called “throw away” code because it may never be used again after the initial development stage. Additionally, such test code may not support regression testing in future releases of the software product.

Software may be tested by sending inputs to and receiving outputs from the software under test (SUT). Sometimes software testing is accomplished by having developers take actions at their computer terminals to cause inputs to be sent to and to receive the outputs from the SUT, in a manual rather than an automated enactment of the planned software execution. This kind of testing may tie up the resources of several developers. Additionally, this kind of testing may interfere with normal daily operations, thus forcing the developers to work late or unaccustomed hours to perform this kind of testing. This kind of manual software testing may not support stress testing or concurrent testing needs.

Sometimes the testing described above may be conceptualized by the computer program developers themselves and used in their early testing. These testing concepts may pass on to a separate team of software testers who may not be as familiar with the technologies employed by the developers in their early testing. Hence, there may be a difficult learning curve for the software testers to climb before they are able to successfully employ the testing concepts used by the developers.

SUMMARY OF THE INVENTION

The present embodiment provides a simulator tool for testing software components, modules, and applications including a simulator to test the software, an interface to promote communication between the simulator and the software, a message including a component utilized by the simulator to promote testing of the software, and a test controller operable to communicate the message to the simulator, wherein the message is utilized by the simulator to test the software.

In one embodiment a method of testing software components, modules, and applications is provided that comprises providing software under test; providing a script to a test controller; communicating the script, via the test controller, to a simulator simulating a software component, module, or application that communicates with the software under test; and testing the software under test by the simulator performing the script.

In one embodiment a system for testing software components, modules, and applications is provided that includes a test scenario operable to maintain a message, a simulator to simulate a software component, module, or application in communication with the software to be tested, a test controller operable to obtain the message from the test scenario and communicate at least a portion of the message to the simulator, and a tool to develop at least a portion of the message and provide at least a portion of the message to the test scenario.

These and other features and advantages will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. It is important to note that the drawings are not intended to represent the only form of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

It should be understood at the outset that although an exemplary implementation of one embodiment of the present invention is illustrated below, the present system may be implemented using any number of techniques, whether currently known or in existence. The present disclosure should in no way be limited to the exemplary implementations, drawings, and techniques illustrated below, including the exemplary design and implementation illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

Turning now toFIG. 1a block diagram of one embodiment a simulation test tool8is depicted. A software under test (SUT)10is being tested. The SUT10may be a software component, a software module, or a software application. A simulator12communicates with the SUT10via a connector14. The simulator12is controlled by a test controller16which sends the simulator12a message18that specifies the communication, interaction, or operation the simulator12carries out with the software10. The simulator12reports the results of this interaction to the test controller16. The test controller16records the results of the interaction between the simulator12and the SUT10in a test log20. The test log20may be examined to determine the result of the test. The test controller16and the simulator12may execute on one general purpose computer system and the SUT10may execute on a second general purpose computer system. General purpose computer systems are discussed in more detail hereinafter.

In this embodiment the simulator12takes the place of some software component, module, or application—referred to herein as componentX for ease of understanding—with which the SUT10communicates or interacts. The simulator12does not necessarily implement the complex functionality of the missing componentX, but instead the simulator12simply supports the communication interface, through connector14built into or utilized by the simulator12, between the SUT10and componentX. Under control of the test controller16, the simulator12may send or receive a sequence of messages18, and report back the result of each interaction with the SUT10to the test controller16.

Although only message18is illustrated in the present embodiment, is should be appreciated that the actual content and effect of message18between the test controller16and the simulator12and between the simulator12and the SUT10differs considerably. For example, when the test controller18sends the message18to the simulator12, the message18may include instructions for the simulator12, but the message18sent between the simulator12and SUT10may include testing information or other instructions for the SUT10. The message18may be an XML message, a copybook-compliant message or a Java object when communicated between the simulator12and the SUT10. The content of the message18exchanged between the simulator12and the test controller16may consist of, for example, only a test script identification identifying a script that is to be executed by the simulator12.

The connector14provides the functionality of communication with the SUT10according to some specific technology or protocol. For example, the connector14may support Message Broker communication, Information Broker communication, Java messaging service (JMS), IBM MQSeries communication, UNIX socket communication, file system communication (perhaps using the file transfer protocol (FTP)), database management system (DBMS) communication, Vitria BusinessWare communication, common object request broker architecture (CORBA) communication, for example. By delegating detailed knowledge of the communication technology to the connector14the problem of developing software for this intercommunication can be solved once and reused many times. The simulator12is simply built with the appropriate connector14needed to communicate with the SUT10. In some embodiments the simulator12may be built with multiple connectors14—each connector14supporting a different communication mechanism—to support testing the different communication modes between the SUT10and the unavailable software—such as componentX—which the simulator12takes the place of.

The message18may be structured to include a directional component indicating whether the communication is outbound from the simulator12(hence the simulator12is to send a message to the SUT10) or is inbound to the simulator12(hence, the simulator12is to receive a message from the SUT10). The message18may be structured to include a data component. This data component may specify the value of data to be sent from the simulator12to the SUT10in an outbound direction or may specify the expected value of data to be sent from the SUT10to the simulator12in an inbound direction. The data may specify the communication mechanism the simulator12is to employ and hence what connector14to employ when communicating with the SUT10.

In the present embodiment, the message18communicated between the test controller16and the simulator12is a test script that includes information or a message within the test script. In this embodiment, the message18or test script includes the directional component. The test controller16, using the message18or test script, instructs the simulator12to execute a particular test script by passing, for example, a test script ID. The simulator12then loads the test script and sends the information to SUT10, via the appropriate connector14.

As a simple illustration, suppose SUT10squares numbers which are sent to it. The test controller16sends a message18with contents {outbound, 3} to the simulator12. The simulator12communicates the number ‘3’ to the SUT10via the connector14. The simulator12reports to the test controller that it has sent the number ‘3’ to the SUT10. The test controller16records this report in the test log20. The test controller16sends a message18with contents {inbound, 9} to the simulator12. This indicates that the simulator12should receive a communication from the SUT10and the communication should contain the value 9 (the square of the number 3 earlier presented to the SUT10).

The simulator12readies itself to receive communication from the SUT10. The SUT10sends the number ‘9’ to the simulator12via the connector14. The simulator12compares the received number ‘9’ against the expected value ‘9’ sent to it by the test controller16. The simulator12reports to the test controller16that it has received the expected value from the SUT10. The test controller records this report in the test log20. Note that the notation “{inbound, value}” is arbitrary and used for convenience here; it is not meant to describe the format of message communication between the test controller16and the simulator12. It is to be appreciated that this example is provided solely for illustrative purposes and that actual messages18and data may be significantly more complicated.

If the SUT10is in communication with more than one other componentX as described above, one simulator12may not suffice for testing the SUT10. Suppose that communication from a first componentX directed to the SUT10induces the SUT10to send a communication to a second componentY, in this case one simulator12does not suffice.

Turning now toFIG. 2, a block diagram of another embodiment of the simulation test tool8is depicted. Here again the SUT10is being tested. A first simulator12communicates with the SUT10via the connector14. A second simulator40communicates with the SUT10via a connector42. The two simulators, the first simulator12and the second simulator42, are substantially the same. Both the first simulator12and the second simulator40are controlled by the test controller16which sends the simulators12and40messages18that specify the interactions the simulators12and40are to carry out with the SUT10. The test controller16records the results of the interaction between the simulator12and40and the SUT10.

The connectors14and42may support the same communication technology or they may support different technologies. While the simulators12and40are shown as having each only a single connector, in some embodiments one or both of the simulators12and40may have multiple connectors.

Testing software typically involves conducting a number of tests each of which differs from the others in perhaps small ways. Each of these tests may be referred to as a test case. A test case in the simulation test tool8environment may be defined by a sequence of messages18that the test controller16handles one at a time, one after another, until the complete sequence of messages18has been traversed. This sequence of messages may be referred to as a message queue. A message queue defines a single test run or test case or test suite.

In software testing it may be desired to test the SUT10while it is conducting multiple independent streams of interactions. This may be accomplished by concurrent testing. In one embodiment concurrent testing is accomplished by having the test controller16sequence two or more independent message queues in independent concurrent processing threads.

Turning toFIG. 3an exemplary concurrent test case or test suite is depicted. An indefinite number n of message queues Q160, Q262, through Qn64are depicted. Each message queue comprises an indefinite number of messages. Message queue Q160comprises a number k messages M1. Message queue Q262comprises a number i messages M2. Message queue Qn64comprises a number g messages Mn. When the test controller16executes this concurrent test case it will spawn a number n processing threads. Each of these n test controller threads will sequence through one of the n message queues60,62, and64. The result is that interactions or operations are presented by the simulators12to the SUT10not one at a time but perhaps several at roughly the same time.

The test controller threads send messages18to simulators12and40at their own pace. It need not be the case that the test controller thread handling Q262send message M22to a simulator12or40at the same time that the test controller thread handling Q160sends message M12to a simulator12or40. If the intent of the test case requires that such messages18be synchronized, then thread synchronizing may be conducted between the test controller threads and may also be stipulated in the test case definition. The messages18may include delays, for example M11and M12, thereby effecting a synchronized delay in transmitting the message18to the simulator12or, depending upon the test script, a delay in the simulators12acting on the message18and sending information to the SUT10. Such capability allows the SUT10to be simultaneously tested from multiple simulators12sending varying, or perhaps the same, information to the SUT10.

The message18may also be referred to as a test script. Note that this term is not intended to imply that any executable instruction is embedded in the message18, though transmitting executable instructions is not prohibited. Also note that this term does not refer to a sequence of messages18but to a single message18.

In one embodiment the message18comprises the following structure. A test operation ID indicates what operation is intended for this message18. A simulator ID indicates what simulator is to act upon this message18. An outbound indicator—either true or false—indicates if the message18is outbound or inbound. A message delay defines an optional delay before the simulator12replies. An expected data section defines the value to be compared by the simulator12against incoming communication from the SUT10. An output data section defines the value that the simulator12sends to the SUT10.

The message18may be articulated in eXtensible markup language (XML). An example XML articulation of a message18is:

The several instances of “. . . ” in the example above would be replaced by actual values. Note that the above example is provided solely to illustrate one method of articulating the message18, but many other methods of articulating the message18will readily suggest themselves to one of ordinary skill in the art and are within the spirit and scope of the present invention. One skilled in the art will readily appreciate how XML or other syntaxes may be employed to articulate such messages18.

In one embodiment the messages18may be classified into four different classes: asynchronous outbound messages18, asynchronous inbound messages18, synchronous outbound messages18, and synchronous inbound messages18.

An asynchronous outbound message18is used by the simulator12to send a communication to the SUT10asynchronously: that is, the simulator12does not wait for the SUT10to respond to its communication. An asynchronous outbound message18is distinguished from a synchronous outbound message18by having an empty expectedData field.

An asynchronous inbound message18is used by the simulator12to receive a communication from the SUT10asynchronously: that is, the simulator12is not waiting for the communication from the SUT10. An asynchronous inbound message18is distinguished from a synchronous inbound message18by having an empty outboundData field.

A synchronous outbound message18is used by the simulator12to send a communication to the SUT10synchronously: that is, the simulator12waits for a response to its communication. A synchronous outbound message18is distinguished from an asynchronous outbound message18by having a valid expectedData field.

A synchronous inbound message18is used by the simulator12to receive a communication from the SUT10synchronously: that is, the simulator12responds to the communication from the SUT10. A synchronous inbound message18is distinguished from an asynchronous inbound message18by having a valid outboundData field.

Turning now toFIG. 3A, in one embodiment the message18is comprised of a message template66component and a message data68component. The message data68component comprises the data portion of the message18. The message template66component comprises the rest of the message18. In this embodiment both the message template66and the message data68are stored in a file system. When the test controller16instructs a simulator12to do something, instead of sending the complete message18it sends only a message id. The simulator12looks up the message template66using the message id. Any data needed for the transaction is also looked up using information from the message template66. The stored messages18may be accessed from the file system; the stored messages18may be loaded into the simulator12process memory; or the stored messages18may be accessed via an universal reference locator (URL).

In this embodiment the simulator12behavior for an outbound message18is different from the simulator12behavior for an inbound message18. For an outbound message18, the simulator12searches for the message template66using the supplied testScipt ID in the stored collection of messages18. The simulator12looks up the output data of the located message template66. The simulator12looks for an appropriate connector14by comparing the testOperation ID with the supported operations of each of the connectors14built into the simulator12. If the connector14is found, the simulator12delegates the output data to the connector14for delivery to the SUT10.

For an inbound message18, the behavior is different because the SUT10does not know about testScript ID. When the simulator12receives an inbound communication from the SUT10, the simulator12is passed an operation name and a business key (or primary key) which are obtained from the inbound communication by the connector14. The simulator12compares the operation name and business key with the testOperation ID of each of the stored message templates66until it finds a match. If the testOperation ID is matched then the message data68associated with the message template66is looked up. This message data68is compared with the business key. If matched, then the output data of the message18is returned.

In the present embodiment, the simulator12is programmed to remain in a waiting mode and the test controller16issues a time-out exception. The test controller16, and not the simulator12, effects the execution ordering of the test scripts within the test suite. The test controller16processes the test scripts or messages18, and based on the content of the test script, the test controller16determines whether it is an inbound or outbound test script. If the current test script is outbound, then the test controller16notifies the appropriate simulator12to execute the test script.

The test controller16may not know what the test script does or will do to the SUT10. Where the message18or test script is inbound, the test controller16will automatically wait for the responsible simulator12to provide status. Since the test controller16goes to wait state on its own and is programmed for when to do time out, for example, 3 minutes after entering the wait state. During the wait time, test controller16monitors whether the response to the test script has been delivered by the particular simulator12. If the response arrives, then the test controller16will get out from the wait state and proceed to the next test script. If the response does not arrive, then the test controller16will send a time out exception, which essentially terminates the execution queue in which the test script is residing, but not the entire test suite since a test suite can have multiple execution queues.

In one embodiment, the simulator12is provided with limited functionality to compare or otherwise manipulate and/or analyze the messages18send to and received from the SUT10and such functionality is provided by the test controller16with the simulator12having a passive communication role. In other embodiments, the simulator12is provided with functionality to compare or otherwise manipulate and/or analyze the messages18, while in yet other embodiments, the test controller16and simulator12are both provided with such functionality or together include such functionality.

Sometimes software communicates with other software via an intermediary middleware software package. Some examples of such middleware may include IBM MQSeries, Vitria BusinessWare, Message Broker, etc. In one embodiment, when the SUT10is designed to communicate with other software components using middleware, the actual middleware is employed in testing the SUT10. Turning now toFIG. 4, the SUT10is shown in communication with the simulator12via a middleware80. Note that the connector14will be selected to match up with the specific middleware80employed. The rest of the simulation test tool8architecture remains unchanged. The middleware80may execute on the same general purpose computer system that the SUT10executes upon, or it may execute upon the general purpose computer system that the simulation test tool8executes on. General purpose computer systems are discussed in detail hereinafter.

The embodiment depicted inFIG. 1contains a single simulator12. The embodiment depicted inFIG. 2contains two simulators12and40. Other embodiments may contain a greater number of simulators. There is no intrinsic limit upon the number of simulators12which may be included in a simulation test tool8. In practice the number of simulators12is determined by how many external software components or software modules or software applications that the SUT10communicates with to be adequately tested.

In bothFIG. 1andFIG. 2the SUT10is depicted by a single box. In some embodiments the SUT10need not be a single software component or a single software module or a single software application. Turning now toFIG. 5the test controller16is shown in communication with simulator12and simulator40. The test controller16is prepared to execute test suite100comprising a series of messages18.

Simulator12has three connectors: a JMS connector102, a CORBA connector104, and a socket connector106. The JMS connector102is employed when the simulator12communicates with a first SUT108. The socket connector106is employed when the simulator12communicates with a second SUT110.

Simulator40has three connectors: a JMS connector112, a CORBA connector114, and a socket connector116. The CORBA connector114is employed when the simulator40communicates with a third SUT118. The socket connector116is employed when the simulator40communicates with the second SUT110. In some embodiments the SUT108,110, and118may also communicate with each other.

The simulation test tool8is useful for testing projects and multi-component systems where a number of applications, some of which may communicate with one another, are involved. As an example of this interaction, the SUT108is tested by the simulator12, which prompts SUT108to transmit information to SUT110. Thus, SUT110is tested by information received from SUT108as well as the simulator40. In the event the SUT108operates incorrectly or otherwise becomes inoperable, an additional simulator, such as simulator12or40can easily be used to replace the SUT108, so that testing of SUT110and118can continue. This example is provided as one illustration of the versatility and flexibility of the simulation test tool8to test applications, including but not limited to, application components, interfaces and entire projects.

In one embodiment additional control and helper software components form a part of the simulation test tool8.FIG. 6depicts a client-server embodiment of the simulation test tool8. A test execution console140is a client of the test execution server142. The test execution server142supports a service agent144, a module manager146, an information manager148, the test controller16, and the simulator12. The service agent144acts as a request and response hub. The test execution console sends requests to the service agent144, and the service agent144sends responses back to the text execution console140. The service agent144also provides for intercommunication among the text execution server142components including the simulator12, the test controller16, the module manager146, and the information manager148. The simulator contains the connector14. The information manager148generates a test run log150, a test module log152, and a test suite result log154. The test execution console156persists a test run report156. The SUT10is typically outside of the test execution server142.

The test execution console140functions as the main window to the test execution server142. It is employed by a user, perhaps a developer or tester, to initiate and abort test runs and to view the test run report156.

The module manager146starts and stops the simulator12, the test controller16, the information manager148, and the service agent144. In some embodiments, the test execution may stop when an error or exception is detected.

In one embodiment, the information manager148captures and persists information produced by the simulator12, the test controller16, and the service agent144in log files150,152, and154. The test run log150records information associated with one or more simulators12. The information may include inbound messages18, outbound messages18, exception records (an example of an exception is when some unexpected condition occurs, usually an error), and processing information. The test run log150may be stored in XML format on the test execution server142. The test module log152records information from test execution server142modules including the simulators12, the test controller16, and the service agent144. The information may include startup and shutdown sequence, non-test run specific information (such as “heart beat” information for socket communications), and stdout and stderr information. (In the UNIX system processes may read from a standard input file [stdin], write to a standard output file [stdout], and write error information to a standard error file [stderr].) The test module log152is stored in free-text format on the test execution server142. The test suite result log154consists of information recording the results of each message18which the test controller16activates. The result of each message18may be passed, failed, skipped, or other status. The test suite result log154is stored in XML format on the test execution server142.

The test execution console140may build the test run report156from information obtained from the information manager148. The test run report156contains the status of all messages18of a test suite100(this information may be derived from the test suite result154) and inbound, outbound, and exception information during test suite execution (derived from the test run log150). In some embodiments this report may be built by the test execution server142and transmitted to the test execution console140.

Again, while only a single simulator12is depicted inFIG. 6, in some embodiments many simulators12may be employed. While only a single connector14is depicted as part of simulator12, in some embodiments the simulators12may have one or more connectors14. While only the SUT10is depicted, multiple SUT10may be tested at once by the simulation test tool8.

The test execution server142, the test execution console140, and the SUT10may each execute on their own dedicated general purpose computer system. A general purpose computer system suitable for these purposes is discussed in detail hereinafter.

Only a few topologies for structuring the simulation test tool8have been described. For example, the structural relationship between the test controller16, the simulators12&40(or other additional simulators). Also, description of the SUT (a single component10or multiple components108,110,118, or more). A description has been provided of the additional control and helper components including the test execution console140, the service agent144, the module manager146, and the information module148. I should be appreciated, however, that other topologies to which the simulator tool8may be extended will suggest themselves to one skilled in the art. All of these other topologies are contemplated by this disclosure.

A graphical user interface (GUI) tool is contemplated which a user may employ to construct messages18and arrange these messages18in series in queues60. This GUI may permit lifting subsections or slices of existing test cases for reuse in creating new test cases or test suites.

The software applications described above may be implemented on any general purpose computer with sufficient processing power, memory resources, and network throughput capability to handle the workload placed upon it.FIG. 7illustrates a typical, general purpose computer system suitable for implementing the present invention. The computer system180includes a processor182(which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage184, read only memory (ROM)186, random access memory (RAM)188, input/output (I/O)190devices, and network connectivity devices192. The processor182may be implemented as one or more CPU chips.

The secondary storage184is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM188is not large enough to hold all working data. Secondary storage184may be used to store programs which are loaded into RAM188when such programs are selected for execution. The ROM186is used to store instructions and perhaps data which are read during program execution. ROM186is a non-volatile memory device which typically has a small memory capacity relative to the larger memory capacity of secondary storage. The RAM188is used to store volatile data and perhaps to store instructions. Access to both ROM186and RAM188is typically faster than to secondary storage184.

I/O190devices may include printers, video monitors, keyboards, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices. The network connectivity devices192may take the form of modems, modem banks, ethernet cards, token ring cards, fiber distributed data interface (FDDI) cards, and other well-known network devices. These network connectivity192devices may enable the processor182to communicate with an Internet or one or more intranets. With such a network connection, it is contemplated that the processor182might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed using processor182, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.

The processor182executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage184), ROM186, RAM188, or the network connectivity devices192

It can be seen from the above disclosure that the several embodiments described can be employed to support various software testing needs. Repeatability of tests and capturing objective test results derives from the structuring of the test inputs as messages18. Concurrent testing and stress testing are supported by the several embodiments. Early testing of software under development is supported by the several embodiments, thus increasing the prospects for early detection and repair of software bugs when the repair costs are less than they are later in the development cycle. The employment of connectors14reduces the learning curve for testers, again resulting in a cost savings.

While several embodiments have been provided in the present disclosure, it should be understood that the present invention may be embodied in many other specific forms without departing from the spirit or scope of the present invention. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein, but may be modified within the scope of the appended claims along with their full scope of equivalents. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.

Also, techniques, systems, subsystems and methods described and illustrated in the various embodiments as discreet or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present invention. Other items shown as directly coupled or communicating with each other may be coupled through some interface or device, such that the items may no longer be considered directly coupled to each but may still be in communication with one another. Other examples of changes, substitutions, and alterations are readily ascertainable by on skilled in the art and could be made without departing from the spirit and scope of the present disclosure.