ANALYZATION OF TEST RESULTS

Abstract: According to the present invention, a system and a method for analyzing processes of the system are disclosed, each process transmitting process-specific messages affecting tested objects. Services are defined for tested objects to be analyzed (S1) and events are defined for the services on the basis of the process-specific messages (S2). Then, analysis is defined for the events (S3) and the defined analysis is performed (S4).

Description: TITLE OF THE INVENTION Services m testing result analyzation. FIELD OF THE INVENTION The present invention relates to analyzing processes m a system, ana m particular to a system anc a method for analyzing processes of the system, eacn process transmitting process-specific messages affecting each other. BACKGROUND OF THE INVENTION It is a goal of testing to find all faults in a system by means of test cases. Therefore, the testing should cover all features or processes of the system. However, if same functionalities are tested several times only because it was not known that the functionality has already been tested n connection with another feature, valuable resources are wasted. Managing the testing of internal processes and interfaces in a system currently is a quite laborious task which consumes large amount of resources. In the future, as the number of products and features in the products are increasing, the management and efficiency of testing will become a crucial factor in software development. Conventionally, software testing is feature oriented. When testing of software functionality is planneα it is usually done on feature level, and test case result analyzation is carried out on inter process message level. A better way is to build up a general model including all functionalities in a process. This kind of models, however, is very laborious to establish. SUMMARY OF THE INVENTION I- is therefore an object of the present invention to facilitate analyzing complex processes in a system. According to one aspect of the present invention, this object is achieved by a method for analyzing processes of a system, each process transmitting process-specific messages affecting tested objects, said method comprising the steps of: defining services for tested objects to be analyzed; defining events for the services on the basis of the process-specific messages; defining analysis for the events; and performing the defined analysis. According to another aspect of the present invention, this object is achieved by a system including processes transmitting process-specific messages affecting tested objects, said system comprising: service defining means for defining services for tested objects to be analyzed; event defining means for defining events for the services on the basis of the process-specific messages; analysis defining means for defining analysis for the events; and performing means for performing the defined analysis. According to the present invention, testing efficiency and accuracy is significantly improved. Furthermore, according to the present invention, testing is service oriented and feature oriented. Thus, if some service is used by numerous features the analysis is described only once. Analyzing complex systems using the service model according to the present invention makes it possible to develop and use automated analyzing and testing tools. Moreover, by means of the service model, a limited amount of service usages can be handled in the same test result data. Thus, data from live systems can be analyzed. Further features of the present invention are defined in the dependent claims . In the following the present invention will be described by way of a preferred embodiment thereof with reference to the accompanying drawings . BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a service instance with service events according to the service model of the present invention; Fig. 2 shows a flowchart illustrating the basic steps for test management according to the present invention; and Fig. 3 shows a test case coverage matrix. DESCRIPTION OF THE PREFERRED EMBODIMENT The idea of the present invention is to use a service model in developing and testing processes in a system, for example processes of network elements like the mobile services switching center (MSC) and the home location register (HLR) in the GSM (global system for mobile communications) network. Service is functionality, information or dialog which an object like a process family (a provider) offers to a service user (client) . A service dialog is defined as communication practice between the service provider and the client. The service is used via the dialog. When a test case is executed m the system it produces test results like message monitorings between process families. This monitorings contain message transactions in time schedule and the content of each message. A service dialog contains message transactions required for asking service functionality from a certain process family and answers, .e. how the process family or service provider reacts. Same messages can contain information for many services. If the service functionality is modeled by a formal language preferably by some service dedicated language, an automatic device can be generated which can automatically analyze if the service functionality is correct. In the following, the service concept according to the present invention is described with reference to Fig. 1. In addition to reading all the messages from message monitoring into a database and then analyzing the whole monitoring in one great step, according to the present invention, analyzation can also be done based on services In quite many cases, the analyzation of the whole results of a test case can be understood as analyzation of a collection of smaller parts. Instead of trying to analyze the results m one step, the results are divided into parts which are more straightforward to analyze. For example, when trying to analyze a circuit group measurement report in statistics application area, it is a laborious job to read all of the monitoring into the database and run through the database, searching all of the messages affecting circuit group measurement. In order to avoid this, a service is defined, for example by using some service dedicated language, which service is capable of handling all of the messages affecting circuit group measurement. By means of this the messages in question can be found and connected together, and the measurement elements can be output one by one. One big advantage of modeling the analyzation by services is that by using services the analyzation can be divided in small parts and thus it can be developed service by service or even event by event. Fig. 1 shows a service instance for a service like the above- mentioned circuit group measurement service or a circuit group to number destination measurement service. The service instance represents an independent state machine with a life cycle from entering the instance to exiting the instance. The service instance can be originated either by an enter event or an usage event. In case the instance already exists, usage events and change events modify the instance. An exit event terminates the service instance. Every service has at least one entry point and at least one exit point. In addition, modification points can be defined. An enter event always creates a new service instance. For example, a circuit reservation is an entry point to a new circuit group measurement instance. A usage event creates a new service instance if no matching instance (i.e. no instance for the service to which the event belongs) has been created before. In case a matching instance is found, the usage event modifies the existing instance. Using an enter event as an entry point of a service instance is quite straightforward, but sometimes there are more than one entry point for a service. For example, in the circuit group to number destination measurement the Erlang calculation is started when the incoming circuit is reserved, an outgoing circuit is reserved and the number destinations are known. This information arrives in three different messages, the order of which is not defined. In view of the fact that an enter event always creates a new service instance this event cannot be used in this case. The above-mentioned messages have to be defined as usage events for the circuit group to number destination measurement service in order to create only one service instance for these messages. When a service instance is already entered, its local data can be modified with either a change event or a usage event. The modification "feature" of usage events is derivable from the fact that several usage events can affect into a single instance as described above. An exit event is valid only if the service instance to which the exit event belongs has already been entered. With the exit event, the service instance is exited, i.e. deleted. For example, a circuit release is an exit point of a circuit group measurement service instance. According to the present invention, services are defined for tested objects, and events are defined for the messages taking place in the tested objects. When an instance of a service is exited, the local data of the instance are deleted together with the instance. Thus, in order to keep the data available, it must be stored to user- defined database before exit. Each service can have its own data declarations. In this data the intermediate state of the service is stored. The declaration of data corresponds to a table definition that is used locally inside the service like a normal table. An event defined for a service represents an impulse or trigger which affects on the service instance. As mentioned above, events can invoke, modify or end a service instance. In declaring events for a service, the type. of event is defined as well as triggering conditions. To be precise, the following parameters have to be defined for an event: event type, event family, event MSG, event direction, event ID and event trigger. As already mentioned above, event types are enter, exit, change and usage. The event family indicates the sending/receiving process of the message triggering the service instance. Event MSG indicates the number of the message triggering the event. Event direction represents the direction of the message from the process of the message point of view, i.e. whether the message arrives at the process from another process or from its own hand or whether the message is sent by the process to another process or to its own hand. In case several directions are defined they are OR-connected, i.e. messages coming from either another process or the own hand are taken into account, for example. Event ID indicates the service instance to which the event belongs. Event trigger defines the basic conditions which must be fulfilled by the message. According to the present invention, for each service triggers are defined for each service event so that the service event is triggered only if it actually affects on the service. In the following, the method for analyzing processes in a complex system using service model according to the present invention will be described with reference to Fig. 2. In a first step SI services are formed for the tested objects to be analyzed, i.e. the functionality of the system is divided into services, and then, m step S2, events associated witn service dialogs are definec for each service as described aoove. After having αefined the services and events for the services, the analysis to be carried out for each event is defined in step S3. Finally, in step S4 analyzation is carried out in each event or merely m the exit event on the service data stored as described above. In practice, analyzations are implemented as subroutines. The service model according to the present invention can be used to map the features of the system to the defined services. Fig. 3 shows a test case coverage matrix. This matrix shows connections between features and services to be created from the test case point of view. Feature 635, for example, affects services cgr, mtr and exs in the statistics application area and a service serv2 in call control application area. Feature 750 affects services exs in the statistics application area, moc and call id m charging application area and servl in the call control application area. Feature pdc0034 affects services fdr m the statistics application area, cigmet in the charging application area, serv45 m HLR application area and serv3 in the call control application area. The nodes in the matrix, which indicate correspondences between features and services, can indicate the functionality, text information or a link to further information. The number of nodes in an application area is limited and can vary between features. The matrix shown in Fig. 3 can be used to optimize the testing. In defining the services according to the present invention, a service hierarchy can be used. In test case descriptions it is referred to services whose functionality can be tested or which will be tested in the test case. This kind of relation can be used to analyze testing coverage and overlapping. Furthermore, already described test cases can be found to meet testing needs for certain purposes. For every test case there is a matrix m wnich it can be indicated whicn service can be tested by the test case. Furthermore, features can be related to services which are gathered under service blocks, i.e. the application areas m Fig. 3. For example, the present invention can be applied to charging and statistics testing in MSC and HLR. According to the present invention, testing is service oriented and feature oriented. Thus, if some service is used by numerous features it is described only once. Testing of complex systems using the service model according to the present invention makes it possible to develop and use automated analyzing and testing tools. Moreover, by means of the service model, a limited amount of service usages can be handled in the same test result data. Thus, data from live systems can be analyzed. While the invention has been described with reference to a preferred embodiment, the description is illustrative of the invention ana is not to be construed as limiting the invention. Various modifications and applications may occur to those skilled in the art without departing from the true spirit and scope of the invention as defined by the appended claims.

Claims: CLAIMS : 1. A method for analyzing processes of a system, each process transmitting process-specific messages affecting tested objects, said method comprising the steps of: defining (Ξl) services for testeα objects to be analyzed; defining (S2) events for the services on the basis of tne process-specific messages; defining (S3) analysis for the events; and performing (S4) the defined analysis. 2. The method according to claim 1, wherein an event triggers the service for which it is defined. 3. The method according to claim 1, wherein a process- specific message triggers an event if the message affects the service for which the event is defined. 4. The method according to claim 1, wherein in said analysis defining step (S3) a matrix is formed which indicates which process affects which service. 5. A system including processes transmitting process-specific messages affecting tested objects, said system comprising: service defining means for defining services for tested objects to be analyzed; event defining means for defining events for the services on the basis of the process-specific messages; analysis defining means for defining analysis for the events; and performing means for performing the defined analysis. 6. The system according to claim 5, wherein an event triggers the service for which it is defined. 7. The system according to claim 5, wherein a process- specific message triggers an event if the message affects the service for which the event is defined. 8. The system according to claim 5, wherem : said analysis defining means forms a matrix which indicates which process affects which service.

Inventors: NIEMINEN JARI-PEKKA, SIISKONEN MARKO, NIEMINEN, JARI-PEKKA, SIISKONEN, MARKO
Applicants: NIEMINEN JARI PEKKA, NOKIA NETWORKS OY, SIISKONEN MARKO, NIEMINEN, JARI-PEKKA, SIISKONEN, MARKO
Application year: 1999
Main classification: G06F11
IPC codes: G06F11