Computer system, test method, and recording medium

A processor acquires a plurality of test items for a simulation model of a target system, each test item including at least one event; detects a set of test items from the plurality of test items under an aggregation condition in which an event in the set is not dependent on an event in another test item in the set; includes a common event, which is an event in the set that is the same as an event in another test item in the set, in an aggregate test item while avoiding overlapping of the common event, and includes a unique event, which is an event in the set that differs from events in other test items in the set, in the aggregate test item; and executes a simulation using the simulation model in accordance with the aggregate test item.

TECHNICAL FIELD

The present invention relates to a computer system.

BACKGROUND ART

An embedded system is constituted by a mechanism which constitutes a control target, hardware which performs a control operation based on a physical amount received from the mechanism and which outputs a control value to the mechanism, and software which runs on the hardware. For example, an embedded system for controlling an automotive engine is constituted by a mechanism such as an engine that is a control target, an electronic device such as a microcontroller or a microprocessor which performs a control operation and controls the engine or the like, and software which runs on the microcontroller or the microprocessor.

Since behavior of software included in an embedded system strongly depends on a configuration of a mechanism that is a control target and on hardware configuration, an analysis of behavior must be based on a combined consideration of the mechanism, the hardware, and the software.

In recent years, embedded systems are becoming increasingly complex due to higher reliability and higher functionality of automobiles, electric devices, and the like. Consequently, in order to shorten work periods, specialization is performed by subdividing respective hardware and software components and simultaneous development is carried out at a plurality of locations. Progress in specialization not only results in increased operation checks for each component but also in increases of performance deficiency and specification failure which are discovered when assembling the components. As a result, there are frequent occurrences of delays in a development period due to rework in a final stage prior to product shipment and a decline in development efficiency has become a problem.

In order to solve this problem, performance evaluations and verification methods are used at a time point of design by a simulation in which a mechanism, hardware, and software cooperate with one another.

When executing the simulation described above, in addition to connecting models simulating operations of an execution target, a disturbance condition must be inserted to each simulator and a mechanism for observing internal states must be added to each simulator. The insertion of a disturbance condition and the observation of internal states are required to not only occur at a time point determined in advance which precedes the start of a simulation but to be also triggered by a specific state reached by a simulator. This is due to the fact that increased complexity of an entire system results in higher importance of analyses of abnormal states originating in complex mutual relationships that exist among components.

However, a development period is affected by the enormous number of test items for a performance evaluation.

PTL 1 describes, in a case where a plurality of test items have a common input data range with respect to software, consolidating the test items into a single integrated test item to reduce the number of steps related to a test.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

The technique described in PTL 1 integrates a plurality of test items only when the test items have common input data and, at the same time, the input data has a common range. Therefore, it is difficult to significantly reduce the time required for a test.

Solution to Problem

In order to solve the problem described above, a computer system according to an aspect of the present invention includes a storage device and a processor coupled to the storage device. The processor is configured to: acquire a plurality of test items for a simulation model of a target system, each test item including at least one event; detect a set of test items from the plurality of test items under an aggregation condition in which an event in the set is not dependent on an event in another test item in the set; include a common event, which is an event in the set that is the same as an event in another test item in the set, in an aggregate test item while avoiding overlapping of the common event, and include a unique event, which is an event in the set that differs from events in other test items in the set, in the aggregate test item; and execute a simulation using the simulation model in accordance with the aggregate test item.

Advantageous Effects of Invention

A time required for a test by a simulation model can be reduced.

DESCRIPTION OF EMBODIMENT

Although information may be described below using expressions such as an “xxx item”, information may be expressed using any kind of data structure. In other words, an “xxx item” can also be referred to as “xxx information” in order to show that information is not dependent on data structure. In addition, in the following description, a configuration of each table represents an example and one table may be divided into two or more tables and all of or a part of two or more tables may constitute one table.

In addition, in the following description, while an ID is used as identification information of an element, other types of identification information may be used in place of, or in addition to, an ID.

Furthermore, in the following description, when describing elements of a same type without distinguishing the elements from one another, reference signs or a common number in reference signs may be used. However, when describing elements of a same type by distinguishing the elements from one another, reference signs of the elements may be used or IDs assigned to the elements may be used in place of the reference signs of the elements.

Furthermore, in the following description, input/output (I/O) signifies write or read and may instead be referred to as an access.

In addition, while a “program” is sometimes used as a subject when describing a process in the following description, since a program causes a prescribed process to be performed by appropriately using a storage resource (such as a memory) and/or an interface device (such as a communication port) and the like when being executed by a processor (such as a central processing unit (CPU)), a “processor” may be used instead as a subject of a process. A process described using a program as a subject may be considered a process performed by a processor or by an apparatus including the processor or may be considered a system. Furthermore, a processor may include a hardware circuit which performs a part of or all of a process. A program may be installed in an apparatus such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor (for example, a CPU) and a storage resource, and the storage resource may further store a distribution program and a program that is a distribution target. Furthermore, by having the processor of the program distribution server execute the distribution program, the processor of the program distribution server may distribute the program that is the distribution target to other computers. In addition, in the following description, two or more programs may be realized as one program or one program may be realized as two or more programs.

FIG. 1shows a configuration of a computer system according to the present example.

The computer system includes a main storage unit110, a CPU120, an external storage device130, an input device140, and a display device150.

The main storage unit110is a main storage device such as a memory and stores programs and data. The CPU120executes a process in accordance with a program stored in the main storage unit110. The external storage device130is an auxiliary storage device such as a hard disk drive (HDD) or a solid state drive (SSD) and stores programs and data. The input device140acquires data input by a user. The display device150displays data.

The main storage unit110stores programs of an aggregation unit220, a test operation unit230, and a simulation execution unit240. The external storage device130stores data including an analysis result310, a test item group320, a simulation model group330, and a test result340. Any of these programs and data may be stored in the main storage unit110, stored in the external storage device130, or stored in a storage coupled via a communication network.

The test item group320includes a plurality of test items. Each test item includes at least one event. Each event represents at least any of a trigger of an event, an operation with respect to a state of the simulation model group330, and an observation of the state of the simulation model group330. Examples of test items include an input test item which is input from the input device140, an aggregate test item which is obtained by aggregating a plurality of input test items, and a retest item which includes an event to be retested among the aggregate test item. An input test item may be received via a communication network from an external device. The test result340includes a determination result list which indicates determination results of events in a test item and a finalized result list which indicates a finalized determination result among determination results of events in the test item.

The analysis result310represents an analysis result or an execution result of the simulation model group330. For example, the analysis result310represents a static analysis result of the simulation model group330or an execution result of the simulation execution unit240using the simulation model group330. The aggregation unit220may perform a static analysis of the simulation model group330and acquire a static analysis result. The test operation unit230may have the simulation execution unit240execute a simulation and may acquire an execution result.

The aggregation unit220generates an aggregate test item by aggregating a plurality of input test items included in the test item group320. The test operation unit230controls the simulation execution unit240in accordance with a test item. The simulation execution unit240executes a simulation using the simulation model group330.

FIG. 2shows a configuration of the simulation model group330.

The simulation model group330includes a plurality of simulation models representing a target system of a test. A simulation model is, for example, a program representing a model of elements of a target system. Elements include a mechanism, hardware, and software. The target system according to the present example is a control system of an automobile.

The simulation model group330according to the present example includes an on-board communication controller model410which is a model of an on-board communication controller, an engine ECU model421which is a model of an engine electronic control unit (ECU) coupled to the on-board communication controller, an engine model422which is a model of an engine coupled to the engine ECU, a brake ECU model431which is a model of a brake ECU coupled to the on-board communication controller, a brake model432which is a model of a brake coupled to the brake ECU, a user interface ECU model441which is a model of a user interface ECU coupled to the on-board communication controller, a user operation panel model442which is a model of a user operation panel coupled to the user interface ECU, a steering ECU model451which is a model of a steering ECU coupled to the on-board communication controller, and a vehicle body attitude model452which is a model of a vehicle body attitude affected by the steering ECU. The configuration of the simulation model group330shown inFIG. 2is an example. The configuration of the simulation model group330is not limited to this example.

The plurality of ECUs in the control system are coupled to one another via a communication network such as a control area network (CAN).

The test operation unit230or the simulation execution unit240may include a probe unit which acquires a state of each unit in the simulation model group330. The probe unit operates or observes analog values or digital values in the simulation model group330.

FIG. 3shows a configuration of an input test item.

FIG. 3shows input test items321aand321bin the test item group320. Each input test item includes an entry for each event. An entry of a single event includes a trigger item510, an operation item520, and an observation item530. Each of the operation item520and the observation item530may not be provided or a plurality of operation items520and a plurality of observation items530may be provided.

The trigger item510includes an indication (or an operation)511which indicates a type of a trigger, a configuration value (or an argument)512which indicates a state used for the trigger, and a condition513which indicates a condition of the trigger. The operation item520includes an indication521which indicates a type of an operation and a configuration value522which indicates an operation target and an operation value. The observation item530includes an indication531which indicates a type of an observation, a configuration value532which indicates an observation target, and a condition533which indicates a determination condition of an observation result. Any of the trigger item510, the operation item520, and the observation item530may not have a value or may include a plurality of values.

For example, in the trigger item510, when the indication511indicates a wait time and the configuration value512indicates a value of the wait time, the trigger of this event is that the wait time elapses. For example, in the trigger item510, when the indication511indicates a function trigger, the configuration value512indicates a function name, and the condition513indicates called, the trigger of this event is that the function represented by the function name is called.

For example, in the operation item520, when the indication521indicates I/O set and the configuration value522indicates an output destination and a value, the operation for this event is to output the value to the output destination. For example, in the operation item520, when the indication521indicates write memory and the configuration value522indicates an operation target and an operation value, the operation for this event is to write the operation value in a memory that is the operation target.

For example, in the observation item530, when the indication531indicates read memory and the configuration value532indicates an observation target, the observation for this event is to read a value from the observation target. Furthermore, in the observation item530, when the condition533indicates a determination condition and the read value satisfies the determination condition, the event is determined to have passed. For example, the indication531indicating “Nop” in the observation item530means that there is no observation item530.

An event for which the condition533of the observation item530is configured may be referred to as a determination event.

The operation item520may involve writing data representing a disturbance to a memory. The operation item520may indicate transmitting a signal to another model via a communication network. The observation item530may indicate receiving a signal from another model via a communication network. The operation item520may indicate a destruction operation for simulating a failure of a target system such as stopping a clock or changing frequencies of the clock.

Each entry may include an identifier of an event. Each entry may include an attribute indicating whether or not aggregation can be performed. Each entry may include a plurality of operation items520or may include a plurality of observation items530.

FIG. 4shows an outline of generation of an aggregate test item.

The aggregation unit220detects a set of input test items satisfying an aggregation condition among the plurality of input test items in the test item group320. For example, the aggregation condition is a condition in which an event in each input test item in the set is not dependent on an event in another input test item in the set. In addition, the aggregation unit220considers a completely matching event among a plurality of input test items in the set to be a common event, considers an event other than a common event to be a unique event, and generates one aggregate test item by eliminating overlapping of common events and arranging common events and unique events in an order of execution time points.

In the example shown inFIG. 4, the aggregation unit220generates a first set including an input test item321ahaving a unique event A, an input test item321bhaving a unique event B, and an input test item321chaving a unique event C, and generates a second set including an input test item321dhaving a unique event D, an input test item321ehaving a unique event E, and an input test item321fhaving a unique event F. In addition, the aggregation unit220generates an aggregate test item322gby arranging the common events and the unique events in the first set in an order of triggers and generates an aggregate test item322hby arranging the common events and the unique events in the second set in an order of triggers.

FIG. 5shows a configuration of an aggregate test item.

FIG. 5shows an aggregate test item322robtained by aggregating input test items321pand321q. A configuration of an aggregate test item is similar to that of an input test item. Moreover, both an input test item and an aggregate test item may be referred to as a test item.

In the aggregate test item322r, each event from number 1 to number 5 is a common event and completely matches between the two input test items321pand321q. A sixth event is a unique event of the input test item321pand a seventh event is a unique event of the input test item321q.

When using the two input test items321pand321q, a second test is performed with respect to the common events numbers 1 to 5. When using the aggregate test item322r, only one test is performed with respect to the common events numbers 1 to 5. In this manner, by aggregating a plurality of input test items into an aggregate test item, the number of tests with respect to common events in the plurality of input test items can be reduced and overall test time can be shortened.

Hereinafter, operations of the computer system will be described.

The computer system performs a test process with an aim to have all determination events among all input test items pass.

In S110, the aggregation unit220acquires a plurality of input test items input to the input device140and registers the plurality of input test items in the test item group320.

In S120, based on the analysis result310, the aggregation unit220analyzes an estimated ignition time point of a trigger item of an event in the plurality of input test items in the test item group320. An estimated ignition time point is a predicted value of a time point at which the trigger item510ignites at a simulation time point. An estimated ignition time point may be a predicted value of an execution time point of an event.

In S130, based on an analysis result of the simulation model group330, the aggregation unit220analyzes an active range of an operation item of each event in the plurality of input test items in the test item group320. An active range is a range affected by an operation item. For example, an active range is an operation target indicated by the configuration value522and is an address range of a memory written by an operation, a name of a signal used in communication by an operation, or the like.

In S140, the aggregation unit220divides the plurality of input test items into a plurality of sets by detecting sets of a plurality of input test items satisfying an aggregation condition in the plurality of input test items. For example, the aggregation condition is a requirement that an estimated ignition time point of an event in a set does not overlap with an estimated ignition time point of an event in another input test item in the set. Alternatively, the aggregation condition may be a requirement that an active range of an event in a set does not overlap with an active range of an event in another input test item in the set. Alternatively, the aggregation condition may be a requirement that, when an input test item in a set includes an event of a destruction operation, an event of a destruction operation is not included in another input test item in the set. Alternatively, the aggregation condition may be a requirement that, when an input test item in a set includes a communication operation and another input test item in the set includes a communication operation, contents of the communication operations differ from each other.

In S150, the aggregation unit220generates an aggregate test item by arranging a plurality of events in the set in an order of estimated ignition time points and aggregating the arranged events into one test item, and registers the aggregate test item in an execution test item group. Accordingly, the test operation unit230can execute events in the aggregate test item in accordance with an execution order of corresponding events in the respective input test items. At this point, the aggregation unit220associates, with the aggregate test item, input test items in the set which correspond to the aggregate test item. At this point, the aggregation unit220may register input test items which have not been aggregated in the execution test item group.

In S160, the test operation unit230sequentially selects test items from the execution test item group as target test items and acquires a determination result list of each test item by executing a test item execution process with respect to a target test item. A determination result list includes a determination result of each determination event. Accordingly, the test operation unit230can perform verification in an order of simulation time points of determination results of each determination event.

In S170, the test operation unit230selects a top event in the determination result list as a target event and determines whether or not a determination result of the target event satisfies a nonaggregable condition. At this point, when a determination result of the target event by the current test item execution process differs from a determination result of the target event by a previous test item execution process, the test operation unit230determines that the target event satisfies a nonaggregable condition. For example, when the target event is determined to have passed in the current determination result list, there is a finalized result list including the target event, and the target event is determined to have failed in the finalized result list, the test operation unit230determines that the target event satisfies a nonaggregable condition.

When the result of S170is yes, in S180, the test operation unit230discards a determination result related to the target test item and configures a nonaggregable attribute with respect to input test items in the set which correspond to the target test item.

In S190, the test operation unit230warns a user that there is a suspicion of multi-point failure and causes the process to make a transition to S210. At this point, the test operation unit230causes the display device150to display a warning. Subsequently, the aggregation unit220does not aggregate an input test item having a nonaggregable attribute and the test operation unit230executes a test item execution process with the input test item as a target test item. Accordingly, the test operation unit230can improve accuracy of the test.

When the result of S170is no, in S210, the test operation unit230determines whether or not the determination result of the target event is a pass preceding the fail in the target test item.

When the result of S210is yes, in S230, the test operation unit230registers the target event and the determination result thereof in the finalized result list and causes the process to make a transition to S250.

When the result of S210is no, in S220, the test operation unit230determines whether or not the determination result of the target event is a first fail in the target test item.

When the result of S220is yes, the test operation unit230causes the process to make a transition to S230.

When the result of S220is no, in S240, the test operation unit230registers the target event in the retest item and causes the process to make a transition to S250. Accordingly, the test operation unit230can retest a determination event of which a determination result has not been finalized.

In addition, the test operation unit230may register an event satisfying a prescribed retest condition in the retest item. For example, an event satisfying the retest condition is a determination event of which a target event occurs after a determination event having a failed determination result among determination events in the target test item. Alternatively, an event satisfying the retest condition may be an event preceding the target event in the target test item or an input test item corresponding to the target event.

In S250, the test operation unit230discards the target event from the determination result list.

In S260, the test operation unit230determines whether or not there are remaining events in the determination result list.

When the result of S260is yes, the test operation unit230causes the process to make a transition to S210.

When the result of S260is no, in S270, the test operation unit230determines whether or not the retest item is empty.

When the result of S270is no, in S280, the test operation unit230registers the retest item in the execution test item group and causes the process to make a transition to S120.

When the result of S270is yes, in S290, the test operation unit230creates a determination result for each input test item based on the finalized result list and ends this flow. In this case, the test operation unit230creates a determination result for each input test item by sequentially selecting events in the finalized result list as target events, specifying an input test item corresponding to the target event, and registering a determination result of the target event in a determination result of the specified input test item.

FIG. 7shows a test item execution process.

In S410, the test operation unit230sequentially acquires events from the top of the target test item as target events.

In S420, the test operation unit230advances a simulation time point by a unit time set in advance and causes the simulation execution unit240to execute a simulation. The simulation execution unit240executes a simulation corresponding to the unit time using the simulation model group330in accordance with the lapse of the simulation time point, and stops the simulation.

In S430, the test operation unit230determines whether or not a state of the simulation model group330satisfies the condition513of the trigger item510of the target event.

When the result of S430is no, the test operation unit230causes the process to make a transition to S420.

When the result of S430is yes, in S440, if the observation item530is configured with respect to the target event, the test operation unit230acquires a state represented by the indication531and the configuration value532of the observation item from the simulation model group330. Furthermore, when the condition533of the observation item is configured, the test operation unit230determines whether or not the acquired state satisfies the condition and registers a determination result of the target event in the determination result list.

In S450, when the operation item520is configured with respect to the target event, the test operation unit230changes a state represented by the indication521and the configuration value522of the operation item in accordance with the configuration value522.

In S460, the test operation unit230discards the target event from the target test item.

In S470, the test operation unit230determines whether or not the target test item is empty.

When the result of S470is no, the test operation unit230causes the process to make a transition to S410.

When the result of S470is yes, in S480, the test operation unit230outputs a determination result list of the target test item and ends this flow.

FIG. 8shows an operation of the simulation model group330in a test item execution process.

Periods of a test item execution process include, in an order of simulation time points, a trigger condition wait period #0, an event period #0, a trigger condition wait period #1, an event period #1, a trigger condition wait period #2, an event period #2, a trigger condition wait period #3, and an event period #3. For example, the trigger condition wait period #0 is a period until a state of the simulation model group330satisfies a condition of the trigger item510of a top event in the target test item. The event period #0 is a period in which the event is executed.

When a determination result of a given event is fail in a determination result list obtained by the test item execution process described above, a determination result of an event subsequent to the given event is not guaranteed.

Next, a working effect of the present example will be described using, as a comparative example, a technique for integrating a plurality of test items as one integrated test item when the plurality of test items have a common input data range.

When a plurality of test items have a same type of input data and ranges of values of the input data overlap each other, the comparative example can integrate the test items. For example, the comparative example can integrate, as one integrated test item, a test item for inputting a value equal to or larger than 100 as a vehicle speed, a test item for inputting a value equal to or larger than 80 and equal to or less than 120 as a vehicle speed, and a test item for inputting a value equal to or less than 120 as a vehicle speed. In addition, for example, the comparative example can integrate, as one integrated test item, a test item for inputting a value equal to or less than 4 MPa as brake hydraulic pressure, a test item for inputting a value equal to or larger than 2 MPa as brake hydraulic pressure, and a test item for inputting a value equal to or larger than 1 MPa as brake hydraulic pressure.

The aggregation unit220according to the present example is capable of aggregating a plurality of test items having mutually different types of events. The aggregation unit220according to the present example is capable of aggregating a larger number of test items than the comparative example and, consequently, a greater effect of reducing test time is produced.

The storage device corresponds to the main storage unit110, the external storage device130, and the like. The processor corresponds to the CPU120and the like. The simulation model corresponds to a simulation model, the simulation model group330, and the like.

While an embodiment of the present invention has been described above, it is to be understood that the described embodiment merely represents an example for illustrating the present invention and the scope of the present invention is not limited to the configuration described above. The present invention can be implemented in various other modes.

REFERENCE SIGNS LIST