Finite state machine method for test case generation and execution of communication protocols

The technology disclosed relates to implementing a novel architecture of a finite state machine (abbreviated FSM) that can be used for testing. In particular, it can be used for testing communications devices and communication protocol behaviors.

BACKGROUND

The technology disclosed relates to implementing a novel architecture of a finite state machine (abbreviated FSM) that can be used for testing. In particular, it can be used for testing communications devices and communication protocol behaviors.

Several approaches have been proposed to implement finite state machines. Some focus on minimizing memory consumption, combining or embedding finite state machine classes written in different programming languages, and exchanging inputs, triggers, or state values between finite state machines. Others implement look-up tables for actions to be performed in each state. Some of these architectures are optimized for hardware rather than software implementation.

Existing FSM architectures are cumbersome for test case modeling and generation, especially architectures that require a user to provide a large number of tables to represent states, inputs, triggers, actions, conditions, and events and that require the user to manage these tables. These architectures do not provide intrinsic support for test related operations.

An opportunity arises to provide users with a novel FSM architecture to create test cases or protocol behavior models. Fast and rapid deployment, configurability, maintenance, scalability, and ability to support multiple communicating finite state machines may result.

SUMMARY

The technology disclosed relates to implementation of a finite state machine. In some implementations, a novel architecture of a finite state machine can alleviate the complexity of modeling, generating, and executing industry-accepted testing for network communication protocol behaviors. Further details and alternative implementations appear in the accompanying figures, claims, and description.

DETAILED DESCRIPTION

The primary example in this disclosure of applying a novel finite state machine to testing is drawn from 3GPP standard TS 36.523-1 pp. 283-285, which describes test criteria for conformance of new LTE systems certain sections of the LTE standard. 3rd Generation Partnership Project, “Technical Specification Group Radio Access Network; Evolved Universal Terrestrial Radio Access (E-UTRA) and Evolved Packet Core (EPC); User Equipment (UE) conformance specification; Part 1: Protocol conformance specification (Release 8).” 668 pp. Valbonne, France (V8.1.0 Mar. 2009) accessible at www.3gpp.org (hereinafter 3GPP TS 36.523-1). Instead of using conventional test scripting in a procedural language to implement the specified test criteria, the technology disclosed provides a user modifiable FSM specification with reduced complexity, relative to FSMs designed for other special purposes.

The conformance test example inFIGS. 12-22uses FSMs instead of scripts. The test criteria specification, 3GPP TS 36.523-1, section 8.1.2 pp. 283-285, describes criteria for testing establishment of a connection when the SUT initiates an outgoing call. There are hundreds of conformance tests in 3GPP TS 36.523-1, to which the novel FSM architecture could be applied. The example of testing connection establishment should not be taken as limiting or preferred. The conformance criteria in this test illustrate the technology disclosed without being overly complicated. This example test is not intended to illustrate all of the technologies disclosed. For instance, it does not illustrate asynchronous operation of parent and child FSMs.

The technology disclosed includes an FSM architecture that separates supporting modules (or methods or objects) of an optionally hidden group, which can be delivered as an executable module, from user-modifiable code that specifies states, messages, and transition handlers. Test-related functions are intrinsic to the supporting modules. This architecture is well suited to system testing, as test states and behaviors are readily modifiable and can be instrumented to observe behavior within communication protocols. Users also can devise test protocols from scratch, using the technology disclosed and the intrinsic test-related features.

FIG. 1illustrates a high level architecture of an example-testing environment100in which the FSM technology disclosed can be used. The example testing environment100tests the protocols that are used to implement an emerging wireless technology known as “long term evolution” (abbreviated LTE), which is used for wireless voice, video and data communications. This is one of the technologies commonly referred to by cellular providers as 4G technology. The testing environment100illustrated includes a traffic generator115, test controller123, network emulator125and user equipment127. These components each include memory for storage of data and software applications, at least one processor for accessing data and executing applications, and components that facilitate communication over wired and/or wireless networks. Other configurations of environment are possible having more or fewer components than depicted inFIG. 1, either dividing roles of equipment components into more boxes or consolidating roles of multiple components into fewer boxes. The processors can be general-purpose processor, a reduced instruction set circuits (RISC processors), field programmable gate arrays (FPGAs), dedicated logic or other computing circuits. Software or firmware that implements the finite state machines can run on appropriate hardware.

The testing environment100emulates an evolved packet system, in a central LTE system network. The technology disclosed also can be applied to testing other packet systems, such as WiMax and other 802.11x networks, UTMS and CDMA networks and variations on these. The packet system illustrated transfers data packets between edge networks and user devices over a radio access network. The novel FSM architecture disclosed can be applied to other environments as well.

The traffic generator115generates traffic carried by an internet protocol-based (abbreviated IP-based) evolved packet core network that handles, for example, voice call management, data call management, and billing.

The test controller123is a computer-based system used to control, configure, debug, and trace test stimuli and responses. In one implementation, it uses a Windows XP or Windows 7 or 8 operating system (abbreviated OS). In other implementations, it uses a Linux, OS X or UNIX OS. It may be constructed as a real machine with dedicated hardware or as a virtual machine over a hypervisor. The test controller can be built on a personal computer (abbreviated PC), workstation, tablet, blade server, computing cluster or other computing device.

In one implementation, the network emulator125emulates a so-called evolved universal terrestrial radio access network (abbreviated E-UTRAN), which comprises a so-called evolved node B (abbreviated eNodeB), consistent with LTE specifications. The eNodeB provides bridging between user equipment127and the traffic generator115. The eNodeB is also a termination point for all the radio protocols towards the user equipment127and acts as data relay between the radio connection and the corresponding IP based connectivity towards the traffic generator115.

The user equipment127is a system under test (abbreviated SUT), otherwise known as a device under test (abbreviated DUT). In the example illustrated, the SUT is a mobile phone. The same approach can be applied to test or emulate other parts of a network that interact with the SUT.

In another example, the SUT could include a base station or a router. Those examples could substitute a different test harness for the network emulator125. For instance, Spirent's Test Center™ hardware could be used to test an Internet core router.

FIG. 2is an example of threading architecture200that can be used in the example-testing environment100. The threading architecture200comprises multiple layers and procedures with their own threads modeled and implemented as finite state machines. Numbering of high level blocks inFIG. 2matches numbering used inFIG. 1.

FIG. 2shows a traffic generator115which, in some implementations, includes the components depicted and described herein.

A serving gateway (abbreviated S-GW)215is the local mobility anchor for the data holders and responsible for terminating the traffic generator115interface towards the network emulator125. It also retains the information about the data holders when the user equipment127is in idle state530.

A packet data network gateway (abbreviated PDN-GW)217provides the connection between the S-GW and external data networks like Internet. It acts as the highest-level mobility anchor in the testing environment100and assigns an IP address to the user equipment127to facilitate its communication with external networks.

A mobility management entity (abbreviated MME)219is responsible for managing security functions, mobility handling, roaming and handovers.

A test controller123that comprises a tracer223. This component handles logging and tracing. Components of the testing environment100can use this component to log and trace different activities and states. A test console application can be implemented as console or GUI to retrieve logs and traces captured by this component.

The network emulator125in this example comprises a protocol stack of layers, including RRC, PDCP, RLC, MAC, and PHY layers. Some of these layers are common to a variety of cellular, mobile and LAN networks.

The RRC layer225implements a signaling protocol used to configure and control radio resources between the network emulator125and user equipment127. This layer is responsible for broadcasting system information, paging, and reporting of user equipment127along with control, management, establishment, modification, and release of connection. The RRC signaling message is carried by a signaling radio bearer (abbreviated SRB).

The user equipment127has three RRC states including the idle state1330, connecting state1335and connected state1339. In the idle state1330, no SRB is registered and RRC connection is not established. While in this state, the user equipment127detects the incoming messages and acquires system information. The connecting state1335is an intermediate state in which the timer450is initiated. In connected state539, an SRB is registered and RRC connection is established. While in this state, the user equipment127establishes a radio resource context with the network emulator125.

The packet data convergence protocol layer (abbreviated PDCP)235manages RRC layer225messages in the control plane and IP packets in the user plane along with data streams in the interface between the network emulator125and user equipment127.

The radio link control layer (abbreviated RLC)245receives/delivers data packets from/to its peer entities.

The medium access control layer (abbreviated MAC)255provides coupling between the RLC layer245and physical layer (abbreviated PHY)265and also comprises a scheduler, which distributes the available bandwidth to the user equipment127.

The PHY layer265provides data transport services on physical channels to the RLC245and MAC255.

FIG. 3illustrates a high level architecture of the LTE-Uu interface comprising the RRC protocol and its procedure. The RRC layer225can include several components, such as those described below.

A connection establishment procedure315establishes an SRB for transmitting a message to the traffic generator115. The user equipment127on request for connection initiates the procedure during the idle state1330. Primarily, this procedure is used to make the transition from the idle state1330to connected state1339. The user equipment127makes the transition to connected state1339before transferring any application data or completing any signaling procedures.

A connection reconfiguration procedure325establishes, modifies, and releases an SRB on being initiated by the network.

A connection re-establishment procedure328re-establishes the connection by resumption and reactivation of an SRB used for transmitting a message to the traffic generator115.

FIG. 4illustrates one general framework of an FSM400as disclosed in this application. The FSM400has a number of states and behaviors that are modeled by exit, entry or/and timer handlers. In addition to the states, an FSM has a number of transitions that lets the FSM switch from a current state to a different state based on the input message. An FSM can use several components for message processing including a message queue415and dispatcher410. The message queue enqueues incoming messages in a first in, first out (abbreviated FIFO) order. The dispatcher410dequeues the messages from the message queue in a FIFO manner to the current state and invokes the respective handlers for each state. If the entry, exit, and/or timer handlers are invoked and completed, the dispatcher checks the return Boolean value of these handlers. If the value is “False”, this means that the FSM is still running and the dispatcher continues to dequeue and dispatch messages. However, if the return value is “True”, the dispatcher stops dispatching any queued messages, and the FSM is considered to be complete.

In the FSM400, the MSG_INIT420is received by the initial state S0 that comprises of an exit handler and entry handler. The state S1 is an intermediate state, also comprising an exit handler and entry handler. The state Sn represents any other states that the FSM400may have along with its exit handler and entry handler. If the message is a timer expiry message, the default timer handler450is invoked.

FSM Transitions

FIG. 5-8andFIG. 10-11are state diagrams that illustrate many configurations of the FSM technology disclosed. Among the diagrams, various combinations of messages, transitions and exit/entry handlers are illustrated.

FIG. 5is a representation of example state-message duple of an FSM500. An FSM can include a special state called the initial state515, which the FSM enters upon initialization or start-up. When the FSM is initialized, a message called MSG_INIT510is enqueued into a message queue415and sent to the FSM500. Upon receipt of this message, the system calls exit handler of the initial state, which contains the instructions for exiting the initial state.

FIG. 6illustrates example dedicated handlers of an FSM600. These state handlers are used to process an incoming message610, received while in a state and when exiting or entering it. One or more exit handlers and entry handlers model the behaviors related to the states of a FSM. The so-called default exit handler655or default entry handler625are methods, function-calls or similar blocks of code that contain instructions to describe and implement the state behaviors. Invoking these handlers serves invokes the state behaviors. A state can also be modeled to handle specific messages by invoking either the so-called dedicated exit handler645or dedicated entry handler635or both. Messages X and Y cause a transition to enter the state; hence they only have a dedicated entry handler. Messages Z and W cause a transition to exit the state; hence they only have a dedicated exit handler.

FIG. 7illustrates a self-transition within a state. This illustration includes multiple self-transitions responsive to various messages. Messages X, Y, Z, and W have their own pairs of dedicated exit and entry handlers. No default handlers are illustrated for this state.

FIG. 8illustrates implementations of state handlers responsive to various transitions. Messages X and Y cause a self-transition. In this example, they have pairs of dedicated exit and entry handlers. Messages A and B cause a transition into the state S0, with dedicated entry handlers. Messages Z and W cause a transition that exits state S0; they have dedicated exit handlers. Message C causes a transition to enter the state; in this example, Message C does not have a dedicated handler. Thus, the default entry handler is invoked for this message. Message D causes a self-transition to the state, without dedicated handlers; thus, the default exit and default entry handlers are invoked for this message. Message E causes a transition to exit the state and does not have any dedicated handler, so the default exit handler is invoked. Thus,FIG. 8provides examples of handler configurations for three transition types.

FIG. 23shows the state representation of an FSM when a dedicated exit and entry handler is defined for the timer expiry message. The technology disclosed herein can be used to define a dedicated exit or entry handler for the initialization message MSG_INIT or for a timer expiry message MSG_TIMER_EXPIRED. The latter can be sent to the FSM when a timer expires. If a timer expires, the timer expiry message “MSG_TIMER_EXPIRED” can be generated and enqueued. This message can be later dispatched to the current state. If a state possesses a dedicated exit or entry handlers for this timer expiration message, those handlers can be invoked.

ApplyingFIG. 4toFIG. 8, a default timer is set, which may generate for a timer expiry message (abbreviated MSG_TIMER_EXPIRED). The default timer handler450is invoked where the timer expiry message is received by a state. The timer handler implementation can include retrieving the name of the current state and checking the status of a guard timer.

FIGS. 9A and 9Bare high-level flow charts900A and900B of an example FSM thread operation. Upon receiving a message910,912, the dedicated exit handler is invoked first914,916, and if the message does not have a dedicated exit handler956, then the default exit handler is invoked958. The exit handler specifies whether a state change takes place918to a next state or the same state remains. In either case, the dedicated entry handler of the next state is invoked922,924and if the state does not have a dedicated entry handler964, the default entry handler is invoked966. If there is no dedicated or default entry handler, no entry handler is invoked928. If the dequeued message is a timer expiry message928, the default timer handler is invoked930as well. If any of the handlers return a “True” value920,926,932, then the FSM thread is completed934. States possess at least one handler, whether it is a default or dedicated handler and whether it is an exit or entry handler.

FIG. 10is an example of an FSM1000with self-transition. In one implementation, the user can define a self-transition in the user class by defining an exit handler1025followed by an entry handler1015of the same state. If a state does not invoke a NextState(.) method that causes transition to a different state, then the message1010causes self-transition back to the same state S0.

A parent FSM can instantiate one or more child FSMs,FIG. 11. The parent FSM and child FSM communicate by exchanging messages but are different and run independently. A child FSM can be instantiated in the exit, entry, and/or timer handlers of the parent FSM. A parent FSM can send a message to its child FSM or can await completion of actions delegated to the child FSM. A child FSM, upon completion, sends a notification message MSG_FSM_COMPLETED to its parent. This notification message alerts the parent FSM to the completion of its child FSM. The MSG_FSM_COMPLETED is processed like any other message by default or dedicated exit/entry handlers.

A child FSM can inherit attributes and behaviors from a pre-existing parent FSM.FIG. 11illustrates a parent FSM1110that instantiates two other child FSMs, namely FSM1120and FSM1125, from a handler of S0 of the parent FSM. For instance, in the parent FSM1, when the exit handler of the initial state S0 is invoked, two other child FSMs, namely FSM2 and FSM3, are instantiated inside the exit handler. The exit handler returns after the child FSMs completed their operations.

A parent FSM can use several procedures or methods to communicate with the child FSM, including an isFSMCompleted(.) method. The isFSMCompleted(.) method is a polling method that checks the Boolean return value of the handlers. A “True” value of the Boolean return value indicates the completion of the child FSM.

In another efficient implementation, which does not use polling, a WaitOnFSMCompletion(.) method waits indefinitely until the child FSM thread is completed. In yet another implementation, an asynchronous non-polling notification message MSG_FSM_COMPLETED can be used to notify the parent FSM of the child FSM completion.

The base class of a child FSM can use routines or subroutines including a default constructor, another constructor, and a default destructor. The constructor of the base class of a child FM sets the current state to the initial state, following which a pointer to the parent FSM is stored.

Test cases implemented using FSMs test certain behaviors through messages, states, or transitions. If the test criteria for behaviors are not satisfied, the test case fails; otherwise it passes. The outcome of a test case, whether passed or failed, is commonly known as a “verdict”. The verdict can be applied through methods defined in the FSM framework. These methods can set a verdict to “Pass” or “Fail” and also check it after FSM completion.

FIGS. 12 and 13illustrate a message sequence chart1200with annotated state information and corresponding FSM diagram1300for an RRC connection establishment.FIG. 12depicts a message sequence chart of RRC connection establishment procedure315at the network emulator125, with state information added.FIG. 13illustrates an FSM that models behaviors described by the message sequence chart inFIG. 12and supports test instrumentation.

When the user equipment127is powered-on, it sends an RRC connection request message1215to network emulator125. The network emulator125sends back an RRC connection setup message1225and starts a timer1223. If the RRC connection setup complete message1235is received, the timer1223is stopped and user equipment127is successfully connected to the network emulator125.

However, if the timer1223expires before the network emulator125receives the RRC connection setup complete message1235, the user equipment127is not connected to the network emulator125and the test case fails.

The RRC connection request message1215includes the identity of the user equipment127and the cause of the connection establishment procedure315. The network emulator125starts the timer1223to define the waiting period for the response of the user equipment127to the RRC connection setup message1225. The procedure fails if the timer1223expires before the network emulator125receives the RRC connection setup complete message1235from the user equipment127.

The RRC connection setup message contains the configuration information for an SRB used for transmitting a message to the traffic generator115. The user equipment127sends an RRC connection setup complete message1235and completes the connection establishment procedure315.

FSM Testing Example

FSMs with the disclosed user-modifiable entry and exit handlers can be used to conduct conformance testing according to criteria established in “RRC Connection Establishment: Success,” TS 36.523-1, section 8.1.2.1, pp. 283-285. Satisfying these criteria is part of the battery of conformance tests applied to test conformance of new LTE systems tests. FSMs can be used to model and test hundreds of conformance testing criteria such as whether user equipment (an SUT) in RRC idle state is able to make an outgoing call and establish an RRC connection, whether an SUT in idle mode can initialize an outgoing call, whether an SUT can transmit an RRC connection request message, whether an SUT can respond to a system simulator (abbreviated SS) or a network emulator transmitting an RRC connection setup message, and whether an SUT is in a RRC connected state.

The test conformance with these criteria, a timer “T300” times these behaviors. Satisfaction of or failure to meet the criteria is reported by a verdict procedure. This timer is the guard timer in the “RRC Connection Establishment: Success” test case drawn from 3GPP standard TS 36.523-1, section 8.1.2.1, pp. 283-285. During conformance testing of new LTE systems, expiration of this guard time would result in a failed test case. The messaging in this sequence chart can be tested using the novel FSM technology disclosed herein. In general test in this standard and elsewhere can be represented in the state diagrams that can, in turn, be implemented using the novel FSM technology disclosed.

InFIG. 13, the incoming message, MSG_INIT1320of the test case is received by the initial state S0 comprising a novel dedicated exit handler. The state S1 is an intermediate state comprising two dedicated exit handlers, whereas the state S2 is the final state comprising only one default entry handler. After MSG_INIT1320is sent to state S0, state S0 waits until it receives a connection request message. When a connection request message is received, state S0 sends a connection setup message, starts a timer, and the current state changes to state S1. If a connection setup complete message is received while in state S1, the current state changes to state S2. Upon entering state S2, the FSM sets the verdict to “Pass” and stops the timer

This test case is implemented using the FSM1300, in which the initial intermediate, and final states are represented by the idle state1330, connecting state1335, and connected state1339, respectively. The FSM1300comprises handlers such as dedicated exit handler1323and dedicated entry handler1328and messages like MSG_INT, MSG_TIMER_EXPIRED, RRC connection request message1215, RRC connection setup message1225, and RRC connection setup complete message1235for state initialization and transition. The FSM1300also includes a timer T300.

The dedicated exit handler1323or other handler handles the duple of idle state1330and RRC connection request message1215. The connecting state1335has two dedicated exit handlers for RRC connection setup complete message1235and a MSG_TIMER_EXPIRED message. The connected state1339uses a default entry handler1328.

When an RRC connection request message1215is received while the FSM is in idle state1330, it sends back an RRC connection setup message1225to the user equipment127. Then, the idle state1330transits to connecting state1335and starts the timer1223. When RRC connection setup complete message1235is received, the connecting state1335transits to the connected state1339. Upon entering the connected state1339, the FSM1300sets the verdict to “Pass” and stops the timer1223. The entry handler of the connected state1339returns a “True” value to signal the completion FSM1300and subsequent execution of connection establishment test case.

FIG. 14is a class diagram1400illustrating the inheritance of a base class by a user class and division of code between them. The base class1435allows easy modeling, seamless implementation, and execution of test cases. It can be delivered as executable code that is not user modified. The user class can be delivered as user editable source code or can be authored by users.

The class diagram1400includes the base class1435. In some implementations, it uses another class CState1425to create state objects. In other implementations, CState1425may be combined with base class1435. The base class1435can have one or more instances of CState1425class where the base class represents the FSM and the CState class represents the FSM's states. Each class has its own attributes and methods. The class CState1425can have several FSM elements including states, state names, and default or dedicated exit, entry, and/or timer handlers.

The base class1435can have several procedures or functions including methods for starting the FSM1300, sending the MSG_INIT1320to the message queue1315, starting, restarting, or stopping the timer1223. In addition, it can also possess methods for setting the verdict to either “Pass” or “Fail” or some other verdict value. In addition, to “Pass” or “Fail,” the verdict can be accompanied by other parameters or information.

The connection establishment procedure test case315can be implemented as FSM1300. The FSM1300can inherit from the base class1435a “GetFSMInfoTable(.)” method and a data structure “FSM_INFO_ENTRY. The user class1445can override the GetFSMInfoTable(.) method. This method when invoked by the base class1435retrieves the information of user class1445that can include state names, state handlers, and message alphabets. The GetFSMInfoTable(.) method returns a pointer to the array that contains information important for the implementation of FSM1300. Entries in this array are represented by the data structure “FSM_INFO_ENTRY”.

The base class1435can have several procedures or functions that facilitate the construction of the user class1445including use of MACROs that create several FSM elements including states, state handlers, timers, messages alphabets, and state transitions. In some implementations, the base classes delivered as executable code that is not user modifiable. Delivery of the base class as executable effectively conceals from the user the details of base class implementation and operation.

The user class1445can implement several test cases such as the connection establishment procedure test case315. The user classes user modifiable to implement tests, such as the example test. In one implementation, the procedure has three states S0, S1, and S2 corresponding to the idle state1330, connecting state1335, and connected state1339in FSM1300respectively. In FSM1300, default and dedicated exit and entry handlers may be defined for each state such as dedicated exit handler1323in state S0, dedicated exit handler1324, dedicated exit handler1325in state S1, and default entry handler1328in state S2.

The base class1435performs several actions such as invoking the GetFSMInfoTable(.) method to retrieve all the user class1445information, receiving the MSG_INIT1320and enqueuing it in the message queue1315for further processing, invoking exit handler1323and entry handler1328of the current state, and invoking default timer handler450if the MSG_TIMER_EXPIRED is dispatched, and implementing methods for starting, restarting, and/or stopping the timer1223and setting or getting a verdict.

The user class1445performs several actions such as declaring message alphabets, idle state1330, connecting state1335, and connected state1339. Additionally, the user class1445can declare and implement exit handler1323and entry handler1328for each state. Furthermore, the default timer handler450can also be declared for timer expiry messages.

The base class1435can possess several routines and subroutines including a default constructor and a default destructor. In the constructor, three messages alphabets MSG_INIT, MSG_TIMER_EXPIRED, and MSG_FSM_COMPLETED are added automatically to the message alphabets. The MSG_INIT allows the FSM1300to receive the initialization message, MSG_INIT1320when FSM is first instantiated or just after entering the idle state1330. The MSG_TIMER_EXPIRED allows the FSM1300to receive a timer expiry message whenever the timer1223expires. MSG_FSM_COMPLETED is received by the parent FSM from a child FSM if the parent FSM instantiates a child FSM.

In the destructor, an FSM thread exit event is signaled and FSM is completed. The destructor waits until the thread is terminated before stopping, removing and flushing all other resources such as timer1223resources and message queue1315contents.

Since the FSM1300can be used for modeling a test case, the initial and default verdict for the test case is “Incomplete” until it is set otherwise by the user class1445. The return values of exit handler1323and entry handler1328are set to be “False” by default indicating that the FSM1300is not completed.

The FSM1300can be started or invoked in a variety of ways, including calling a Run(.) method. This method starts by creating two events an exit event and a queue event. The first is used for signaling to exit the FSM thread while the latter is used for signaling that the message queue1315has awaiting messages. Inside the Run(.) method, the GetFSMInfoTable(.) method is invoked by the base class1435. Depending on the number of states that exists in the user class1445, one or more state objects can be created. Each state object can hold information such as state name and state handlers. All message alphabets retrieved from the user class1445are saved.

The MSG_INIT1320can be sent to the message queue1315by the base class. The base class further initializes the current state to the idle state1330. The FSM thread is then created and started. The dispatcher dequeues the MSG_INIT1320and dispatches it to the current state.

In some implementations, the FSM1300can be stopped and re-run to start all over again by calling the Stop(.) method and Run(.) method respectively. In this implementation, the message queue1315is flushed and FSM1300starts from the idle state1330.

The SendMessage(.) method can be used to send the MSG_INIT1320. This method first checks whether a message is among the FSM message alphabets and if so, the message is enqueued for further processing. However, if the message is not among the FSM message alphabets, it is discarded. Whenever a message is queued in the message queue1315, the queue event is signaled to indicate that there are messages waiting in the queue. Since the message queue1315is a shared resource between this method and the FSM thread, it is locked before it is used for enqueuing and finally unlocked after the method is finished using it.

The current state of the FSM can be changed to next state by using the NextState(.) method. The state change takes effect in the exit handler1323(whether default or dedicated handler). If a state change occurs in the entry handler1328or timer handler450, can be considered void and not given effect. In yet another implementation, the current state of the FSM1300can be retrieved by using the GetCurrentState(.) method.

The Run(.) method creates a thread, which is used to start the FSM1300. This thread has an infinite loop, inside which several events such as the message queue1315receiving a new message, indication of termination of FSM1300, and receiving WM_TIMER message from the OS as a result of an expiry of the timer1223can occur.

If a new message is enqueued, then the thread dequeues this new message and forwards it to the exit handlers and entry handlers of the current state and the next state respectively. In addition, if the dequeued message is MSG_TIMER_EXPIRED, which indicates that the timer1223has expired, the default timer handler450is invoked as well. If the completion of an event requires termination of the FSM, the thread exits the infinite loop and terminates itself.

The timer1223can expire when a timeout value elapses. When this event occurs, OS sends a WM_TIMER message to the FSM thread, and the thread forwards this message to a callback static method called TimerExpired(.). In the latter method, the WM_TIMER message is replaced by MSG_TIMER_EXPIRED message and queued into the FSM message queue1315for further processing.

If the event is message queue event, the thread dequeues the Head-Of-Line (abbreviated HOL) message from the message queue1315and starts dispatching it to the exit handlers and entry handlers of the current and next state. If the dequeued message is a MSG_TIMER_EXPIRED, the default timer handler450is invoked as well.

The handler can return several variables including a Boolean return value, which signals whether the FSM1300is completed or not. If this Boolean value is “True”, then this signals FSM1300completion and subsequent thread termination. When the FSM thread terminates, all FSM messages are removed from the message queue1315.

The algorithm used for scheduling the invocation of default exit and entry handlers has an O (1) time complexity. The dedicated exit and entry handlers are stored in a balanced binary tree data structure and hence the algorithm used for scheduling their invocation has an O (log N) time complexity, where N is the number of specified dedicated handlers.

Intrinsic support in the base class for a verdict function is useful for test case modeling and execution. The base class1435allows this through several methods including SetVerdict(.) and GetVerdict(.). The verdict can be set to various values including “Pass”, “Fail”, or “Incomplete”.

The base class1435also provides several procedures or methods to start, restart, and/or stop the timer1223. These methods use underlying OS timer methods to identify the timer by its Id. To match the requirements of different protocol stack behaviors where timers are commonly identified using their names, the base class1435can use several unique identifications including “timer name” for modeling the timer1123.

A StartTimer(.) method can be used by the base class1435to start the timer1223. This method starts by stopping the timer1223if it is running and then calls the OS timer method to start a new timer. The OS timer method returns a timer Id. Once the timer is started, timer information such as timer name, timer Id, and timeout values are stored and saved.

A ReStartTimer(.) method can be used by the base class1435to restart the timer1223while it is active. It first retrieves the timer Id for the corresponding timer name, stops the timer and then finally starts it again.

A StopTimer(.) method can be used by the base class1435to stop the timer1223while it is active. It first retrieves the timer Id for the corresponding timer name and then stops it.

Upon receiving a timer expiry message, an FSM can use several procedures or methods such as calling a method called CallbackMethod(.) to initiate its timer expiry thread. This method is invoked by the OS and passes the timer Id as a parameter. Furthermore, this method first retrieves the timer name corresponding to the timer Id and then creates a new message called MSG_TIMER_EXPIRED containing the timer name. This new message is then sent to the message queue1315for further processing.

The user class1445can use several procedures or methods for declaring and implementing dedicated or default exit/entry handler, timer handler, or message alphabets.FIG. 15illustrates declaration of handlers in a user class1445using MACROs. In one implementation, MACROs are used inside the definition of user class1445to define and declare the different handlers. The use of MACROs is to provide ease of use and less effort in writing the code for these handlers.

FIG. 16illustrates implementation of handlers in a user class1600using MACROs. The MACROs are used to implement the handler header. The user can continue to implement the body of these handlers. In each handler, the user can write the exact behavior or instructions that are to be executed through code when this handler is invoked by the base class1435.

FIG. 17illustrates implementation of the overridden method, GetFSMInfoTable(.), in an FSM user class1700. The user class1445can use several procedures or methods to register state-message duples and handlers including, implementing the GetFSMInfoTable(.) method using MACROs in the registration section 2120. Depending on the user design of the FSM and state handlers, the user can use one or more of these MACROs to declare states names, state handlers, and message alphabets.

The Connection Establishment procedure test case is implemented when user class1445is executed by instantiating an object of the user class1445using the default constructor. Following the instantiation, the Run(.) method is called. Then the program waits until the FSM is completed and checks whether the verdict is “Pass” or “Fail”. Finally, object of the user class1445is destructed. An FSM can be stopped after it has started running by calling the Stop(.) method. If a running FSM is stopped, its verdict is set to “Incomplete”.

FIG. 18is the C++ header file of an example user class of the connection establishment procedure test case.FIG. 19is the C++ source file of the example user class of the connection establishment procedure test case.FIG. 20is the C++ code2000for the execution of the Connection Establishment procedure test case. This C++ code2000is the main application. Some modules or codes within the main application can handle instantiating the user class1445object, calling the Run(.) method to run the user class1445object, waiting until the FSM completes by calling the WaitOnFSMCompletion(.) method, checking whether the verdict is “Pass” or “Fail” by calling the GetVerdict(.) method, and destructing the user class1445object.

FIG. 24AandFIG. 24Bshow the FSM model used to describe the voice activity model inFIG. 23. It has the same number of states and its timer expiry events are set to the same steady-state equilibrium values of 3 seconds and 2 seconds to obtain a VAF of 40%. Voice, video, data coders or traffic generators or sinkers can be represented using a continuous-time finite state machine where the main characteristics of the traffic can be described by probabilistic transitions or the time elapsed while being in each state. The finite state machine can be flexibly and efficiently used to describe and model continuous-time finite state machine such as voice, video, data coders or traffic generators. A voice coder can be modeled using a two-state voice activity model as shown inFIG. 24AandFIG. 24Bwhere a human voice speech coder can be characterized by a transition from “SILENT” to “TALK” states and vice-versa also called ON/OFF model. The transition between the two states is not triggered by messages but instead by a probability. As shown inFIG. 24AandFIG. 24B, the transitions between the two states “SILENT” and “TALK” can be in the form of the probabilities a, b, 1-a, and 1-b. For such a voice activity model, the voice activity factor (abbreviated VAF) can be defined to be the steady-state equilibrium of the voice model being in the “TALK” state. VAF is the percentage of time where the voice model is in “TALK” state. Given the probabilistic transitions a and b, the steady-state equilibrium of the voice model can be mathematically computed using the formula VAF of 2/(2+3) or 40% as shown inFIG. 24AandFIG. 24B.FIG. 25shows the C++ code for the voice activity model showed inFIG. 23.

In other examples, the technology disclosed can also be used to implement other test cases based on the compliance criteria in 3GPP standard TS 36.523-1 and similar standards documents. For example, the novel FSM architecture can be used to implement a test case corresponding to section 6.1.1, p. 13 that tests the idle mode operations in a pure E-UTRAN environment public land mobile network (PLMN). When an SUT is switched on, it attempts to make contact with a PLMN. The SUT looks for a suitable cell of the chosen PLMN and chooses that cell to provide available services, and tunes to its control channel. The SUT registers its presence in the registration area by means of a location registration (abbreviated LR). If the SUT is unable to find a suitable cell to camp on, or the SIM is not inserted, or if it receives certain responses to an LR request (e.g., “illegal SUT”), it attempts to camp on a cell irrespective of the PLMN identity, and enters a “limited service” state in which it can only attempt to make emergency calls. The user equipment has several states in the PLMN selection process such as the trying registered PLMN state, the not on PLMN state, the trying PLMN state, and the no SIM state. Message-sate duples and entry/exit handlers can be used to implement behaviors of the states or to test or record system conditions in various states.

The states and transitions of this test can be implemented using the FSM technology disclosed herein. The behaviors of these states can be modeled using FSM state handlers. The communication between the SUT and PLMN can be established through message alphabets, message queue, and dispatcher of the FSM. The FSM timer can time all the test case operations and the operation results can be reported using FSM verdict.

Other examples of test implementation using the FSM technology disclosed include: medium access control mapping between logical and transport channels by the SUT (3GPP standard TS 36.523-1, section 7.1.1, p. 50), random access channel (RACH) selection by the SUT drawn from 3GPP standard TS 36.523-1, section 7.1.2, p. 54, and downlink-shared channel (abbreviated DL-SCH) processing by the SUT drawn from 3GPP standard TS 36.523-1, section 7.1.3, p. 85. Some complex tests implementations can benefit from parent FSMs invoking child FSMs, as described next.

FIG. 21is a high level flow chart2100of one implementation of the technology disclosed herein. Other implementations may perform the steps in different orders and/or with different or additional steps than the ones illustrated inFIG. 21. For convenience, this flowchart is described with reference to the system that carries out a method. The system is not necessarily part of the method.

At step2105, the system initializes a test case application. Upon execution, the test case creates a user class object that inherits the public methods defined in the base class at step2110.

Upon running the FSM, the system registers the message alphabets and state-message duple using the MACROs defined in the user class. This step2120is referred to as the registration section.

The handlers registered in the user class are invoked by the base class at step2130to implement the instructions outlined in the handler. In this implementation, the handler invoked by the base class is an exit handler and contains the instructions for next state transition.

The timer handler implementation includes retrieving the name of the current state. At step2140, a timer method using a guard timer, times the state transition executed by the exit handler at step2150which represents state transition from state to another. Step2160constitutes a verdict method that is used to report results of the FSM operations. If the guard timer expires, it results in a verdict method declaring a “Fail” value at step2160.

FIG. 22is a block diagram of an example computer system, according to one implementation. Computer system2210typically includes at least one processor2214that communicates with a number of peripheral devices via bus subsystem2212. These peripheral devices may include a storage subsystem2224including, for example, memory devices and a file storage subsystem, user interface input devices2222, user interface output devices2220, and a network interface subsystem2216. The input and output devices allow user interaction with computer system2210. Network interface subsystem2216provides an interface to outside networks, including an interface to network emulator125, and is coupled via network emulator125to corresponding interface devices in other computer systems.

Storage subsystem2224stores programming and data constructs that provide the functionality of some or all of the modules and methods described herein. These software modules are generally executed by processor2214alone or in combination with other processors.

Memory2226used in the storage subsystem can include a number of memories including a main random access memory (RAM)2230for storage of instructions and data during program execution and a read only memory (ROM)2232in which fixed instructions are stored. A file storage subsystem2228can provide persistent storage for program and data files, and may include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media cartridges. The modules implementing the functionality of certain implementations may be stored by file storage subsystem2228in the storage subsystem2224, or in other machines accessible by the processor.

Bus subsystem2212provides a mechanism for letting the various components and subsystems of computer system2210communicate with each other as intended. Although bus subsystem2212is shown schematically as a single bus, alternative implementations of the bus subsystem may use multiple busses.

Computer system2210can be of varying types including a workstation, server, computing cluster, blade server, server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of computer system2210depicted inFIG. 22is intended only as one example. Many other configurations of computer system2210are possible having more or fewer components than the computer system depicted inFIG. 22.

Some Particular Implementations

In one implementation, a method is described that includes implementing a novel architecture of a finite state machine (abbreviated FSM) using code including a built-in base class and a user-modifiable user class. The method can include running the code on a processor. When running, the user-modifiable class inherits from the base class and registers a message alphabet and state-message duples. The method further includes defining exit and/or entry handlers for state-message duples including at least one exit handler that has a next-state specification connecting a first state that is exited in response to particular message in the message alphabet and a second state that is entered from the first state. The method can further include inheriting a test timer method used to set time limits on execution of operations during a test and a verdict method used to record test results.

This method and other implementations of the technology disclosed can each optionally include one or more of the following features and/or features described in connection with additional methods disclosed. In the interest of conciseness, the combinations of features disclosed in this application are not individually enumerated and are not repeated with each base set of features. The reader will understand how features identified in this section can readily be combined with sets of base features identified as implementations.

The method can include storing in computer readable memory as executable code the base class that is not user-modifiable and the user class as user-modifiable source code. It can further include the base class concealing operating details of the FSM, invoking the entry and exit handlers and the user class registering the state-message pair specifications.

The method can include the entry and exit handlers specifying behavior of the FSM. It can include the user class defining one or more default entry or exit handlers that apply to multiple state-message duples when dedicated handlers are not defined for particular state-message duples. It can further include the base class comprising a virtual method table that instantiates objects corresponding to the entry and exit handlers defined in the user class and being overridden by the user class to retrieve the user class information.

The method can further include the base class comprising the test timer method that instantiates objects corresponding to particular test timer methods invoked by the user class and a verdict method that instantiates objects corresponding to the verdict methods invoked by the user class. It can further include the verdict method recording a result specified in any of the entry and exit handlers and automatically recording the state of the FSM and a time at which the result is generated.

The method can include providing a user-modifiable test case that tests compliance with an industry standard for its enhancement. It can further include the test case implementing one or more actors that exercise the industry standard, interacting with one or more systems under test (abbreviated SUT) to be tested against the industry standard, and invoking the FSM after transmitting the communications channel parameters to the channel emulator by its program.

The method can include the test case program that invokes the FSM specifying communications channel parameters of a channel emulator that is coupled to the SUT, the base class transmitting the communications channel parameters to the channel emulator, and the entry or exit handler setting communications channel parameters of a channel emulator that is coupled to the SUT. It can further include executing the processor executable code in any programming language and the base class implementing a state tracker that keeps track of a current state of one or more FSMs during a test.

Other implementations may include a non-transitory computer readable storage medium storing instructions executable by a processor to perform any of the methods described above. Yet another implementation may include a system including memory and one or more processors operable to execute instructions, stored in the memory, to perform any of the methods described above.