Abstract:
Methods, devices and systems for synchronizing the execution of tests between or among multi-sided test components in a wireless environment are described. Multiple flows are initiated substantially simultaneously. Execution is then paused in at least one of the flows. The paused flow sends a request for permission to resume execution to a sync servlet. The sync servlet does not respond affirmatively to the request until an appropriate synchronization signal is received from an unpaused flow, and then only after a desired test component in the unpaused flow has been executed. The paused flow is resumed upon receiving the appropriate synchronization signal from the sync servlet.

Description:
RELATED PATENT APPLICATION  
       [0001]     U.S. patent application Ser. No. ______ filed on even date herewith to Bingham et al. entitled “Multi-Sided Sharing of Dynamic Data in a Wireless Environment” having Agilent Invention Disclosure No. 10051091 and assigned to Agilent Technologies is hereby incorporated by reference herein in its entirety. 
     
    
     FIELD OF THE INVENTION  
       [0002]     The present invention relates to an apparatus and method for synchronizing flows between and among multi-sided tests, and more particularly to an apparatus and method for testing one or more networks and synchronizing the execution of multiple flows in multi-sided tests.  
       BACKGROUND  
       [0003]     The term ‘test software’ is used to describe a software product whose primary function is to examine how a system is functioning. The test software collects its input from agents, which are individual members of a system being tested.  
         [0004]     Some examples where test software is employed include mobile or wireless phone networks and other types of telecommunication networks or systems, river water quality monitors and automobile performance test systems. Networks or systems are often geographically and/or temporally distributed. The test software for such systems must collect data from different locations or at different times to test such a system.  
         [0005]     Despite the many advances that have been made in the field of wireless network testing, certain problems persist. These problems include an inability to synchronize the execution of test flows between or among flows or sides in an open manner that employs common standards, an inability to synchronize execution between flows (or virtual threads of execution) except when using a proprietary interface, and an inability to test wireless services that depend on orders of execution between flows unless a proprietary interface is included in the system architecture.  
         [0006]     Agilent&#39;s Wireless QOS Manager (WQM) is capable of running multi-step, multi-sided tests. However, the SIGOS Site application with which WQM currently integrates is a proprietary interface. In consequence, data being shuttled between one flow and another (or from one side to another) must first flow through the proprietary SIGOS Site infrastructure. A system being tested that does not contain a SIGOS Site infrastructure will not permit synchronization between or among flows or sides.  
         [0007]     What is desired is a Wireless QOS Manager having the ability to synchronize test execution between or among sides and/or flows, whether or not a SIGOS or other proprietary site infrastructure is being used.  
         [0008]     The following publication and patent application, each hereby incorporated by reference herein in its respective entirety, disclose various prior art teachings concerning WQMs: 
        Agilent Technologies, Inc. Information Sheet entitled “Agilent OSS QoS Manager” viewable at http://we.home.agilent.com/cgi-bin/bvpub/agilent/Product/cp_Product.jsp?OID=536882909&amp;NAV_ID=-536885380.536882909.00&amp;LANGUAGE_CODE=enq&amp;COUNTRY_CODE=ZZ&amp;CT=PRODUCT&amp;JPID=/comms/firehunter     U.S. patent application Ser. No. 10/736,835 entitled “Sequential Coordination of Test Execution and Dynamic Data” to Bingham        
 
         [0011]     As those skilled in the art will appreciate after having read and understood the present Summary, Figures, Detailed Description and Claims, at least some of the devices, systems and methods disclosed in the foregoing publication and patent application may be modified advantageously in accordance with the teachings of the present invention.  
       SUMMARY OF THE INVENTION  
       [0012]     Methods, devices and systems for synchronizing the execution of multi-sided tests between or among multi-sided test components in a communication or test environment, such as a wireless telecommunication environment, are disclosed. Multiple flows in a system such as a wireless test configuration are initiated substantially simultaneously. Execution is then paused in at least one of the flows. From the flow not paused, a synchronization signal is sent to a sync servlet. The paused flow requests permission from the sync servlet to resume flow. The paused flow is resumed upon receiving a synchronization signal from the sync servlet.  
         [0013]     The present invention includes within its scope the methods of synchronizing described above, hereinbelow and in the Figures, as well as systems and test agents capable of carrying out such methods.  
         [0014]     Various embodiments of the present invention may be characterized by one or more of the following aspects:  
         [0015]     (a) Synchronization between or among multiple flows in a wireless service test is permitted, even if a proprietary interface is not employed in the system being tested;  
         [0016]     (b) A sync servlet permits a flow to pause execution (await sync), while waiting for a go ahead message from another flow (send sync), thereby permitting wireless services to be tested that require such functionality;  
         [0017]     (c) A sync servlet permits control of execution between or among flows (or virtual threads of execution);  
         [0018]     (d) An open common standard for data transfer such as HTTP, HTTPS, FTP, may be employed to effect the synchronization of test execution between or among multiple flows and/or between multiple third-party applications;  
         [0019]     (e) Data may be securely transferred using an HTTPS protocol;  
         [0020]     (f) Timeouts may be configured as required;  
         [0021]     (g) The location of a sync servlet may be changed, which is useful for configurations having firewalls;  
         [0022]     (h) Sync servlets may be run on any Wireless QOS agent or on any web server that supports servlets;  
         [0023]     (i) If a synchronization event has lived beyond a configurable time, it is automatically deleted.  
         [0024]     The foregoing and other aspects and advantages of various embodiments of the present invention will become apparent and more readily appreciated after the detailed description of the preferred embodiments, the drawings and claims have been read and understood  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0025]      FIG. 1  illustrates one embodiment of the present invention;  
         [0026]      FIG. 2  illustrates another embodiment of the present invention; and  
         [0027]      FIG. 3  illustrates yet another embodiment of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0028]      FIG. 1  shows one embodiment of the present invention, where flows A and B in a multi-sided test system are synchronized using sync servlets  20 ,  30  and  40  located, respectively, in Active Test Probe  50  corresponding to Flow A, Active Test Controller  60 , and Active Test Probe  70  corresponding to Flow B.  
         [0029]     In a preferred embodiment of the present invention, each of Active Test Probe  50 , Active Test Controller  60  and Active Test Probe  70  is an individual agent in system  10 , although other configurations are possible and fall within the scope of the present invention. Note that sync servlets  20  and  40  in  FIG. 1  are not employed in the example given below, but may be employed in other configurations or flows not disclosed explicitly herein. Depending on the system configuration at hand, the appropriate destination for a particular signal or message, such as an “Await Synch” or “Send Synch” signal, may or may not need to be specified, although in a preferred embodiment of the present invention such a destination must be specified as indicated in the schemata set forth below. For example, the “Send Sync” signal originating in Flow B of  FIG. 1  may be specified as having an ultimate destination in Flow B, and the “Await Synch” signal originating in Flow A may be specified as having an ultimate destination in sync servlet  30 .  
         [0030]     In  FIG. 1 , Flow A is represented by Test Component  1 , Test Component  2  (“Await Sync”) and Test Component.  3  in Active Test Probe  50 . Flow B is represented by Test Component  1 , Test Component  2  (“Send Sync”) and Test Component  3  in Active Test Probe  70 . First and second flows A and B are initiated substantially simultaneously. Execution of Flow A pauses upon Flow A having proceeded to the second “Await Sync” test component or event. Execution of the “Await Sync” test component in Flow A causes a message or instruction to be sent or posted to sync servlet  30  stating that Flow A awaits the receipt of a “Send Sync” signal or message from Flow B. The amount of time the “Await Sync” test component in Flow A will await receipt of a “Send Sync” signal may be a configurable timeout. When Flow B has proceeded to the “Send Sync” test component, a “Send Sync” signal is posted to sync servlet  30 . The next time Flow A requests permission to resume execution (i.e., “Has a “Send Sync” signal been received from Flow B?”), the “Send Sync” signal has already been received by sync servlet  30 , and Flow A receives permission to resume execution by running the next step (i.e., execution of Test Component  3  now occurs).  
         [0031]     By way of illustrative example only and with no intention of limiting the scope of the present invention, FlowA:test 1  in  FIG. 1  may be a test that navigates to a webpage requesting authentication information for logging into a 802.11 wifi network, and having the bill and login information sent to a cell phone account. FlowB:test 1  then receives the confirmation message on the cell phone with the username and password. FlowA:test 3  next uses the username and password to login to the wifi network. FlowA:test 3  cannot login before FlowB:test 1  has received proper authentication information. The FlowA:await sync causes execution to pause until FlowB:send sync has been sent, at which point execution continues. As will now be appreciated, send sync and await sync functionalities may be implemented in multiple flows to allow more complicated multi-sided and multi-flow networks or systems to be tested.  
         [0032]     Note that the embodiment of the present invention shown in  FIG. 1  is specifically adapted for use in a wireless service test setting such as a wifi login application, although numerous other applications are contemplated and nevertheless fall within the scope of the present invention.  
         [0033]     Continuing to refer to  FIG. 1 , each of sync servlets  20 ,  30  and  40  may be a multi-threaded program running on any of agents  50 ,  60  and  70  in system  10 . Each agent should possess sync servlet functionality, even though such functionality may not be used in respect of a given servlet according to a particular system configuration. For example, sync servlet functionality may be imparted to agents  60  and  70  and not  50 , while still permitting the testing procedures described above to occur. In an optimal system configuration, however, each agent possesses dynamic content servlet functionality.  
         [0034]     Tests may be invoked by a caller located remotely from any of the agents of system  10  if the caller is connected to a network to which such agents are connected, or to which one of such agents is connected. Such a network may be any network, such as the internet. In one embodiment of the present invention, agents  50 ,  60  and  70  may be computers and/or server.  
         [0035]     According to one embodiment of the present invention, dynamic content servlet functionality is added to a sync=servlet. While dynamic content servlet functionality differs from that of a sync servlet, dynamic content servlet functionality may generally be piggybacked onto a sync servlet to avoid running multiple servlets, as shown in  FIG. 1 .  
         [0036]     In one embodiment of the present invention, test descriptions or requests to synchronize may be incorporated into a predefined XML schema (or even plain text) in either an HTTP post or an HTTPS post. HTTP (Hyper Text Transfer Protocol) is a generic message transport standard, or a standardized way of communicating over the internet. An HTTP Get is what a web browser does; it only receives data. But an HTTP Post allows sending of a body of data and also receipt of an answer.  
         [0037]     The body of data sent by an HTTP Post is organized via the XML schema. XML (Extensible Markup Language) is a standardized way of encoding data, or in other words, genericizing the data, enabling definition, transmission, validation, and interpretation of the data between applications.  
         [0038]     Interface design details implementing one embodiment of the invention illustrated in  FIG. 1  are set forth below in Example 1, where text representations of sync servlet input schema are shown. While the sync servlet implementation set forth hereinbelow is written in Java, such functionality may be implemented in any suitable programming language, or run under any web server that supports servlets. Other standards-based servers and protocols may also be used to implement sync-servlets. Sync servlets may accept HTTP posts with a defined XML encoded body only to enter or get data. Using such a functionality, a sync object contained by each test running in a given flow may then post data to or get data from the sync servlet. The sync servlet may be posted to using HTTP or HTTPS (secure socket layer) to ensure security.  
       EXAMPLE 1  
     Wqm Sync Servlet Interface Design  
       [0000]     (A) Input and Output Interface to the Sync/Dynamic Content Servlet  
         [0000]     (1) Brief Description  
         [0039]     The Dynamic Data Servlet will run on any QOSM WQM agent. It provides a data store for dynamic data of running tests and processes requests to “add”, “remove” or “get” dynamic data for a specific uniqueid, or to “delete” all data for that uniqueid.  
         [0000]     (2) Dynamic Data Design related assumptions  
         [0000]    
       
         
           
              The following attributes are found in the Wireless section of fh.conf. They may be overwritten by hidden (fh prefix) attributes in the Wireless Chain Test itself. 
            DcsLocation—-(String) default is the same as the Host (ATC) attribute.     DcsPort—-(Integer) default is 16716     DcsTimeout—-(Integer) default  0  (look once, no waiting)     DcsLookup—-(Boolean) default true (check DCS if not in local store)    
         
              The following must be available in the flow level data stores: 
            DcsLocation     DcsPort     DcsTimeout     DcsLookup     ChainId-(String) a unique id for this test execution     FlowId-(String) the name of the current flow,    
         
              A DcsControl test component will be available to change the values of the Dcs* attributes described above  
              The DcsPurgeTime entry in the Wireless section of fh.conf will define the age at which to purge old data in DCS (in hours) 
 
 (3) Technical Requirements 
 
              Ability to run a servlet on the QOSM agent.  
              Defined XML interface for the HTTP Request and Response.  
           
         
       
     
         [0056]     (4) Definition or Design Description 
        The interface to the servlet will be an HTTP Post.     Example url!         http://&lt;DcsLocation&gt;:&lt;DcsPort—&gt;/DynamicContentServlet—    The body of the post will be XML. The XML interface is defined herein.     The behavior of the DCS mimics a Hashtable: 
            It is not legal to store a null key or value, a NullPointerException will result.     A “put” returns the value previously stored (or null if no value was found)     A “get” returns the value stored (or null if no value was found)     A “remove” returns the value previously stored (or null if no value was found)    
            A “delete” removes all data for a unique id, and returns all values previously stored.     The DCS has some special behavior when storing “Sync” datum is stored. 
            The name of the Sync datum is “dcs_sync&lt;fromFlow&gt;:&lt;toFlow&gt;”    The value stored for a Sync type datum is an integer.     If a value already exists for this datum, it is incremented by another “put”    The value for this datum is decremented by a “get”, if the resulting value is 0, the datum is removed.    
            Purging of old data not properly removed at the end of test execution is carried out. 
            The DcsPurgeTime value found in fh.conf is used as the age to delete old test data, with a default of 24 hours.     The frequency of checking for old data to purge is based on the aging value ( 1/10th the value). 
 
 (5) SITE Dependencies 
   
            TDL code that can call the DynamicContentServlet—,wait for the response and accept the output. 
 
 (6) Generating and parsing the XML in Tcl and Java 
    Description: 
            Learning curve on parsing XML    
            Resolution/Progress: 
            Learning the Document Object Model in jdk1.4.1 to parse XML     Utilizing procedures already present in Tdl for handling XML 
 
 (7) Building the outgoing XML in java 
   
            Description: 
            Learning curve on parsing XML    
            Resolution/Progress: 
            Learning the Document Object Model in jdk1.4.1 to parse XML 
 
 Assumptions 
   
            The XML sent must be in the correct form or will not be validated. 
 
 Technical Requirements 
    The interface is an HTTP Post that contains an XML body as the input to the servlet.     The output is the Post response, also in XML format.        
 
         [0087]     (B) Definition or Design Description  
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         Input “put” Request Syntax                &lt;DcsRequest&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; put &lt;/Action&gt;           &lt;DynamicData&gt; (can contain multiple Datum tags)                &lt;Datum&gt;                &lt;Name&gt; datumName &lt;/Name&gt;           &lt;Type&gt; datumType &lt;/Type&gt;           &lt;Value&gt; datumValue &lt;/Value&gt;                &lt;/Datum&gt;                &lt;/DynamicData   &gt;                &lt;/DcsRequest&gt;            Input “put” Sync Request Syntax                &lt;DcsRequest&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; put &lt;/Action&gt;           &lt;DynamicData&gt;                &lt;Datum&gt;                &lt;Name&gt; dcs_sync:&lt;fromFlow&gt;:&lt;toFlow&gt; &lt;/Name&gt;           &lt;Type&gt; Integer &lt;/Type&gt;           &lt;Value&gt; 1 &lt;/Value&gt;                &lt;/Datum&gt;                &lt;/DynamicData   &gt;                &lt;/DcsRequest&gt;            Input “get” Request Syntax                &lt;DcsRequest&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; get &lt;/Action&gt;           &lt;Timeout&gt; 60 &lt;/Timeout&gt;           &lt;DynamicData&gt; (can contain multiple Datum tags)                &lt;Datum&gt;                &lt;Name&gt; datumName &lt;/Name&gt;                &lt;/Datum&gt;                &lt;/DynamicData   &gt;                &lt;/DcsRequest&gt;            Input “get” Sync Request Syntax                &lt;DcsRequest&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; get &lt;/Action&gt;           &lt;DynamicData&gt;                &lt;Datum&gt;                &lt;Name&gt; dcs_sync:&lt;fromFlow&gt;:&lt;toFlow&gt; &lt;/Name&gt;                &lt;/Datum&gt;                &lt;/DynamicData   &gt;                &lt;/DcsRequest&gt;            Input “remove” Request Syntax                &lt;DcsRequest&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; remove &lt;/Action&gt;           &lt;Timeout&gt; 60 &lt;/Timeout&gt;           &lt;DynamicData&gt; (can contain multiple Datum tags)                &lt;Datum&gt;                &lt;Name&gt; datumName &lt;/Name&gt;                &lt;/Datum&gt;                &lt;/DynamicData   &gt;                &lt;/DcsRequest&gt;            Input “delete” Request Syntax - deletes all data for a particular chain id                &lt;DcsRequest&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; delete &lt;/Action&gt;                &lt;/DcsRequest&gt;            Output Response Syntax if no Exception                &lt;DcsResponse&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; (action from request) &lt;/Action&gt;           &lt;DynamicData&gt; (can contain multiple Datum tags)                &lt;Datum&gt;                &lt;Name&gt; datumName &lt;/Name&gt;           &lt;Type&gt; datumType &lt;/Type&gt;           &lt;Value&gt; datumValue &lt;/Value&gt;                &lt;/Datum&gt;                &lt;/DynamicData   &gt;                &lt;/DcsResponse&gt;            Output Response Syntax if Exception                &lt;DcsResponse&gt;                &lt;UniqueId&gt; chainId &lt;/UniqueId&gt;           &lt;Action&gt; (action from request) &lt;/Action&gt;           &lt;Exception&gt;                &lt;Type&gt; exception type &lt;/Type&gt;           &lt;Message&gt; exception message &lt;/Message&gt;           &lt;StackTrace&gt; exception stack trace &lt;/StackTrace&gt;                &lt;/Exception&gt;                &lt;/DcsResponse&gt;            SQM (Firehunter) Dependencies                None            SITE Dependencies                Ability to do an HTTP Post to the servlet with the correct XML body.           Ability to receive the post response XML.                      
 
         [0088]     The sync servlet schemata set forth above and illustrated in the Figures are only examples. The present invention is not limited to such schemata; other schemata may be used and still fall within the scope of the present invention; provided, however, that appropriate changes are made to the sync servlets and their corresponding test components. Moreover, schemata of the present invention are not limited to XML; other markup (“tag”) languages, or even plain text, and their corresponding test components, may also be employed.  
         [0089]      FIG. 2  shows another embodiment of the present invention, where three Flows A, B and C are initiated concurrently. Each flow runs its first test component. When Flow A runs the “Await Sync” test component, Flow A is caused to pause execution until a “Send Sync” signal or message is received via sync servlet  30  from Flow B. The amount of time the “Await Sync” test component in Flow A will await receipt of a “Send Sync” signal may be a configurable timeout. When Flow B has proceeded to the “Send Sync” test component, a “Send Sync” signal is posted to sync servlet  30 . The next time Flow A requests permission to resume execution (i.e., “Has a “Send Sync” signal been received from Flow B?”), the “Send Sync” signal has already been received by sync servlet  30 , and Flow A receives permission to resume execution by running the next step (i.e., execution of Test Component  3  now occurs). When Flow C reaches its “Send Sync” test component, a “Send Sync” signal or message is sent to Flow B via sync servlet  30 . Once the “Send Sync” signal or message originating from Flow C is received by the “Await Sync” test component in Flow B, Flow B resumes execution to test component  4 .  
         [0090]      FIG. 3  shows another embodiment of the present invention, where firewall  100  is disposed between Active Test controller  60  and Flows A, B and C. Flows A, B and C are initiated concurrently. Each flow runs its first test component. Because firewall  100  is disposed between Active Test Controller  60  and Active Test Probes  50 ,  70  and  80 , however, sync servlet  30  may not be used. Nevertheless, in the embodiment of the present invention shown in  FIG. 3 , any sync servlet may be accessed by any of Active Test Probes  50 ,  70  or  80 , thereby rendering system  10  operative. In  FIG. 3 , sync servlet  20  of Active Test Probe  50  is employed to fulfill such a function.  
         [0091]     When Flow A has proceeded to the “Await Sync” test component, execution of Flow A is paused while awaiting receipt of a “Send Sync” signal or message from Flow B. When Flow B reaches second test component “Send Sync” sometime later, a “Send Sync” signal or message is posted to sync servlet  20  from Flow B. The “Await Sync” test component in Flow A receives the “Send Sync” signal from Flow B via sync servlet  20 , which triggers the resumption of Flow A. Flow B continues execution to the “Await Sync” test, and is paused awaiting receipt of a “Send Sync” signal or message from Flow C. When Flow C reaches its “Send Sync” test component, a “Send Sync” signal or message is sent to Flow B via sync servlet  20 . Once the “Send Sync” signal or message originating from Flow C is received by the “Await Sync” test component in Flow B, Flow B resumes execution to test component  4 .  
         [0092]     Various protocols are described herein, such as HTTP and HTTPS. The present invention in not limited to such protocols, however, and explicitly contemplates the use of other protocols such as FTP (“File Transfer Protocol”), SCP (“Secure Copy Protocol”) and SFTP (“Secure File Transfer Protocol”). Similarly, various networks are described herein, such as a LAN, a wireless network, a wire line network, and the internet. The present invention is not limited to such networks; other networks may be employed and fall within the scope of the present invention.  
         [0093]     The present invention may be employed successfully over a wide range of wireless network types and standards, including, but not limited to, wifi 802.11a, wifi 802.11b, wifi 802.11g, wimax, GSM/GPRS and CDMA, as well as other standards yet to be devised or implemented. Moreover, the present invention is not limited in scope to wireless test applications, and includes within its scope wireline test applications such as Ethernet/LAN and dial-up networks.  
         [0094]     In a preferred embodiment, the synchronization methods of the present invention are carried out using an Agilent QoS (“Quality of Service”) Manager Agent, many aspects of which are described at: http://we.home.agilent.com/cgi-bin/bvpub/agilent/Product/cp_Product.jsp?OID=536882909&amp;NAV_ID=-536885380.536882909.00&amp;LANGUAGE_CODE=eng&amp;COUNTRY_CODE=ZZ&amp;CT=PRODUCT&amp;JPID=/comms/firehunter  
         [0095]     Note that certain aspects and applications of various embodiments of the present invention may be employed in the devices, systems and methods disclosed in U.S. patent application Ser. No. 10/736,835 entitled “Sequential Coordination of Test Execution and Dynamic Data” to Bingham, which is hereby incorporated by reference herein in its entirety.  
         [0096]     Although a few embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes may be made to such embodiments without departing from the scope and spirit of the invention, the scope of which is defined in the appended claims and their equivalents. For example, a system or agent carrying out one or more synchronization methods of the present invention by means of one or more proxies falls within the scope of the present invention.